Unanswered

How can I scroll a website if I use two screens?

(0) Share

Report

Posted on by CU-19081228-9

Hello Everyone.

I am running my PAD flow off a laptop which is connected to a second screen.

If my flow uses the laptop screen as the main browser window, then the flow works fine. I have parts of the flow where the mouse has to move to certain co-ordinates on the screen and click in order to scroll down a webpage and that works.

However, if I close the laptop, the flow obviously transfers itself to the connected monitor. This is where the same steps where the mouse cursor has to click on certain co-ordinates on the screen on the web page seems to break.

So, my questions are:

1 - How else can I scroll down the web page? It seems that pressing the down arrow or the page up or page down keys do nothing and the page refuses to scroll (other websites work fine).

2 - How can I get it so that my steps where the mouse has to scroll to certain co-ordinates and click works on the connected monitor (which is a larger size than the laptop screen)? I have these steps set to reference the screen and not the active window - could that be the reason?

For reference, the website in question is a monitoring service called NewRelic and I am basically trying to take screenshots of specific graphs on a chosen dashboard.

Many thanks for any help.

Regards.

Categories:

Power Automate Desktop

All responses (5)

Answers (0)

Sort by

Pavel_NaNoi 1,074 on at

Like
a
(1)

Report
Copy link

Link copied!
For point 1, A couple of round about ways exist yeah:
An easy option would be to use "Take screenshot of web page" in your case, you'll need to tell it to take a screenshot of the graph UI element if possible, this will scroll down for you. You could also just use "focus text field on webpage" and tell it to focus on a UI Element on the web-page and then screenshot by w/e means you want to get the same effect.
You can also use javascript to simply scroll down the webpage, just by using the "Run javascript on webpage", a guide on it can be found here https://learn.microsoft.com/en-us/power-automate/desktop-flows/how-to/scroll-web-page
Finally the nuclear option would be to just use the "Click image on screen" action and tell it to click on the "down arrow" on the scrollbar.
For point 2, no, if you use screen it uses the pixels of that screen or application, i.e. if laptop is 1920x1080 then you'll only have that many pixel coordinates to work with, if your screen is bigger than that you'll have more pixel coordinates on the screen, this includes application windows, if you have a window that is 600x800 in size, it will use that many pixels, if you make the application bigger it will have more pixels to work with. In other words, to prevent this problem, you'll need to do some math, use the "Get screen resolution" action, take these numbers, divide them by your laptops' screen resolution respectively and then use these factors and multiply them with the coordinates in your mouse click actions, thus no matter what, it should always click in the same "respective" spot.
CU-19081228-9 on at

Like
a
(1)

Report
Copy link

Link copied!

Thank you @Pavel_NaNoi for such a detailed response, and sorry for the late reply.

I've been working on this for the past few days. It's quite the challenge!

For some NewRelic dashboards, I got it to scroll using the 'Run JavaScript on webpage' option, but on a different dashboard that same command simply made Edge display a 'Power Automate is debugging this web browser' message, which I've never seen before, and it didn't scroll.

Similarly, using the 'focus text field on web page' option refused to scroll. It simply said that the text UI element could not be found, even though it was there. I then thought maybe I could use that to fake scroll by focusing on text on the page above the graph I wanted (and so it would scroll to it and the actual graph I wanted would be in view), and then focusing on the graph text itself that I wanted. So, two-step scrolling.

That didn't work, it just highlighted the text instead of the first graph UI element and refused to scroll down to it.

Currently I have managed to get it to scroll by sending a mouse down mouse click event and just moving the mouse cursor to where the graph is, effectively manually clicking and dragging the scrollbar. That seems to work for now.

I've not even started to get my head around point 2 yet, but thank you so much for explaining it, I'll give it a shot when I can.
CU-19081228-9 on at

Like
a
(0)

Report
Copy link

Link copied!

@Pavel_NaNoi I took your advice and used the 'Wait for Image' option, and I then used that step and took an image of the graph I wanted to capture. This action outputs me X and Y co-ordinate values.

I then want to click the menu button on the graph whose image I captured. So, the next action is to click the graph UI element and I went to the advanced options of this action, where it lets you select the X and Y co-ordinates and passed in the X and Y variables from the previous step.

My understanding is that the flow will wait for the graph to appear on screen and click the UI element at the X and Y co-ordinates I gave - is that correct?

The reason I ask about this method is that it seems that the span IDs change each time the web page loads. So that means the flow will click the links sometimes, and then next time it will fail as it cannot be found.

When I tried this 'waiting for image' method, I took a picture of the button, the flow runs and gets the X and Y co-ordinates. But when I then use those to send a mouse click the flow fails and says that the co-ordinates are out of bounds - how is that possible?

Thanks so much!
Pavel_NaNoi 1,074 on at

Like
a
(1)

Report
Copy link

Link copied!

Hmm, can you give me a link to an example graph I can just work with? I'll just send you a code snippet if I get it to work.
CU-19081228-9 on at

Like
a
(1)

Report
Copy link

Link copied!

Thank you so much for taking the time to reply @Pavel_NaNoi.

I may actually have got it working, not sure, just testing the flow now.

Originally I was using the 'Click link on website' action to go through the steps of clicking the menu button on the graph and then clicking the 'Expand' menu option to display it full screen before taking a picture and saving it.

I instead used the recorder to record the actions of clicking the graph menu button and clicking the Expand button and it seems to actually do it as expected. I also updated PAD to the latest version when asked as well - not sure if that was a thing too?

So the NewRelic part of my PAD flow goes like this:

1 - I already have a browser instance running, so a new tab is added to it. This is used to log in to NewRelic with a URL for the specific dashboard I want to take graph pictures of (for this step PAD didn't accept the URL as it was and said 'Syntax error' - probably characters needed to be escaped? I had to convert to a Bitly link and use that).

2 - It then clicks the dashboard category link which contains the graph I want. The graph in question is the very first one in the list.

3 - Each graph has a menu button (shown) and PAD should click that and then click the 'Expand' option.

4 - The graph appears full size, a picture is taken and saved to OneDrive and then the menu is closed.

5 - The next graph category is clicked and the process is repeated before NewRelic is logged out of and the browser tab closed.

I've literally just watched my entire flow complete twice without issue now, so no idea what's happening really or what changed. I'm just glad it works.

Thanks so much for all your kind advice and help.