Unanswered

Sending keys to foreground window inconsistent in unattended mode

(1) Share

Report

Posted on by mbenyishay

I am running a desktop flow called from a cloud flow, which extracts data from a web page, scrolls the web page (it uses page-down keys) and then re-extracts. This is running in unattended mode on a virtual machine. What is very strange is that this works well when I manually trigger the cloud flow (still running unattended) but not when it runs the schedule. I have it logging how many rows it retrieves, which tells me it is often not successfully scrolling down when it runs on the schedule. (Without scrolling, the table initially retrieves the first 100 rows, so if exactly 100 rows are retrieved, I know it is not scrolling).

The workflow:

Load the web page
Focuses the window
Sets focus on the text field on the web page which houses the table (the table has a scrollbar but is not in an iframe).
Send keys ({Next}{Next}{Next}{Next}{Next}).
Extract data from web page.

I am sharing screenshots. I cannot share the web page because it is a private database with sensitive information. Again, when I click "run" on the cloud flow, it seems to work fine. But when I runs on the schedule, it's mostly not scrolling down.

In the screenshot of the log, the highlighted row with the higher number of rows was a successful run where I triggered the cloud flow.

PAD web data extract with foreground selection.png Power BI data extract log.png

Categories:

Power Automate Desktop

I have the same question (0)

All responses (26)

Answers (0)

VJR 7,635 on at

Like (0)

Report

Have you also tried the javascript approach of scrolling the webpage?

https://learn.microsoft.com/en-us/power-automate/desktop-flows/how-to/scroll-web-page

Was this reply helpful? Yes No
momlo 1,527 Super User 2024 Season 1 on at

Like (0)

Report

Hi @mbenyishay
You write it focuses on text field, is the data loaded dynamically in this filed only when you scroll?
You should be fine with extracting entire data with just one action if this is just text filed.
I would investigate this, then maybe extract text data with JS if regular action does not work, only then to go with solution provided by @VJR

Was this reply helpful? Yes No
mbenyishay 77 on at

Like (0)

Report
Hi @momlo , yes, the page loads only 100 rows by default, then loads more as you scroll. That's why I'm trying to scroll the page.
The scroll bar is inside an element rather than for the whole page. Given that, I just tried to use the instructions here to scroll inside an element. https://learn.microsoft.com/en-us/power-automate/desktop-flows/how-to/scroll-web-page
document.getElementById('divElem').scrollTop -= 10;
I replaced the 'divElem' with the ID of the element that seems to contain the table, which is 'lk-dataflux-data-table'. I'm not getting any scrolling so far.
Here is the screenshot of how the results page looks. The scrollbar on the right is for the element. The highlighted first column header was what I was previously using to focus the "text box" to send page-down keys. That works consistenly in attended mode (or even unattended mode when I trigger the cloud flow) but not consistently when it's scheduled.

Was this reply helpful? Yes No
VJR 7,635 on at

Like (0)

Report

Hi @mbenyishay

Some more options to try (each of the below are different options):

- See if you can create an UI element of the scrollbar itself. Maybe you already tried?

- Also do a click on some other suitable element

- Focus/click on some other element other than the textbox you are currently using

- In your case, will it help in increasing the row limit to beyond 5000 in that box, so that there is no need to scroll

- Is there some kind of Export or Download option to get these rows

Was this reply helpful? Yes No
mbenyishay 77 on at

Like (0)

Report

Hi @VJR , the limit is 5000 rows, but I've written the queries so that we never hit more than half of that limit. But not all 500 rows are initially displayed to the web page, it's the first 100 rows that are retrieved until you scroll.
The manual workflow when running reports on this page is to click the gear icon (next to the run button) which brings up a download option. However, I have never been able to successfully identify the element of the gear icon and successfully have PAD click it. Hence extract data from the page instead.

Was this reply helpful? Yes No
MattJimison 577 Most Valuable Professional on at

Like (0)

Report
@mbenyishay if there's an export option I'd try to go that route - it just simplifies the process and will be less prone to errors. Here are a few questions to (hopefully) help get you to the point where you can export the file:

What is the initial selector you're getting when you select the gear icon that you're saying you can't click? If you can share it I can try and help. You can also use document.querySelector in your browser's dev tools to manually test the selector and make sure you're getting what you need.
The default behavior of the Hosted RPA Bot VM Image will be to just download the item when you click a link to download - if you do get a selector working, a few things to keep in mind:
Make sure that clicking the link isn't spawning a pop-up, which the default image will do if clicking this link causes a popup (if this does happen I can help - actually finishing up a blog post soon on this)
Since you're clicking the link and it will just download automatically, you're going to have to wait for the file to download before you can expect it to be available. You'll need to determine a wait based on how large the file is and/or based on #3 below do some kind of check to see if what you expect is in the folder yet
You can use the 'get specialized folder' to get the Documents folder path on the VM and then do a "Replace text" and replace 'Documents' with 'Downloads' in it and then you'll have the path to the VM's downloads file (would be nice if 'get specialized folder' had downloads as an option but this will work). You can then use "get files in folder" on that download folder and sort by creation time descending to get the last file downloaded (that being the file you'd have just downloaded)
At this point you'd want to open the file and process as-needed

Find this post helpful? Please mark it as the solution and/or provide kudos so that it will help others in the future.

Cheers,
Matt

Was this reply helpful? Yes No
mbenyishay 77 on at

Like (0)

Report

Thanks Matt. Here is how it looks to export: you first click the gear, click the download button in the flyout (first screenshot), then select a few options in the download menu (second screenshot).

However, I have never gotten the PAD web automation selectors to pick up on the gear icon at all, even when I used the inspect menu (third screenshot) to locate it.

Was this reply helpful? Yes No
momlo 1,527 Super User 2024 Season 1 on at

Like (1)

Report
That works consistenly in attended mode (or even unattended mode when I trigger the cloud flow) but not consistently when it's scheduled.
This got me thinking - in untattended mode, by defualt, screen resolution is set to 800x600.
I'm sure you are developing your flow on much higher resolution - this means, that when it runs in unattended, some of the UI elements might not appear on 800x600 screen.

Force screen resolution for unattended session to resolution you have on your dev machine (you have action for that) - if this works, keep it that way for find the lowest that still works.

Was this reply helpful? Yes No
MattJimison 577 Most Valuable Professional on at

Like (0)

Report

I just noticed the shortcut to download on the menu you shared; have you tried sending keys to enact that? (Up+Ctrl+L)

Was this reply helpful? Yes No
MattJimison 577 Most Valuable Professional on at

Like (0)

Report

Are you saying when you go to record or add a new UI element that you can never get PAD to show you a red box around the gear? If that's the case, you could just click something else in the UI, then look at the html of the page to write your own selector. You might also want to try a different browser to see if you have any luck selecting on a different one.

Was this reply helpful? Yes No