Add wait time between paging on Webpage Extraction

(0) Share

Report

Posted on by apaulso9

200

Hello all,

I am extracting a table of data from a web page that requires paging. Some of the tables have 100 pages each with 15 entries. I am running into an issue where around page 20 the web extraction fails because the "Data element can't be found" however, it is truly because the page takes a moment to load. The paging occurs within the "Extract Web page Data" action so I can't set up a wait action. Is there a way to force the paging the extraction to wait for the webpage to load after paging to the next entries in the table?

Update:

Even when manually moving to the next page the data extraction fails on Page 21. Is there a limit to how many times you can use extract data in a loop?

Here is the error message I get in both case:

Correlation Id: b68ebefb-6212-43c0-abf7-45bc0049809a

Data extraction element not found.: Microsoft.PowerPlatform.PowerAutomate.Desktop.Actions.SDK.ActionException: Failed to extract data (web page error while extracting data). ---> System.Exception: Data extraction element not found.
at Microsoft.Flow.RPA.Desktop.UIAutomation.WebAutomation.Core.DataExtraction.LegacyExtractorBase`1.GetExtractionResults(Stopwatch stopwatch, Nullable`1 timeout)
at .....

Categories:

Power Automate Desktop

I have the same question (0)

All responses (4)

Answers (0)

Sort by

BrendonKoz 60 on at

Like (1)

Report
Copy link

Link copied!

Could you run an action within the Browser automation category for "Run JavaScript function on the webpage" to cause a blocking wait time until whatever you need completely loaded has completed?

Was this reply helpful? Yes No
Agnius Bartninkas Most Valuable Professional on at

Like (1)

Report
Copy link

Link copied!

I recommend not using the paging functionality if Extract data from web page. It's not exactly reliable anyway and it doesn't really work with most pages.

I would suggest building a loop and extracting each page individually, and then merging the results. See this tutorial for reference: https://www.youtube.com/watch?v=yvoUBwCqdn0&ab_channel=AndersJensen

-------------------------------------------------------------------------
If I have answered your question, please mark it as the preferred solution. If you like my response, please give it a Thumbs Up.

I also provide paid consultancy and development services using Power Automate. If you're interested, DM me and we can discuss it.

Was this reply helpful? Yes No
apaulso9 200 on at

Like (0)

Report
Copy link

Link copied!

It turns out that even when I iterate through the pages it always fails on page 21. Whether it uses paging or manual clicks. Is there a limit to paging?

Was this reply helpful? Yes No
Agnius Bartninkas Most Valuable Professional on at

Like (0)

Report
Copy link

Link copied!

There shouldn't be. Maybe there's something different with the web page once you reach page 21? Are there more pages afterwards?

Was this reply helpful? Yes No