Add wait time between paging on Webpage Extraction

Like (0) Share

Report

Posted on 6 Oct 2023 20:43:36 by apaulso9

200

Hello all,

I am extracting a table of data from a web page that requires paging. Some of the tables have 100 pages each with 15 entries. I am running into an issue where around page 20 the web extraction fails because the "Data element can't be found" however, it is truly because the page takes a moment to load. The paging occurs within the "Extract Web page Data" action so I can't set up a wait action. Is there a way to force the paging the extraction to wait for the webpage to load after paging to the next entries in the table?

Update:

Even when manually moving to the next page the data extraction fails on Page 21. Is there a limit to how many times you can use extract data in a loop?

Here is the error message I get in both case:

Correlation Id: b68ebefb-6212-43c0-abf7-45bc0049809a

Data extraction element not found.: Microsoft.PowerPlatform.PowerAutomate.Desktop.Actions.SDK.ActionException: Failed to extract data (web page error while extracting data). ---> System.Exception: Data extraction element not found.
at Microsoft.Flow.RPA.Desktop.UIAutomation.WebAutomation.Core.DataExtraction.LegacyExtractorBase`1.GetExtractionResults(Stopwatch stopwatch, Nullable`1 timeout)
at .....

All responses (4)

Answers (0)

Agnius Bartninkas 10,045 Most Valuable Professional on 10 Oct 2023 at 08:02:16

Like (0)

Report

Re: Add wait time between paging on Webpage Extraction

There shouldn't be. Maybe there's something different with the web page once you reach page 21? Are there more pages afterwards?
apaulso9 200 on 09 Oct 2023 at 20:19:06

Like (0)

Report

Re: Add wait time between paging on Webpage Extraction

It turns out that even when I iterate through the pages it always fails on page 21. Whether it uses paging or manual clicks. Is there a limit to paging?
Agnius Bartninkas 10,045 Most Valuable Professional on 07 Oct 2023 at 03:37:12

Like (1)

Report

Re: Add wait time between paging on Webpage Extraction

I recommend not using the paging functionality if Extract data from web page. It's not exactly reliable anyway and it doesn't really work with most pages.

I would suggest building a loop and extracting each page individually, and then merging the results. See this tutorial for reference: https://www.youtube.com/watch?v=yvoUBwCqdn0&ab_channel=AndersJensen

-------------------------------------------------------------------------
If I have answered your question, please mark it as the preferred solution. If you like my response, please give it a Thumbs Up.

I also provide paid consultancy and development services using Power Automate. If you're interested, DM me and we can discuss it.
BrendonKoz 60 on 06 Oct 2023 at 21:50:15

Like (1)

Report

Re: Add wait time between paging on Webpage Extraction

Could you run an action within the Browser automation category for "Run JavaScript function on the webpage" to cause a blocking wait time until whatever you need completely loaded has completed?