Unanswered

Scrapping multiple pages

(1) Share

Report

Posted on by gaute98

I am trying to scrape multiple pages from this "https://www.portofaden.net/en/ship-traffic-job/index?page=14" website. I get the first page, but despite hitting set page as element i dont get anything more. Below is the "code" if you can say and the screen from the app.

WebAutomation.LaunchChrome.LaunchChrome Url: $'''https://www.portofaden.net/en/ship-traffic-job/index?page=1''' WindowState: WebAutomation.BrowserWindowState.Maximized ClearCache: False ClearCookies: False WaitForPageToLoadTimeout: 60 Timeout: 60 BrowserInstance=> Browser
Excel.LaunchExcel.Launch Visible: True LoadAddInsAndMacros: False Instance=> ExcelInstance2
WebAutomation.ExtractData.ExtractTableUsingPagingFromNumberOfPagesInExcel BrowserInstance: Browser Control: $'''html > body > div > div > div:eq(1) > div > div > div:eq(1) > article > div > div:eq(1) > div > div > div > div > table > tbody > tr''' ExtractionParameters: {[$'''td:eq(0)''', $'''Own Text''', $'''''', $'''Value #1'''], [$'''td:eq(5)''', $'''Own Text''', $'''''', $'''Value #2'''], [$'''td:eq(7)''', $'''Own Text''', $'''''', $'''Value #3'''] } MaxWebPagesToProcess: 10 UsePhysicalClick: True PagerCssSelector: $'''div[id=\"w0\"] > ul > li:eq(12) > a, html > body > div > div > div:eq(1) > div > div > div:eq(1) > article > div > div:eq(1) > div > div > div > ul > li:eq(12) > a''' PostProcessData: False TimeoutInSeconds: 60 ExcelInstance=> ExcelInstance2
Excel.LaunchExcel.Launch Visible: True LoadAddInsAndMacros: False Instance=> ExcelInstance2

Categories:

Power Automate Desktop

All responses (2)

Answers (0)

Sort by

trice602 16,679 Super User 2026 Season 2 on at

Like
a
(1)

Report
Copy link

Link copied!

Hi @gaute98 ,

I know there is an easier way to do this but having said that, I put this together really fast and it works. Please review.

Essentially in step #3 I go to the last page in the pager by clicking on the ">" element and then capture 492 as a number in 4 and 5.

Step #7 is how I loop through pages.

From there for each page, I extract the entire html table, write to excel, and then go to the next page, repeat, etc. I didn't go past page three here but works in testing and in theory will go through page 492. If this was helpful, please mark as a solution so others can find it too.
Agnius Bartninkas Most Valuable Professional on at

Like
a
(0)

Report
Copy link

Link copied!

The reason why you did not succeed and why @trice602 suggested a more complex workaround that actually works, is because the paging functionality in Extract data from web page does not really work too great. It is actually quite bad. So, the best way to do something like this is actually to build a loop that will click through pages and extract each page individually.

Here's a pretty good tutorial on that: https://www.youtube.com/watch?v=yvoUBwCqdn0&ab_channel=AndersJensen

-------------------------------------------------------------------------
If I have answered your question, please mark it as the preferred solution. If you like my response, please give it a Thumbs Up.

I also provide paid consultancy and development services using Power Automate. If you're interested, DM me and we can discuss it.