Unanswered

How to extract dynamic content from web page using web scrapper

(0) Share

Report

Posted on by srinibashdash19

Hi there,

I have a web page where data loading dynamically means when we scrolldown it updates new data. It's kind of when ever we scroll , we will get new data. I have tried using "Extract data by Web Scraper" but it is only extracting top 10 results. can someone please let me know if there is any twik to achieve this scenario . for reference I have added screenshot of web page.

Categories:

Power Automate Desktop

I have the same question (0)

All responses (6)

Answers (0)

momlo 1,527 Super User 2024 Season 1 on at

Like (0)

Report

scroll the page down to the bottom then extract 🙂

Was this reply helpful? Yes No
srinibashdash19 66 on at

Like (0)

Report

This is a never ending scroll bar. after every 10 seconds we can scroll and get new data.

Was this reply helpful? Yes No
momlo 1,527 Super User 2024 Season 1 on at

Like (0)

Report

Ok, I get it, so what is your goal?
If it is to download entire table -you need to scroll right to the bottom.
If you want to extract 100 items - scroll down few times manually and count (remember it depends on screen resolution) until 100 items are displayed, then create flow where you scroll down that many times and extract 100 items.

Was this reply helpful? Yes No
srinibashdash19 66 on at

Like (0)

Report

Aim is to download entire table. but it is never ending scroll bar. But second option looks good and will try out that. Thanks for the suggestion.

Was this reply helpful? Yes No
momlo 1,527 Super User 2024 Season 1 on at

Like (1)

Report

If you san it is never ending table - you cannot download entire table, as it is never ending 😄

If the goal is to download entire table, you need to scroll to the bottom until it reaches the very bottom, but i guess 1000s or more as this depends on your one drive/spo logs retention.

So you really need to know how many logs you want to download and then create flow accordingly.
I would explore if i can get these logs in a form of excel/csv somewhere 🙂

Was this reply helpful? Yes No
MrBruceWayne 2 on at

Like (0)

Report

You can accomplish by using PAD run JavaScript on webpage. The JavaScript will keep scrolling and wait for content load if you need to extract during each scroll or check you could check and then keep scrolling. I had similar page where had to keep scrolling to capture all the data.

Was this reply helpful? Yes No