Skip to main content

Notifications

Community site session details

Community site session details

Session Id :
Power Automate - Power Automate Desktop
Unanswered

How to extract dynamic content from web page using web scrapper

Like (0) ShareShare
ReportReport
Posted on 28 Oct 2022 05:36:24 by 66

Hi there,

 

I have a web page where data loading dynamically means  when we scrolldown it updates new data.  It's kind of when ever we scroll , we will get new data.  I have tried using "Extract data by Web Scraper"  but it is only extracting top 10  results.  can someone please let me know if there is any twik to achieve this scenario .   for reference I have added screenshot of web page. 

 

srinibashdash19_2-1666935102328.png

 

 

 

 

  • MrBruceWayne Profile Picture
    2 on 08 Feb 2024 at 07:22:47
    Re: How to extract dynamic content from web page using web scrapper

    You can accomplish by using PAD run JavaScript on webpage.  The JavaScript will keep scrolling and wait for content load if you need to extract during each scroll or check you could check and then keep scrolling.  I had similar page where had to keep scrolling to capture all the data.

  • momlo Profile Picture
    1,527 Super User 2024 Season 1 on 28 Oct 2022 at 07:32:24
    Re: How to extract dynamic content from web page using web scrapper

    If you san it is never ending table - you cannot download entire table, as it is never ending 😄

     

    If the goal is to download entire table, you need to scroll to the bottom until it reaches the very bottom, but i guess 1000s or more as this depends on your one drive/spo logs retention.

     

    So you really need to know how many logs you want to download and then create flow accordingly.

    I would explore if i can get these logs in a form of excel/csv somewhere 🙂

  • srinibashdash19 Profile Picture
    66 on 28 Oct 2022 at 07:22:36
    Re: How to extract dynamic content from web page using web scrapper

    Aim is to download entire table. but it is never ending scroll bar.  But second option looks good and will try out that. Thanks for the suggestion. 

  • momlo Profile Picture
    1,527 Super User 2024 Season 1 on 28 Oct 2022 at 07:12:24
    Re: How to extract dynamic content from web page using web scrapper

    Ok, I get it, so what is your goal?

    If it is to download entire table -you need to scroll right to the bottom.

    If you want to extract 100 items - scroll down few times manually and count (remember it depends on screen resolution) until 100 items are displayed, then create flow where you scroll down that many times and extract 100 items.

  • srinibashdash19 Profile Picture
    66 on 28 Oct 2022 at 07:06:34
    Re: How to extract dynamic content from web page using web scrapper

    This is a never ending scroll bar.  after every 10 seconds we can scroll and get new data. 

  • momlo Profile Picture
    1,527 Super User 2024 Season 1 on 28 Oct 2022 at 06:48:37
    Re: How to extract dynamic content from web page using web scrapper

    scroll the page down to the bottom then extract 🙂

Under review

Thank you for your reply! To ensure a great experience for everyone, your content is awaiting approval by our Community Managers. Please check back later.

Helpful resources

Quick Links

Understanding Microsoft Agents - Introductory Session

Confused about how agents work across the Microsoft ecosystem? Register today!

Warren Belz – Community Spotlight

We are honored to recognize Warren Belz as our May 2025 Community…

Congratulations to the April Top 10 Community Stars!

Thanks for all your good work in the Community!

Leaderboard

#1
WarrenBelz Profile Picture

WarrenBelz 146,731 Most Valuable Professional

#2
RandyHayes Profile Picture

RandyHayes 76,287 Super User 2024 Season 1

#3
Pstork1 Profile Picture

Pstork1 66,079 Most Valuable Professional

Leaderboard
Loading started