web
You’re offline. This is a read only version of the page.
close
Skip to main content

Notifications

Announcements

Community site session details

Community site session details

Session Id :
Power Platform Community / Forums / Power Automate / Unable to extract data...
Power Automate
Unanswered

Unable to extract datas more that 30 records from webpage because of infinite scrolling component or scroll down page

(0) ShareShare
ReportReport
Posted on by 117

Dear All,

On this website https://global-industrie.com/fr/liste-des-exposants i'm unable to grab the datas i want because it is happening up to 30 records and then stops.

May someone help to see how can i extract the URLs of all records instead of the 30?

Previous topics was opened dealing with extraction but each time there is a small issue like this one which stop the flow.

@VJRmay we have your knowledge on that?

thanks

Frederic

I have the same question (0)
  • VJR Profile Picture
    7,635 on at

    This is how the infinite scrolling works manually.

    Scroll the page and then it loads a certain number of records (about 30).

     

    So I tried to mimic the same using the PAD flow, with the only difference of scrolling to the very end of the page.

    In doing that it was found that after scrolling for 57 times with a delay of 3 seconds between each, then it shows all the records ie; 1709 rows.

     

    If you think that in future there could be more records so you could increase the Loop no. of 57 to some higher number. There is no harm if you give it to a higher number even if there are no more records. But unnecessary processing time will increase.

    You can also remove the first "Extract data" action because there is already one in the loop.

     

    VJR_0-1651233660169.png

     

    Starting from 0, the 1708th row is the last record with Z alphabet.

     

    VJR_1-1651233845991.png

     

     

     

    @@source: 'Recorder'
    WebAutomation.LaunchChrome.AttachToChromeByUrl TabUrl: 'https://global-industrie.com/en/liste-des-exposants' AttachTimeout: 10 BrowserInstance=> Browser
    WebAutomation.ExtractData.ExtractList BrowserInstance: Browser Control: $'''html > body > div:eq(0) > div:eq(1) > section > section > div > div > div:eq(1) > div > div > div:eq(0) > a''' ExtractionParameters: {[$'''div > span''', $'''Own Text''', $''''''] } PostProcessData: False TimeoutInSeconds: 60 ExtractedData=> OutputData
    LOOP LoopIndex FROM 1 TO 57 STEP 1
     WebAutomation.ExecuteJavascript BrowserInstance: Browser Javascript: $'''function ExecuteScript() 
    {
    window.scrollTo(0, document.body.scrollHeight);
    }''' Result=> Result
     WAIT 3
     WebAutomation.ExtractData.ExtractList BrowserInstance: Browser Control: $'''html > body > div:eq(0) > div:eq(1) > section > section > div > div > div:eq(1) > div > div > div:eq(0) > a''' ExtractionParameters: {[$'''div > span''', $'''Own Text''', $''''''] } PostProcessData: False TimeoutInSeconds: 60 ExtractedData=> OutputData
    END
    Display.ShowMessageDialog.ShowMessage Message: $'''COMPLETE''' Icon: Display.Icon.None Buttons: Display.Buttons.OK DefaultButton: Display.DefaultButton.Button1 IsTopMost: False ButtonPressed=> ButtonPressed
    
    # [ControlRepository][PowerAutomateDesktop]
    
    {
     "ControlRepositorySymbols": [],
     "ImageRepositorySymbol": {
     "Name": "imgrepo",
     "ImportMetadata": {},
     "Repository": "{\r\n \"Folders\": [],\r\n \"Images\": [],\r\n \"Version\": 1\r\n}"
     }
    }
    

     

    Paste the above flow in a new editor and test it. Once working fine then add it to your original flow.

    Currently it takes 5 mins to grab the entire records but is far much better than not being able to fetch the records at all.

  • PAuserFromFranc Profile Picture
    117 on at

    @VJRHello VJR, i don't understand why it's not working for me the solution you provide...

    It seems the page is not scrolling down with the script...and after 57 loop, i'm still having 30 records, don't forget i'm retreiving the URLs maybe it's différent for the h or div tag o?

     

  • PAuserFromFranc Profile Picture
    117 on at

    Sorry @VJR my missunderstood, i just figure out i only need to copy paste the window.scroll function ....

    now it's working well, thank agin for all those tips!

    may i ask you what's your job? if we can keep in touch via whatsapp (0033652022485) , and are u guy working for Microsoft or are u MVP? all those questions because i might have job opportunity ...what is your speciality in coding? which languages and for what?

    regards,

    Fred

  • PAuserFromFranc Profile Picture
    117 on at

    I'm really sorry @VJR but now access to get the company card request a JWT Token and even logged in i'm unable to get the cards

    do you have a way to have access? then i can proceed my flow? it was not the case few days before and now struggling to do my database ...cheers

Under review

Thank you for your reply! To ensure a great experience for everyone, your content is awaiting approval by our Community Managers. Please check back later.

Helpful resources

Quick Links

Forum hierarchy changes are complete!

In our never-ending quest to improve we are simplifying the forum hierarchy…

Ajay Kumar Gannamaneni – Community Spotlight

We are honored to recognize Ajay Kumar Gannamaneni as our Community Spotlight for December…

Leaderboard > Power Automate

#1
Michael E. Gernaey Profile Picture

Michael E. Gernaey 522 Super User 2025 Season 2

#2
Tomac Profile Picture

Tomac 364 Moderator

#3
abm abm Profile Picture

abm abm 243 Most Valuable Professional

Last 30 days Overall leaderboard