Answered

How to scrape different webpage from URL listed in excel

Like (0) Share

Report

Posted on 11 May 2023 04:31:30 by SumitP2018

I want to scrape 3 information from huge number of webpages which I have listed in excel. The issue is when the query reads the current item from excel (which is the URL of the webpage), it pastes the URL in the browser but the page loads incomplete. For example, the URL in my excel is 'https://ABCD/-/XYZ/12128/4/2/ ' but the page which loads in browser is 'https://ABCD/-/XYZ/12128 '. The later part "4/2" does not load. I want to scrape the data from 'https://ABCD/-/XYZ/12128/4/2/ ' and not from https://ABCD/-/XYZ/12128 .

Thanks in advance.

All responses (9)

Answers (1)

Nived_Nambiar 17,503 Super User 2025 Season 1 on 13 May 2023 at 07:40:31

Like (0)

Report

Re: How to scrape different webpage from URL listed in excel

Hi @SumitP2018

could you please how the issue was resolved, it would be helpful for others if they are looking for same solution 🙂
SumitP2018 13 on 13 May 2023 at 04:51:57

Like (1)

Report

Re: How to scrape different webpage from URL listed in excel

@Nived_Nambiar Thanks for looking into this. I guess the problem was that my excel was saved on sharepoint hence it was not reading the URLs correctly. But @Srini007 guided me to how to do it.

Thank a lot both of you 🙂
SumitP2018 13 on 13 May 2023 at 04:50:07

Like (0)

Report

Re: How to scrape different webpage from URL listed in excel

Thanks for your guidance, the problem is solved
Nived_Nambiar 17,503 Super User 2025 Season 1 on 11 May 2023 at 10:41:18

Like (0)

Report

Re: How to scrape different webpage from URL listed in excel

Hi @SumitP2018

Do one thing

Launch https://www.google.com using launch chrome action

Use Go to webpage action to go to https://ABCD/-/XYZ/12128/4/2/

See whether it works or not 🙂
SumitP2018 13 on 11 May 2023 at 08:40:35

Like (0)

Report

Re: How to scrape different webpage from URL listed in excel

@Nived_Nambiar

The value when getting extracted. it in below format
https://ABCD/-/XYZ/12128/4/2/

then the webpage opens with this https://ABCD/-/XYZ/12128/4/2/ but when it completes loading it ends up like https://ABCD/-/XYZ/12128.

Thanks a lot
Nived_Nambiar 17,503 Super User 2025 Season 1 on 11 May 2023 at 08:30:10

Like (0)

Report

Re: How to scrape different webpage from URL listed in excel

Hi @SumitP2018

when value is extracted from excel, is url value like this ?
https://ABCD/-/XYZ/12128/4/2/

or like this

https://ABCD/-/XYZ/12128

Could u confirm pls ?
Verified answer

SriniK 3,025 Super User 2025 Season 1 on 11 May 2023 at 06:03:56

Like (1)

Report

Re: How to scrape different webpage from URL listed in excel

@SumitP2018

If possible can you share the url, so that I can try and see the exact issue?

Thanks,
Srini
SumitP2018 13 on 11 May 2023 at 06:02:27

Like (0)

Report

Re: How to scrape different webpage from URL listed in excel

Hi Srini,

thanks for your reply. Yes, I tried that, I copy pasted the URL in the browser and it opened the exact page I needed with 4/2. Even when the flow is running, the webpage opens with complete URL but till the page loads, 4/2 vanishes. So manually it works, only when the flow is running it does not. Thanks again.
SriniK 3,025 Super User 2025 Season 1 on 11 May 2023 at 05:58:13

Like (0)

Report

Re: How to scrape different webpage from URL listed in excel

@SumitP2018

Can you try manually and see if you type the entire URL is it still going to the /4/2 page?
Also, check while opening the url, the full URL is in the browser or not?

Thanks,
Srini