Answered

How to get all URLs on a web page?

(0) Share

Report

Posted on by fskdnl

Hi all - I would like to get all the URLs available on a web page similar to the Chrome console function that prints all the URLs in the console.

var urls = document.getElementsByTagName('a');

for (url in urls) {
 console.log ( urls[url].href );}

Is the 'Run JavaScript Function on Web Page' the right function to use? If yes, how can I modify the code above to render the URLs? I have tried the following, but it only returns [object Object] as the result.

var urls = document.getElementsByTagName('a');
var urlList = [];
for (var i = 0; i < urls.length; i++) {
 urlList.push(urls[i].href);}
return urlList;
}

Any pointers would be very much appreciated! Thanks

Categories:

Power Automate Desktop

I have the same question (0)

All responses (3)

Answers (1)

Verified answer

VJR 7,635 on at

Like (1)

Report
Add this in the javascript on webpage action.

function ExecuteScript() { var links = document.getElementsByTagName('a'); var list = ""; for(var i = 0; i< links.length; i++) { list = list + links[i].href + "\n"; } return(list); }

Output in a Text variable:

Split it by newline using Split Text if you want the result in a List variable so that you can iterate through it.

List variable:

Was this reply helpful? Yes No
fskdnl 6 on at

Like (0)

Report

Thanks so much for the solution! Much appreciated.

As a follow-up question, instead of getting all URLs on a web page, I would like to get all the URLs on a website when given a domain name. Can this be achieved via Power Automate Desktop as well? Thanks!

Was this reply helpful? Yes No
VJR 7,635 on at

Like (2)

Report

This will definitely need you to know all links on that website.

Some sites do have a sitemap, so you can try accessing that.

Another thing you can look at is:

- Launch <browser>

- Get details of web page (this will give a variable)

- Use that variable in "Recognise entities in Text" (select URL in dropdown)

Then again go into each of the links generated using above steps and then again more links from those links.

But of course this is going to extract every single thing that looks like an URL eg; image links, etc.

You can do some research and see where/how it goes.

Was this reply helpful? Yes No