Automating login and file download to server with anti-bot detection

(2) Share

Report

Posted on by markbowler7

I feel I may have hit a dead-end with what I am trying to achieve but thought maybe some clever person out there has a solution.

I am trying to automate (has to be runnable in the cloud) the retrieval of a file from one our 3rd party suppliers websites, a link to which is sent to me in an email. The website requires a login so I need a few steps to get to the download:

HTTP GET on the url in the email
- The server responds with a redirect to the login page with the original url encoded as a parameter
HTTP GET on the login page (ie carry out the redirect)
HTTP POST on the login page with credentials added to the payload
HTTP GET on the original url to retrieve the file

I'm extracting and reusing cookies at the various steps but the problem I am getting is that the final GET on the file, fails with another redirect to the login page. After quite a bit of digging I have worked out that the website is using Cloudflare and from what I can see when analysing the transactions that happen if I do this manually with Chrome, the server is injecting extra javascript into the login page that then does another POST to some challenge tools on the server (which I think are installed as part of using Cloudflare).

So I think I'm stumped because afaik there is no way to get HTTP (or any other action) to run javascript. Has anyone managed overcome anything similar to this?

Categories:

Building flows

I have the same question (0)

All responses (2)

Answers (0)

Suggested answer

trice602 15,404 Super User 2025 Season 2 on at

Like (0)

Report

Hi,

Yes this is VERY doable, and you can do with Power Automate for Desktop. This can run unattended from the cloud on a VM. Everything you described in steps 1-4 look like easy tasks in PAD. Recommendation: build your desktop flow in attended mode, get it right, then connect to a cloud flow and let it run unattended.

------------------------------------------------

⭐ If this was helpful, please like and mark as a verified answer to help others find this too!

Always glad to help! 💯💯💯💯💯

Tom

Follow me on LinkedIn - Thomas Rice, PMP | LinkedIn

Was this reply helpful? Yes No
markbowler7 24 on at

Like (0)

Report

Thanks @trice602, I have used PAD successfully to fire up Edge, fill in the login details and click through to the file download but I’m not sure I’m going to be able to get much further than this for what I need.

Firstly, I need to trigger the process when receiving an email with the link to the file, and although my MS 365 licence gives me access to premium connectors (Outlook, Excel etc) in PA Cloud, I don’t seem to have access to those actions in PAD.

Secondly, making this runnable unattended I think would require me to have a Power Automate Process licence, which is pretty expensive ($150 per bot/month).

P.S. I tried to post this reply last week but I wasn't able to ?!

Was this reply helpful? Yes No