Unanswered

Click menus based on text content instead of pixel coordinates?

(0) Share

Report

Posted on by rmortier

103

We recently upgraded our ERP software and noticed one of our flows was dumping out incorrect data. Upon investigation we noticed that the ERP software changed a menu (see below). The bot is clicking UI elements based on pixel coordinates but is there a way to instead click based on the content of the menu, i.e. clicking "Open" regardless of where it is in the dropdown?

Previous version:

Current Version:

Categories:

Power Automate Desktop

I have the same question (0)

All responses (7)

Answers (0)

Sort by

XRQ57 206 on at

Like
a
(0)

Report
Copy link

Link copied!

Hi
you can use "Set dropdown value" in form filling menu (french traduction).

Was this reply helpful? Yes No
rmortier 103 on at

Like
a
(0)

Report
Copy link

Link copied!

That sounds like the solution but unfortunately this software doesn't seem to have comboboxes or dropdowns recognized in the UI selector.

Was this reply helpful? Yes No
ryule 929 Super User 2024 Season 1 on at

Like
a
(0)

Report
Copy link

Link copied!

Have you tried using images (move mouse to image and click)? I use that successfully in some of my bots with dropdowns

Was this reply helpful? Yes No
rmortier 103 on at

Like
a
(0)

Report
Copy link

Link copied!

Yes, I was messing around with that earlier. I was able to get it to work when using OCR on the entire screen/window, but I couldn't get the images to work. I'm also not sure how subregions work and was unsuccessful at finding any decent documentation on them. I guess I need more time to mess around with it.

Was this reply helpful? Yes No
Swatiba 24 on at

Like
a
(0)

Report
Copy link

Link copied!

U can make use of mouse and keyboard(move mouse to text on the screen and click) which is based on OCR.

Was this reply helpful? Yes No
rmortier 103 on at

Like
a
(0)

Report
Copy link

Link copied!

I've got this working now but my worry is not being able to limit it by UI element. Like let's say I'm looking for text "Status", but what if there is another menu somewhere on the screen with the label "Status" as well? Does anyone know how the sub regions work? Like what is it relative to? Is it relative to the entire screen or just the foreground window? Can the sub region be dynamic in situations where the window isn't maximized?

Was this reply helpful? Yes No
VJR 7,635 on at

Like
a
(0)

Report
Copy link

Link copied!

Start typing using Sendkeys the first few alphabets and it will automatically go to that text.

And then if it becomes necessary press Enter.

Was this reply helpful? Yes No