web
You’re offline. This is a read only version of the page.
close
Skip to main content

Announcements

News and Announcements icon
Community site session details

Community site session details

Session Id :
Power Platform Community / Forums / Power Automate / Find text on screen (a...
Power Automate
Unanswered

Find text on screen (an OCR conundrum)

(0) ShareShare
ReportReport
Posted on by 33

I am a little puzzled by the issue I came across but before I get to it, what is the best way to search for a plain text on a webpage? I have some success using "if text on screen (OCR)" function and had very inconsistent result with "Move mouse to text on screen (OCR)" function.

I have following flow: after opening an Excel file, and getting necessary data from it, I split the text with a comma character, and I end up creating a variable that contains a unique list of values (containing alpha-numerical characters) that I am going to search for on screen (internal company's website) one-by-one (the website screen also has a list of strictly unique values). If one is found, it will click on a checkmark next to the text. 

The issue: while testing, some text consistently is being skipped as if OCR function did not locate it, when I clearly see it on the screen (and can confirm it is present using Ctrl+F function).  For example, the first two values are found, but the third one is skipped, the last two are also found. There's no error message, so nothing to troubleshoot.

 

0 ABC123
1 XYZ456
2 QWE789
3 TYU987
4 YUI654

 

Where can I go from here troubleshooting wise?

  • All of the text is visible during the flow
  • I tried to move the 'offender' value to a different row just to see if that would make any difference. Of course it didn't.
  • I tried creating "subflow" for testing purposes, and declared a variable manually to test the same steps. The results were inconsistent (I found the value a couple of times, but majority was a miss)

As a side note: the OCR functionality sometime appear to work on a subflow but not the main flow. Very odd.

 

I haven't been using Power Automate Desktop for long but everything seems logical until issues like that....

I have the same question (0)
  • VJR Profile Picture
    7,635 on at

    Hi @yevreeva 

     

    If all of them are checkboxes then you can capture an UI element for one the checkboxes, then use a dynamic selector for that UI element by adding a variable in the selector. This variable will contain the value of the text to be searched.

     

     

  • yevreeva Profile Picture
    33 on at

    It's unfortunately not that simple, not all of the checkboxes on the screen will be checked, which is why I need to find the position/location of the text next to it.

  • yevreeva Profile Picture
    33 on at

    After some excessive testing (changing font type in browser; changing brightness/contrast settings of my monitor) I've determined that OCR:

    1. Does not see about 50 pixels from the bottom of the screen, even if it's visible and the scroller is all the way down) thus cut off one of my tables therefore I was not able to find one of the search strings.
    2. Is not able to read a certain combo of alpha-numerical characters (i.e. if a text started with "D0U", or "S2").

    I added following into my workflow that helped me to resolve the issue:

    1.  Instead of searching for the text on the full screen I specified what region should be searched by declaring specific coordinates
    2. For the text that started with specific alpha-numeric characters, "Get subtext" to extract the majority of text and used that to search the specified region.

    Edit: I spoke too soon. I am testing the next row and set of variables from the Excel file and I am getting inconsistent results with searching full or a partial text on the screen.

     

  • ryule Profile Picture
    929 Super User 2024 Season 1 on at

    I agree with your comment on there are certain character combinations that it just can not recognize. Luckily it was in a situation where I had control over the names of the stuff, and I modified them to be all caps. I think I even just opted to use images for a couple of them, which seemed to work out (I used the Advanced algorithm with 15 tolerance)

Under review

Thank you for your reply! To ensure a great experience for everyone, your content is awaiting approval by our Community Managers. Please check back later.

Helpful resources

Quick Links

Season of Sharing Community Challenge Winners!

Congratulations to our community stars!

Kudos to our 2025 Community Spotlight Honorees

Expanding mentorship, skilling, and AI innovation

Congratulations to the June Top 10 Community Leaders!

These are the community rock stars!

Leaderboard > Power Automate

#1
11manish Profile Picture

11manish 223

#2
David_MA Profile Picture

David_MA 210 Super User 2026 Season 1

#3
Haque Profile Picture

Haque 175

Last 30 days Overall leaderboard