Power Platform Community Forum Thread Details

Product

Power Automate Desktop

Issue Description

I have built a Power Automate Desktop flow that retrieves a patient's appointment number and then opens the corresponding appointment from a table displayed on the screen.

The process works as follows:

The flow receives the appointment number as input.

An OCR action scans the appointment table and extracts all appointment numbers along with their coordinates.

The flow compares the extracted appointment numbers with the input appointment number.

Once a match is found, the flow uses the coordinates returned by OCR to click on the appointment number and open the appointment details.

Problem

In most cases, the flow works as expected. However, there are instances where OCR correctly identifies the appointment number, but the coordinates returned for that appointment number are incorrect.

As a result, the mouse click occurs in the wrong location, causing the workflow to fail or open an incorrect record.

Expected Behavior

When OCR identifies the correct appointment number, it should return the corresponding coordinates accurately so that the flow can click the correct appointment entry.

Actual Behavior

OCR successfully identifies the appointment number, but occasionally returns incorrect coordinates, leading to an incorrect click position.

Troubleshooting Performed

Verified that the appointment number is correctly identified by OCR.

Reviewed OCR output data and confirmed that the matched appointment number exists in the OCR results.

Tested the flow multiple times to rule out intermittent application performance issues.

Observed that the issue occurs only in certain cases and is not consistently reproducible

Questions

Has anyone experienced similar issues with OCR coordinate extraction in Power Automate Desktop?

Are there any known limitations or accuracy concerns with OCR coordinate mapping?

Would you recommend an alternative approach (for example, UI elements, image recognition, anchor-based targeting, or additional validation steps) to improve reliability?