How to read/click grid data from desktop application

(0) Share

Report

Posted on by Rohit4044041

286

Hello

We have been using PAD for lot of automation and it works great in browser.

We have got now a desktop application which has lot of rows. Each row has a print button which downloads PDF.

It seems this Desktop application is based on JAVA (not sure though)

When we try to grab UI element from the desktop application, only PANE can be selected and not the grid data or rows.

Is there any way to get this data? (OCR or Recorder are not an option since its dynamic data and not all rows are to be clicked)

I did follow steps from this thread however seems it down not still recognize rows from GRID . Any thoughts please? (Before we drop idea of automating this)

https://powerusers.microsoft.com/t5/Power-Automate-Desktop/Unable-to-Capture-the-Java-application-Objects/m-p/831684/highlight/true#M1454

Categories:

Power Automate Desktop

I have the same question (0)

All responses (3)

Answers (0)

Sort by

momlo 1,527 Super User 2024 Season 1 on at

Like (1)

Report
Copy link

Link copied!
Hi @Rohit4044041
Have you seen these? Maybe will be of any help.

https://learn.microsoft.com/en-us/power-automate/desktop-flows/how-to/java
https://learn.microsoft.com/en-us/power-platform-release-plan/2021wave1/power-automate/support-java-apps-java-applets

I am working with Delphi apps and have similar lists problem. But was able to workaround problems. Here I gave overview how got it working:
https://powerusers.microsoft.com/t5/Power-Automate-Desktop/Extract-data-from-window-Does-not-return-any-rows/m-p/1895459

If you will not make JRE fully spyable, I would:

1. Use fiters i see on top of the apps to filters out rows you dont want to print
2.See if i can get the value of that counter i see on the rigth
3. I would then get the window fullscreen and use "move mouse" and "send click" combination to click the pritn links.
Of course, I would build the logic to handle whatever happens after clicking print button, between the move and click.

So example would be:

Open app (maximize window)
Filter out data
Get counter = "10'
Set the initial location of XY offset from the corner of nearest spyable UI element. Let's say x=100, y=200
Check/record the constant difference/space value between middle of the adjacent rows, for example it equals 10px
Initialize the gap variable = 0
Loop using counter
Move mouse to X=100, y=200+%gap%
Wait for "something to happen after the click"
Set %gap% = %gap%+ 10

Was this reply helpful? Yes No
Rohit4044041 286 on at

Like (0)

Report
Copy link

Link copied!

@momlo Thank you for the suggestion. With Rows being dynamic and no filters given from the applicaiton. We were going to rely on the color of the rows. So, only rows with yellow background are to be downloaded. With no ability to select individual rows, i have started feeling this might not work with the Pixel based calculation (considering number of ways it can go wrong, starting with screen resolution 😞 )
But it was great suggestion. I will keep trying to see if there is way to get grid data /rows.

Was this reply helpful? Yes No
momlo 1,527 Super User 2024 Season 1 on at

Like (0)

Report
Copy link

Link copied!

Yeah, working with pixels is a last resource indeed, however if you can control the environment - it can work
I have a reliable bot handling up to 10k of documents in app that was automated using pixel based mouse movement.

Was this reply helpful? Yes No