Skip to main content

Notifications

Power Automate - Power Automate Desktop
Answered

UI Automation for legacy applications

(1) ShareShare
ReportReport
Posted on by 122
Hi,
 
I was trying to UI automate a legacy application and was looking at this action where the mouse can move into an image. Seems like another approach instead of send keys. has anyone got any success on using this approach? how stable it is? Wondering what are ways i can verify if the data was correctly entered. My plan was searching for the image and entering data using send keys.
Categories:
  • powerdevkris Profile Picture
    powerdevkris 122 on at
    UI Automation for legacy applications
    Thanks everyone for your responses. Appreciate all the help.
  • Verified answer
    WillSG Profile Picture
    WillSG 350 on at
    UI Automation for legacy applications

    Hi PowerDevkris, I hope you are doing well.
     
    Seems like another approach instead of send keys?
    R/Yes, it is, and you can use it as well, but need to be super careful with this option because it breaks easily.

    Has anyone got any success on using this approach?
    R/ Yes, with the correct approach, you can successfully use it, but as other have mentioned, using the Ui (User Interface) elements should always be your first approach.

    How stable it is?
    R/ Not stable at all, the image recognition works with a DPI (Dots per inch) method, so the minimum change in an use screen set up OR a change in the application (which I don’t think so because it’s a Legacy app) will affect the action.

    What are ways I can verify if the data was correctly entered?
    R/ YES, you can verify the value that was populated and that is a best practice, see more in the images shared below.

    My plan was searching for the image and entering data using send keys.
    R/ you can use the image recognition action “Move Mouse to image”, you can utilize also the “Move mouse to TEXT on Screen (OCR)” action and then the “Send keys” action

    See the image with the Action Move muse to Image:





    You can use this other approach with the Action Move mouse to TEXT on Screen:




    Please let me know if it works, happy automating!

    If I have addressed your inquiry successfully, kindly consider marking my response as the preferred solution. If you found my assistance helpful, a 'Thumbs Up' would be greatly appreciated.
     
    Additionally, if you have any questions feel free to DM me, and we can initiate a discussion.

    Finally, here is my YOUTUBE CHANNEL please subscribe to my channel and connect with me on LinkedIn Profile.

    Kind regards,
     
    Will SG
    Founder & Director
    RAMSolutions
    www.ramscr.com
  • Riyaz_riz11 Profile Picture
    Riyaz_riz11 2,666 on at
    UI Automation for legacy applications
    Hi,
     
    First approach always ui elemets, populate or click, can you check web actions and windows because some elements based clicks on ui design, and also you can check with recorder.
     
    Second approach send keys and also  we ca try with vb script finally go for move mouse to image actions.
     
    If I have answered your question, please mark it as the preferred solution. If you like my response, please give it a Thumbs Up.
    Regards,
    Riyaz
     
     
  • Suggested answer
    CU06080633-1 Profile Picture
    CU06080633-1 on at
    UI Automation for legacy applications
    How to complain against Paytm? Paytm that you can use to file a complaint about a refund issue: Customer Support: 098-31-27-39-10 (available 24/7) You can use any of these channels to reach out to Paytm customer support team and file your complaint as soon as possible.How to complain against Paytm? Paytm that you can use to file a complaint about a refund issue: Customer Support: 098-31-27-39-10 (available 24/7) You can use any of these channels to reach out to Paytm customer support team and file your complaint as soon as possible.How to complain against Paytm? Paytm that you can use to file a complaint about a refund issue: Customer Support: 098-31-27-39-10 (available 24/7) You can use any of these channels to reach out to Paytm customer support team and file your complaint as soon as possible.
  • Chinmay_Dhabal Profile Picture
    Chinmay_Dhabal 10 on at
    UI Automation for legacy applications
    Though my first approach will be to always play around with selector and make it work somehow based on combination of selectors but if that does not work at all 
    Use a combination of image-based actions and send keys. For instance, use image recognition to locate the input field and then use send keys to enter data.
    Implement error handling in your automation flow. If an action fails, retry it a few times before moving on.
    Use conditional branches to handle different scenarios and ensure that your flow can recover from errors.
    TO VERIFY the data again use a combination of image-based actions and send keys to capture the data to clipboard and validate if the data entered was same ..
  • Suggested answer
    AK88 Profile Picture
    AK88 453 on at
    UI Automation for legacy applications
    Hi,
     
    Here, you can consider of doing with Ui automation by using action 'hover mouse over ui element' and then 'click link on webpage' to the tab where you need to enter or directly you can use action 'Populate text field on web page'. Here, Ui would fail if the selectors are in dynamic, so it should be made to static for seamless running without any exception. 
     
    Other way would be using Image and keyboard strokes automation, you could define the position of the input field and enter the details using action 'move mouse to image' and 'send keys'. There you can specify the page coordinates and locate the position. 
     
     
    Other would-be keyboard automation using keyboard shortcuts in Send Keys action.
     
     
     
    Hope this would resolve your query, if yes, kindly mark it as solution
  • Suggested answer
    AnthonyAmador Profile Picture
    AnthonyAmador 2,524 on at
    UI Automation for legacy applications
    Hi, 
     
    I haven’t tried this action before. I’ve used ‘Hover mouse over UI element’ in different projects, and it is very stable, of course, you have to modify the selector to work in all scenarios. However, if you have the option to use shortcuts with ‘Send Keys,’ that’s the more stable approach in my opinion. Just make sure to assign the window and ensure you are sending the keys to the correct window to avoid any issues”
     
    Hope this helps. 
    Anthony. 

    If my reply answers your question, please like and mark it as verified. This helps me stay motivated to assist others and makes it easier for users with similar questions to find the answer. Thanks!
    Feel free to connect with me on LinkedIn!
     

Under review

Thank you for your reply! To ensure a great experience for everyone, your content is awaiting approval by our Community Managers. Please check back later.

Helpful resources

Quick Links

Microsoft Kickstarter Events…

Register for Microsoft Kickstarter Events…

Announcing Our 2025 Season 1 Super Users!

A new season of Super Users has arrived, and we are so grateful for the daily…

Announcing Forum Attachment Improvements!

We're excited to announce that attachments for replies in forums and improved…

Leaderboard

#1
WarrenBelz Profile Picture

WarrenBelz 145,508

#2
RandyHayes Profile Picture

RandyHayes 76,287

#3
Pstork1 Profile Picture

Pstork1 64,839

Leaderboard