Unanswered

Can't Click Web Button

(1) Share

Report

Posted on by Community Power Pla...

Robin.Core.ActionException: Button-press failed. ---> System.NullReferenceException: Object reference not set to an instance of an object.
at Robin.Modules.WebAutomation.Actions.WebAutomationRuntime.PressButtonOnWebPage(WebBrowserInstanceVariant webBrowserInstanceVariant, TextVariant cssSelectorOfButtonToPress, Boolean waitForPageToLoad, TextVariant dialogButtonToPress, WebPageCourseOfActionIfDialogAppearsEnum courseOfActionIfDialogAppears)
at Robin.Modules.WebAutomation.Actions.WebAutomationActions.PressButtonOnWebPage(Variant webBrowserInstance, Variant cssSelectorOfButtonToPress, Variant dialogButtonToPress, Boolean waitForPageToLoad, Int32 courseOfActionIfDialogAppears)
--- End of inner exception stack trace ---
at Robin.Modules.WebAutomation.Actions.PressButtonBase.Execute(ActionContext context)
at Robin.Runtime.Engine.ActionRunner.RunAction(String action, Dictionary`2 inputArguments, Dictionary`2 outputArguments, IActionStatement statement)

The pic above shows the error message. I have tried numerous workaround including:

1. Move mouse and click command

2. Changed the command to click radio button

3. Edit the UI element (Name is not selected)

4. There are 9 selectors and 5 subsections in the edit UI Element so not sure what to try and change to get the button to work

Please assist

All responses (16)

Answers (0)

dewrimburak 28 on at

Like (0)

Report

Re: Can't Click Web Button

I suggest you prees button in window as a suggestion. I think the definitive result in web automation.

is PRESS BUTTON İN WİNDOW
HEATFreight 1,020 on at

Like (1)

Report
Re: Can't Click Web Button

@DamoBird365

In these cases where we get a web automation action that runs once and fails on subsequent page loads, we edit the selector to be more general than the dynamic selector that PAD grabs automatically from the data generated by a specific web page instance. It's dynamic in the sense that a sort of random number or unpredictable string of text is being uniquely generated on every page load. We need a more general way to reference it! Other elements that do not contain such dynamic data are more likely to work the first try and all subsequent page loads as well. So we get rid of parts of the string that are unique per page load.

See below...
Default selector, before edits. Notice long random looking number!👎
(ignore the 2 in
Selectors of UI element" <select> 'load_status' 2"
–that's a result of tagging the same element twice, the first time being the edited selector in my SOLUTION)

SOLUTION
Edited drop-down selector that just works 👍

Tip!
Try comparing different versions of the same selector. Tag it once, reload the page, and tag it again. Open 'selector builder' for both of these and copy the string of text to compare them closely. I do not suggest that you attempt to edit the selector by directly opening up that string of text and deleting stuff, you will break the syntax. It's better to use the check boxes to enable or disable specific segments of the code. And use the Operator field to broaden the scope, as I mentioned before, from "equals" to "starts with". Right there next to that Operator field is where you should edit the dynamic value to chop it off at the dynamic part.

Anyway, editing selectors has been my single greatest revelation when it comes to building robust PAD web automations! Good luck, we're all counting on you.
DamoBird365 8,942 Microsoft Employee on at

Like (0)

Report

Re: Can't Click Web Button

@HEATFreight I’ll have a play, best way for it. Thanks for the tips.
HEATFreight 1,020 on at

Like (1)

Report
Re: Can't Click Web Button

With drop-downs, first distinguish between whether you have a Web type or UI type (i think of it as a Window(s) action vs. a Browser action), but know that you may have some luck, counterintuitively, switching to the wrong type. Specifically, I have seen UI type actions work when Web actions would not.

Anyway, try both actions while you are on the page where the drop down is, and make sure you have a "Run from here" criteria as the action just before the drop-down selection action you are testing. So I use "Launch New Edge" with "Attach to existing" and then "Run from here" (or select the action and click ALT + F5). Thus the drop-down selection action I am testing is the first thing to run after PAD finds my running Edge instance. See this part is important because if the selector is dynamic, then it may fail on every "re-run" whereas that dynamic selector does not change if you don't reload the page or navigate away from that element somehow.

So if I'm not being clear, you need to leave the drop-down element where it is in the browser after you "Add New UI Element" -DO NOT NAVIGATE AWAY OR RELOAD- then immediately after you've selected that element as the target of your drop-down selection action, do "Run from here" on the "Launch new Edge - attach to existing" which immediately precedes the drop-down selection you are testing.

So flow goes
Launch new Edge - attach to existing Select UI Element on Web Page
And you either see the drop-down turn yellow and make your selection or it fails. You can go by index (I can't remember if it starts with 0 or 1 but it just counts down from the top item in the list), or you can go by value (type out the exact value or string of text to be selected from the drop-down if you can't be sure of which index it will be).

So if it turns yellow, you are golden! Close the browser or navigate away or reload the page and run the thing again. Did it work? If yes, you truly are golden. That action will probably work every time unless the code running that element on the web page changes. Unless that happens, it will be an extremely robust action that will work 100% of the time (as long as the web page isn't too janky) and will not require coddling with error traps and timeouts and whatnot. It just works. And fast!

If reloading the page or closing the browser window caused the drop-down action (or whatever web action) to fail, you can try editing the selector. In my case, the most useful thing to do is to find the selector segment which has some random looking number, and change it from "equals" to "starts with" and delete off just that number and everything after it.

That has worked every time that I have a "only works once" scenario with some web element.

Got it?
DamoBird365 8,942 Microsoft Employee on at

Like (0)

Report

Re: Can't Click Web Button

Fair play @HEATFreight, definitely a cheat / tip solution than the preferred solution then, do you have any examples on how to tie down dynamic drop downs? I note your tips on the scroll wheel which I definitely didn't know about and whilst I have seen the yellow flash on the Web browser I hadn't appreciated the difference between Web and UI, so i've taken a few things out of this discussion already. Appreciated.
HEATFreight 1,020 on at

Like (0)

Report

Re: Can't Click Web Button

@DamoBird365 that is not an advisable way to do it if you can actually use real selectors. What you are describing should only be a last resort.

Mouse and Keyboard actions are not reliable. Results may vary. Edge cases abound.

The better way is to have every action tied to a Web or UI element. If the element exists, the action performs as desired, don't even need error traps or timeout logic. Runs far faster and more reliably than any flow built on what I call "naked clicks" where you are sending clicks to XY coordinates on screen and using Mouse and Keyboard actions to edit text and highlight fields and whatnot. This is all just hope and pray tactics. You can't possibly consider all the ways such a flow will go wrong. For instance, if anything should happen to take the focus away from the active window, your "send keys" actions will potentially cause all sorts of devastation. Just imagine you are giving a toddler free reign to mash buttons on your computer. That is what Mouse and Keyboard actions generally amount to. You are leaving the reliability of your flow entirely up to chance.

Trust me... Do. Not. Do. That. That's how I learned, and now I know better. Read my explanation of Web vs. UI actions, and learn how to use them. There is no substitute for doing it right.

My most complicated desktop flow runs 5 times faster and flawlessly now because I got rid of all naked clicks and generally all Mouse and Keyboard type actions. There are a few extremely stubborn web page elements we have to work with, so I do occasionally use a "move mouse to image on screen (and send click)" action when all else fails or a "send keys {Return}" because I have found web popup dialogues that are simply not selectable as UI or Web elements and which cannot be inspected to find the corresponding code. Other than difficult edge cases, I no longer use any Mouse and Keyboard type actions, and my flows have flourished as a result.
DamoBird365 8,942 Microsoft Employee on at

Like (0)

Report

Re: Can't Click Web Button

Hi, my little cheat / tip is to think like you are an end user clicking or selecting buttons. If you are using a drop down and want to select a field called person, literally pass the keys p e r s o n via the keyboard and hit {return}. Then you don't need to worry if it's a dynamic list. Not good if you have similar fields mind you, but sometimes I try to emulate the shortcuts that I use as an end user.

It's like when the list is of all the countries and united kingdom is near the end, I start typing u n i t e d k and soon as I get there, united kingdom is ready to be selected.

Let me know if that works for you.

Damien
HEATFreight 1,020 on at

Like (0)

Report

Re: Can't Click Web Button

It's easier than you think. Seems overwhelming at first until you have some sense of what's going on.

Also, get Microsoft support on the phone! I have had great experience requesting support calls for Power Automate. I've had hour long calls with Shanghai, New Delhi, Mumbai, Barcelona, etc. The support folks are always super knowledgeable, and 100% of the time they have successfully helped me fix whatever Power Automate / PAD issues I was having. Seriously, Microsoft is "batting a thousand" at supporting me! Super impressed with the entire Power Automate platform. Good luck getting such great service with Google Cloud!
Community Power Pla... on at

Like (0)

Report

Re: Can't Click Web Button

Hi @HEATFreight

Many thanks for your comprehensive reply. I will work through the above and try it out.

I certainly agree workarounds aren't the solution, however a basic webclick flow is the start of the process for the work I am doing so cannot spend too long trying to ensure every UI element parameter is reviewed / changed.

Your assistance is appreciated! Thank you for contributing, I am sure it will help me and many others on this forum.
HEATFreight 1,020 on at

Like (0)

Report

Re: Can't Click Web Button

Don't use the recorder! I just got really good at knowing which actions to try and knowing how to test different actions to find the best one.

For instance, any web automation (where you are interacting with a web page) should use mostly "Web Automation" type actions -as opposed to "UI Automation" type actions.

Web Automation actions are the ones that highlight fields in yellow as they interact with them. These actions can happen with your browser window minimized -no problem!

Any element that fits under the Windows user interface umbrella generally requires a UI Automation type of action. Get used to the difference. The Web vs. UI types have many analogous actions, like clicking buttons or drop-downs and filling text fields and selecting radio buttons etc, but they are not the same. Get them mixed up and you will have problems.

However, if you are having issues with one type of action, it often pays to try the other type. I have found many ostensibly "web" type elements that would not work unless I used UI Automation actions that typically only apply to elements falling under that Windows user interface umbrella. I do not think the reverse applies... You probably won't have much luck using web automation actions on Windows UI elements.

So, there is a reliable and rigorous way of manually building PAD flows by stepping through the actions and tagging elements as you go. Use the desktop recorder to get a feel for the types of actions that you should be using, but build your flows entirely on your own. Manually adding actions has worked great for me! I never had much luck with the recorder anyway. Use "Run from here" to save yourself time and quickly iterate through several test runs of unreliable actions until you have the selectors dialed in. You may have to manually edit your selectors to make them less specific, in the case of dynamic web content that has a different selector every time.

Also, USE THE SCROLLWHEEL!

USE THE SCROLLWHEEL!

USE THE SCROLLWHEEL!

USE THE SCROLLWHEEL!

--

I cannot reiterate this enough: USE THE SCROLLWHEEL!

When you are in the element selector -where UI and web elements are highlighted by a red box as you hover over them- you can use your scrollwheel to step through stacked elements so you can find the exact right element and so you can see hidden elements that are hard to select. Scrolling your finger forward/up takes you higher up in the source code of the web page, to more top level elements. Scrolling back/down takes you to the bottom level element, the deepest nested element over which your cursor is hovering.

I promise that if you spend some time experimenting with selecting different elements with different actions, you will get a feel for it all, and the result will me a faster and more reliable flow.

Also, you can easily step through your existing flows and re-select any failed selectors that are giving you errors (well, to the extent that you are able to remember which element you were using in the first place). Then once your flow is free of pre-runtime errors, run it and step through the errors and hangs, reselecting elements as you go or editing their selectors.

Any time you have an action that works once but does not work on subsequent runs, you are in good shape! That is a good position to be in. I think it's easier to deal with that than actions that you can't even get to work once, although the troubleshooting steps may be similar.

If the action works exactly once, but fails whenever you reload the web page, then you are in a "edit the selector" situation. Find that element and click the "edit selector". Then you need to check for items with random looking numbers. If you see something like "iFrame = frameID_93854890174", what you want to do is go edit that selector criteria and change the "equals" drop-down to "starts with" and then delete off the number, leaving "iFrame = frameID_". After that, the action should work on every subsequent run. Don't edit the selector directly except as last resort. That's where you double click the code at the bottom and you can edit the entire string of selector code at once. Avoid that except when all else fails. The proper way to do it is to find the checkbox line where a particular piece of selector code is built, and change that one item by editing the equation. Only items with the blue checkbox checked will show up in the selector code. You can check or uncheck at will, but that is not likely to help much. What you have to do is look for overly specific selector code and try to make it less specific. You may have to toggle off some of the checkboxes and rarely you may have to toggle a checkbox on, or you may just have to change the equation a bit, like with the iFrame ID situation above. There are usually quite a lot of selector element checkboxes in this section. It's quite overwhelming. So in general, you want to make your selector code LESS specific, not more. If the action runs once but fails after page reload, reducing selector specificity should help you. There may be other situations, say where the action won't even run once, in which you should actually make the selector more specific by checking unchecked boxes and whatnot. Mostly in my experience I am reducing selector specificity, not the other way around.

So take it from me, teach yourself how to fish and you'll never go hungry! Stop using the desktop recorder!