Unanswered

Acessing "Select a certificate for authentication" window fails

(0) Share

Report

Posted on by Community Power Pla... Microsoft Employee

Within my company you need to authenticate via Client certificate residing on a PKI card.

When i build a Browserflow of course internal sites ask for authentication. Then usually a windows security popup occurs asking to select a certificate.

Every attempt to focus/select this window manually or using the recorder fails so far.

When i select a certificate manually in the flow the following window, asking for the related pw/pin, then can be automated again.

I know this has been asked before but there hasn't been a satisfying answer. And in general i know that in UIPath for example this is possible.

Any idea how to work around this?

Moe

Categories:

Power Automate Desktop

I have the same question (0)

All responses (13)

Answers (0)

Sort by

fraenK 2,125 on at

Like (0)

Report
Copy link

Link copied!

Which Windows edition/version are you using?

Did you already try to inspect which UI elements can be accessed using inspect.exe?

https://docs.microsoft.com/en-us/windows/win32/winauto/inspect-objects

In general, if UI automation doesn't work, you could try using:

1. window + keys

2. image + clicks

3. keys only

But you need to add some verification steps to make sure you're working on the right UI element plus avoiding dangerous inputs to the wrong element. Also hijackings might be possible.

Was this reply helpful? Yes No
MichaelAnnis 5,727 Moderator on at

Like (0)

Report
Copy link

Link copied!

I have one of these, but it always naturally pops up and is the top window and naturally in focus, and the most current key is already selected.

I have a ‘send mouse to image’ to click the OK, and an On Error (after 10 seconds) to an Error Subflow which just presses {Enter} for if the image fails (which is typical if I am remote).

Was this reply helpful? Yes No
Community Power Pla... Microsoft Employee on at

Like (0)

Report
Copy link

Link copied!

I am using Win10, PAD 2.15.284.21354, which should be the latest. Just tried out inspect, where i can find the select window. Now let me ask how this helps me to make Power Automate Desktop recognize this window?

After this i will try to go for using keys.

Was this reply helpful? Yes No
fraenK 2,125 on at

Like (0)

Report
Copy link

Link copied!

Wich elements of the window are you able to select using inspect? The ones you need or none?

Was this reply helpful? Yes No
Community Power Pla... Microsoft Employee on at

Like (0)

Report
Copy link

Link copied!

In inspect i can choose the one i need.

Was this reply helpful? Yes No
fraenK 2,125 on at

Like (0)

Report
Copy link

Link copied!

And these actions don't let you select the element? https://docs.microsoft.com/en-us/power-automate/desktop-flows/actions-reference/uiautomation#click

https://docs.microsoft.com/en-us/power-automate/desktop-flows/actions-reference/uiautomation#waitforwindowcontentaction

What kind of window/application is it? Does it have a taskbar icon? Can you see it in task manager?

Was this reply helpful? Yes No
Community Power Pla... Microsoft Employee on at

Like (0)

Report
Copy link

Link copied!

No, those actions do not work. This is a normal browser app, not showing in the taskbar additionally.

As part of the company portal the browserapp requires a certificate for authentication. As the browser is not handling the smarcard interface, it is handing this to the OS, which seems to create a separate window, which you cannot move aside. When i tried to add a UI element to the flow, it does not seem to see this window at all, as it is highlighting parts of the underlying browserwindow with a red frame instead.

I managed to work around this in the flow using send keys "Tab" for now.

But i really, really like to understand, why PAD does not see this window at all, even if it is visible in inspect.exe.

Was this reply helpful? Yes No
fraenK 2,125 on at

Like (0)

Report
Copy link

Link copied!

Just to make sure: For accessing the "separate window" you are using UI automation actions, not browser/web automation actions, right?

Can you post any example, output of inspect.exe?

Was this reply helpful? Yes No
Community Power Pla... Microsoft Employee on at

Like (0)

Report
Copy link

Link copied!

Below the excerpt from inspect.exe where you can see SSO page of Edge Browser and the overlaying "Select a certificate ....." window with at least the OK-Button i want to reach:

OK Button output in inspect.exe:
How found: Selected from tree...
Name: "OK"
ControlType: UIA_ButtonControlTypeId (0xC350)
LocalizedControlType: "Schaltfläche"
BoundingRectangle: {l:952 t:399 r:1020 b:431}
IsEnabled: true
IsOffscreen: false
IsKeyboardFocusable: true
HasKeyboardFocus: false
ProcessId: 6704
RuntimeId: [2A.1A0954.4.0.0.1BC]
AutomationId: ""
FrameworkId: "Chrome"
ClassName: "MdTextButton"
IsControlElement: true
IsContentElement: true
ProviderDescription: "[pid:6704,providerId:0x0 Main(parent link):Unidentified Provider (unmanaged:msedge.dll)]"
IsPeripheral: false
AriaRole: "button"
AriaProperties: "readonly=true;expanded=false;multiline=false;multiselectable=false;required=false"
LiveSettingProperty: Off (0)
IsPassword: false
IsRequiredForForm: false
DescribedBy:
ControllerForProperty:
FlowsTo:
FlowsFrom:
ClickablePoint: {x:986 y:415}
Culture: 127
Orientation: 0
OptimizeForVisualContent: false
PositionInSet: 1
SizeOfSet: 2
AnnotationObjects:
FullDescription: ""
IsDialog: false
LegacyIAccessible.ChildId: 0
LegacyIAccessible.DefaultAction: "Drücken"
LegacyIAccessible.Description: ""
LegacyIAccessible.Help: ""
LegacyIAccessible.KeyboardShortcut: ""
LegacyIAccessible.Name: "OK"
LegacyIAccessible.Role: Schaltfläche (0x2B)
LegacyIAccessible.State: markierbar (0x100000)
LegacyIAccessible.Value: ""
IsAnnotationPatternAvailable: false
IsDragPatternAvailable: false
IsDockPatternAvailable: false
IsDropTargetPatternAvailable: false
IsExpandCollapsePatternAvailable: false
IsGridItemPatternAvailable: false
IsGridPatternAvailable: false
IsInvokePatternAvailable: true
IsItemContainerPatternAvailable: false
IsLegacyIAccessiblePatternAvailable: true
IsMultipleViewPatternAvailable: false
IsObjectModelPatternAvailable: false
IsRangeValuePatternAvailable: false
IsScrollItemPatternAvailable: true
IsScrollPatternAvailable: false
IsSelectionItemPatternAvailable: false
IsSelectionPatternAvailable: false
IsSpreadsheetItemPatternAvailable: false
IsSpreadsheetPatternAvailable: false
IsStylesPatternAvailable: false
IsSynchronizedInputPatternAvailable: false
IsTableItemPatternAvailable: false
IsTablePatternAvailable: false
IsTextChildPatternAvailable: false
IsTextEditPatternAvailable: false
IsTextPatternAvailable: false
IsTextPattern2Available: false
IsTogglePatternAvailable: false
IsTransformPatternAvailable: false
IsTransform2PatternAvailable: false
IsValuePatternAvailable: false
IsVirtualizedItemPatternAvailable: false
IsWindowPatternAvailable: false
IsCustomNavigationPatternAvailable: false
IsSelectionPattern2Available: false
FirstChild: [null]
LastChild: [null]
Next: "Cancel" Schaltfläche
Previous: "Certificate information" Link
Other Props: Object has no additional properties
Children: Container has no children
Ancestors: "" Region
"" Region
"" Region
"Select a certificate for authentication" Dialogfeld
"SSO - Microsoft Edge" Region
"SSO - Work - Microsoft Edge" Fenster
"Desktop 1" Bereich
[ No Parent ]

Output for "Select ......" window in instect.exe
How found: Selected from tree...
Name: "SSO - Microsoft Edge"
ControlType: UIA_PaneControlTypeId (0xC371)
LocalizedControlType: "Region"
BoundingRectangle: {l:234 t:21 r:1543 b:1070}
IsEnabled: true
IsOffscreen: false
IsKeyboardFocusable: false
HasKeyboardFocus: false
ProcessId: 6704
RuntimeId: [2A.1A0954.4.0.0.3A]
AutomationId: ""
FrameworkId: "Chrome"
ClassName: "BrowserRootView"
IsControlElement: true
IsContentElement: true
ProviderDescription: "[pid:6704,providerId:0x0 Main(parent link):Unidentified Provider (unmanaged:msedge.dll)]"
IsPeripheral: false
AriaRole: "region"
AriaProperties: "readonly=true;expanded=false;multiline=false;multiselectable=false;required=false"
LiveSettingProperty: Off (0)
IsPassword: false
IsRequiredForForm: false
DescribedBy:
ControllerForProperty:
FlowsTo:
FlowsFrom:
ClickablePoint: {x:888 y:545}
Culture: 127
Orientation: 0
OptimizeForVisualContent: false
AnnotationObjects:
FullDescription: ""
IsDialog: false
LegacyIAccessible.ChildId: 0
LegacyIAccessible.DefaultAction: ""
LegacyIAccessible.Description: ""
LegacyIAccessible.Help: ""
LegacyIAccessible.KeyboardShortcut: ""
LegacyIAccessible.Name: "SSO - Microsoft Edge"
LegacyIAccessible.Role: Bereich (0x10)
LegacyIAccessible.State: normal (0x0)
LegacyIAccessible.Value: ""
IsAnnotationPatternAvailable: false
IsDragPatternAvailable: false
IsDockPatternAvailable: false
IsDropTargetPatternAvailable: false
IsExpandCollapsePatternAvailable: false
IsGridItemPatternAvailable: false
IsGridPatternAvailable: false
IsInvokePatternAvailable: false
IsItemContainerPatternAvailable: false
IsLegacyIAccessiblePatternAvailable: true
IsMultipleViewPatternAvailable: false
IsObjectModelPatternAvailable: false
IsRangeValuePatternAvailable: false
IsScrollItemPatternAvailable: true
IsScrollPatternAvailable: false
IsSelectionItemPatternAvailable: false
IsSelectionPatternAvailable: false
IsSpreadsheetItemPatternAvailable: false
IsSpreadsheetPatternAvailable: false
IsStylesPatternAvailable: false
IsSynchronizedInputPatternAvailable: false
IsTableItemPatternAvailable: false
IsTablePatternAvailable: false
IsTextChildPatternAvailable: false
IsTextEditPatternAvailable: false
IsTextPatternAvailable: false
IsTextPattern2Available: false
IsTogglePatternAvailable: false
IsTransformPatternAvailable: false
IsTransform2PatternAvailable: false
IsValuePatternAvailable: false
IsVirtualizedItemPatternAvailable: false
IsWindowPatternAvailable: false
IsCustomNavigationPatternAvailable: false
IsSelectionPattern2Available: false
FirstChild: "Microsoft Edge" Region
LastChild: "Select a certificate for authentication" Dialogfeld
Next: [null]
Previous: "" Bereich
Other Props: Object has no additional properties
Children: "Microsoft Edge" Region
"Select a certificate for authentication" Dialogfeld
Ancestors: "SSO - Work - Microsoft Edge" Fenster
"Desktop 1" Bereich
[ No Parent ]

Was this reply helpful? Yes No
Community Power Pla... Microsoft Employee on at

Like (0)

Report
Copy link

Link copied!

Tried both, Browser and UI, none works.

Output in Inspect.exe:

Output Inspect.exe for "Select...."-Window:

How found: Selected from tree...
Name: "Select a certificate for authentication"
ControlType: UIA_PaneControlTypeId (0xC371)
LocalizedControlType: "Dialogfeld"
BoundingRectangle: {l:664 t:97 r:1112 b:447}
IsEnabled: true
IsOffscreen: false
IsKeyboardFocusable: false
HasKeyboardFocus: false
ProcessId: 6704
RuntimeId: [2A.1A0954.4.0.0.1BB]
AutomationId: ""
FrameworkId: "Chrome"
ClassName: "RootView"
IsControlElement: true
IsContentElement: true
ProviderDescription: "[pid:6704,providerId:0x0 Main(parent link):Unidentified Provider (unmanaged:msedge.dll)]"
IsPeripheral: false
AriaRole: "dialog"
AriaProperties: "readonly=true;expanded=false;multiline=false;multiselectable=false;required=false"
LiveSettingProperty: Off (0)
IsPassword: false
IsRequiredForForm: false
DescribedBy:
ControllerForProperty:
FlowsTo:
FlowsFrom:
ClickablePoint: {x:888 y:272}
Culture: 127
Orientation: 0
OptimizeForVisualContent: false
AnnotationObjects:
FullDescription: ""
IsDialog: true
LegacyIAccessible.ChildId: 0
LegacyIAccessible.DefaultAction: ""
LegacyIAccessible.Description: ""
LegacyIAccessible.Help: ""
LegacyIAccessible.KeyboardShortcut: ""
LegacyIAccessible.Name: "Select a certificate for authentication"
LegacyIAccessible.Role: Bereich (0x10)
LegacyIAccessible.State: normal (0x0)
LegacyIAccessible.Value: ""
Window.IsModal: true
IsAnnotationPatternAvailable: false
IsDragPatternAvailable: false
IsDockPatternAvailable: false
IsDropTargetPatternAvailable: false
IsExpandCollapsePatternAvailable: false
IsGridItemPatternAvailable: false
IsGridPatternAvailable: false
IsInvokePatternAvailable: false
IsItemContainerPatternAvailable: false
IsLegacyIAccessiblePatternAvailable: true
IsMultipleViewPatternAvailable: false
IsObjectModelPatternAvailable: false
IsRangeValuePatternAvailable: false
IsScrollItemPatternAvailable: true
IsScrollPatternAvailable: false
IsSelectionItemPatternAvailable: false
IsSelectionPatternAvailable: false
IsSpreadsheetItemPatternAvailable: false
IsSpreadsheetPatternAvailable: false
IsStylesPatternAvailable: false
IsSynchronizedInputPatternAvailable: false
IsTableItemPatternAvailable: false
IsTablePatternAvailable: false
IsTextChildPatternAvailable: false
IsTextEditPatternAvailable: false
IsTextPatternAvailable: false
IsTextPattern2Available: false
IsTogglePatternAvailable: false
IsTransformPatternAvailable: false
IsTransform2PatternAvailable: false
IsValuePatternAvailable: false
IsVirtualizedItemPatternAvailable: false
IsWindowPatternAvailable: true
IsCustomNavigationPatternAvailable: false
IsSelectionPattern2Available: false
FirstChild: "" Region
LastChild: "" Region
Next: [null]
Previous: "Microsoft Edge" Region
Other Props: Object has no additional properties
Children: "" Region
Ancestors: " SSO - Microsoft Edge" Region
" SSO - Work - Microsoft Edge" Fenster
"Desktop 1" Bereich
[ No Parent ]

Was this reply helpful? Yes No