web
You’re offline. This is a read only version of the page.
close
Skip to main content

Notifications

Announcements

Community site session details

Community site session details

Session Id :
Power Platform Community / Forums / Power Automate / Using my custom AI mod...
Power Automate
Unanswered

Using my custom AI model to process an image uploaded

(0) ShareShare
ReportReport
Posted on by 60

I have a Microsoft Flow which is triggered by a user submitting a Microsoft Form. The form contains questions as follows: name and also submitting a screenshot. The screenshot would contain import data which I was hoping to extract. I tried doing it with the following: Parsing the screenshot using OCR and then applying string manipulation to extract relevant information. However, thanks to the help of the community, I was told to build an AI model and use confidence scores to disregard incorrect data. 

 

Since the screenshots are device captures (from Samsung, Google, iPhone, etc - examples shown below), they all look somewhat different (i.e. placement, amount of information, etc). To configure this, I created collections in my AI model, with each collection corresponding to a different device. 

 

Now I would like to develop a Flow which would take in the screenshot and apply my custom AI model to retrieve the relevant information. Currently my AI Model looks like: 

 

ai model.PNG

 

Now I realize that submitting a response with an image requires parsing JSON. Therefore, my current flow is as follows, I'm just having difficulty with implementing it in order to send an email containing those two fields (PCI and EarFCN). 

 

My current flow looks like: 

AI model flow.PNG

 

However, I don't see an option to add the values from my AI Model. Is there something I'm doing wrong? Additionally, how would I use confidence intervals to ensure that my model isn't reading incorrect values? 

 

--------------------------------------------------------------------------------------------------------------------------------

Device Captures: These are just two of the possible screenshots that could happen. Initially I was doing an OCR and text parsing (just because of the endless possibilities of the text positions and various devices). If you guys think that a better approach is better, let me know!

Dev7 - Service mode.jpg.pngsamsung1.jpg

Categories:
I have the same question (0)

Under review

Thank you for your reply! To ensure a great experience for everyone, your content is awaiting approval by our Community Managers. Please check back later.

Helpful resources

Quick Links

Forum hierarchy changes are complete!

In our never-ending quest to improve we are simplifying the forum hierarchy…

Ajay Kumar Gannamaneni – Community Spotlight

We are honored to recognize Ajay Kumar Gannamaneni as our Community Spotlight for December…

Leaderboard > Power Automate

#1
Michael E. Gernaey Profile Picture

Michael E. Gernaey 503 Super User 2025 Season 2

#2
Tomac Profile Picture

Tomac 321 Moderator

#3
abm abm Profile Picture

abm abm 237 Most Valuable Professional

Last 30 days Overall leaderboard