web
You’re offline. This is a read only version of the page.
close
Skip to main content

Announcements

News and Announcements icon
Community site session details

Community site session details

Session Id :
Power Platform Community / Forums / Power Automate / I'm trying to perform ...
Power Automate
Answered

I'm trying to perform OCR on an image with Power Automate

(0) ShareShare
ReportReport
Posted on by 60

As the title describes, I'm trying to do OCR on an image which a user submits from Microsoft Forms. The flow is as follows: (i) User submits a form (which includes questions and prompts the user to upload an image), (ii) Gather responses and perform OCR on the image, and (iii) Send an email with the responses and the parsed image. To accomplish this, I setup the following flow: 
The Flow ProcessThe Flow Process

 

With this, I am receiving the following error: 

Error messageError message

 

I'm not sure what the error is since I double checked and I did submit a png image. Any ideas on what I can do differently? 

Categories:
I have the same question (0)
  • Verified answer
    JoeF-MSFT Profile Picture
    Microsoft Employee on at

    Hi @tareenmj - thanks for the question and cool use case! I didn't know files can now be submitted in Microsoft Forms! 💚

     

    The add a screenshot... response contains a number of metadata on the file. There are two steps you need to add to your flow to get the image content.

     

    1. A Parse JSON action. Select the add a screenshot... field for Content. And add the following for Schema

     

    {
     "type": "array",
     "items": {
     "type": "object",
     "properties": {
     "name": {
     "type": "string"
     },
     "link": {
     "type": "string"
     },
     "id": {
     "type": "string"
     },
     "type": {},
     "size": {
     "type": "integer"
     },
     "referenceId": {
     "type": "string"
     },
     "driveId": {
     "type": "string"
     },
     "status": {
     "type": "integer"
     },
     "uploadSessionUrl": {}
     },
     "required": [
     "name",
     "link",
     "id",
     "type",
     "size",
     "referenceId",
     "driveId",
     "status",
     "uploadSessionUrl"
     ]
     }
    }

     

    2. Add a Get file content action from the OneDrive for Business connector to get the actual image that was uploaded on the Form. Use the id output coming from the 'Parse JSON' action for the File input.

     

    3. You can then use the file content output from the previous OneDrive action in the AI Builder OCR action. 

     

    JoeFMSFT_0-1651938228446.png

     

    Let us know if this works for you!

     

  • tareenmj Profile Picture
    60 on at

    Such a thoughtful and well-written answer. I didn't realize I needed to perform actions upon the uploaded file but thank you so much for the help! It worked out perfectly. Thanks!

  • JoeF-MSFT Profile Picture
    Microsoft Employee on at

    Glad to hear you got it working - congrats @tareenmj! 🙂

     

    I'm curious to learn more about your scenario (and if you can share) - what type of images are your users submitting through the Microsoft Form you are doing OCR? Is it a photo of a document or similar? 

  • tareenmj Profile Picture
    60 on at

    Thanks! Absolutely, the problem I am trying to solve is actually a bit more complicated than just doing OCR. I'm trying to read OCR of screenshots: 

     

    Service-Mode-With-Samsung-device.png

     

    The problem I'm trying to solve is a bit more complicated than just OCR. I need to store the values in the screenshot as variables. So for example, the TAC of 3107 into a separate variable. Also, since the screenshots can vary quite a bit, I can't just build an AI function to extract certain pieces of the screenshot. Some screenshots will have more information than others. 

  • JoeF-MSFT Profile Picture
    Microsoft Employee on at

    This is really interesting. Thanks for sharing @tareenmj!

     

    Do the screenshots vary jut in the number of rows? If so, you could look at building an AI Builder Form Processing model and extract the table: https://docs.microsoft.com/en-us/ai-builder/create-form-processing-model#tag-tables

     

    You could also look at combining OCR with AI Builder Entity Extraction. You can train an Entity Extraction model to extract certain elements from a body of text: Create an entity extraction custom AI model - AI Builder | Microsoft Docs

     

  • tareenmj Profile Picture
    60 on at

    Not a problem, I'm glad you found it interesting. I was thinking of using an AI Builder Entity Extraction but thing is since the screenshots can vary, I'm not sure if this is the right step. The screenshots can vary in not only number of rows but also the title (for example screenshot A might contain value1 = 35 while screenshot B will contain value1-trio = 26C). I was thinking that I could parse the string obtained from OCR. In the case of the previously mentioned example, find the substring position at "value1=" and then adding a certain number to be able to get the value 35. Similarly, I would do another substring search for "value1-trio =" and then offset to get the value of "26C". Do you think this would be a better approach or could I get this done using the custom extraction? 

Under review

Thank you for your reply! To ensure a great experience for everyone, your content is awaiting approval by our Community Managers. Please check back later.

Helpful resources

Quick Links

Introducing the 2026 Season 1 community Super Users

Congratulations to our 2026 Super Users!

Kudos to our 2025 Community Spotlight Honorees

Congratulations to our 2025 community superstars!

Congratulations to the March Top 10 Community Leaders!

These are the community rock stars!

Leaderboard > Power Automate

#1
Haque Profile Picture

Haque 605

#2
Valantis Profile Picture

Valantis 340

#3
11manish Profile Picture

11manish 284

Last 30 days Overall leaderboard