Object detection and text recognition together

(0) Share

Report

Posted on by jinivthakkar

4,181

Hi All,

I want to do object detection and also extract text from the image, is there anyway to do both in one go ?

We want to detect objects like different grocery products, and then get the barcode details, weight etc.

Thanks in Advance.

Categories:

AI Builder

I have the same question (0)

All responses (11)

Answers (1)

JoeF-MSFT on at

Like (0)

Report

Re: Object detection and text recognition together

Hi @jinivthakkar,

In a canvas Power App you will need to add the two AI Builder controls Object Detector and Text Recognizer.

One approach you can take is perform to object detection in a flow in Power Automate to identify the product, and then use the Power App to perform text recognition with the text recognizer control.

Was this reply helpful? Yes No
jinivthakkar 4,181 on at

Like (0)

Report

Re: Object detection and text recognition together

Hi Joe,

I understand that by using two AI builder controls it can be done.

But I want to do that using only one control, do you know a way it can be done ?

Detect object and text from the same object.

Was this reply helpful? Yes No
Verified answer

JoeF-MSFT on at

Like (1)

Report

Re: Object detection and text recognition together

Hi @jinivthakkar,

Unfortunately there is no AI Builder control today in Power Apps that can do both at the same time.

Was this reply helpful? Yes No
sossie07 79 on at

Like (0)

Report

Re: Object detection and text recognition together

I have the same requirement, I want to detect an "Asset Tag" in an image, then read the text in the Asset Tag.

Has there been any advancement since 2020 where this functionality is possible with AI Builder and Flow?

Was this reply helpful? Yes No
AVFC 147 on at

Like (0)

Report

Re: Object detection and text recognition together

Hi @JoeF-MSFT
I'm trying to create an meter reading app so i've trained my object detection model but now need to store those objects and perform text recognition on them. Can you elaborate on this or point me in the right direction please?

Was this reply helpful? Yes No
sossie07 79 on at

Like (0)

Report

Re: Object detection and text recognition together

Hi AVFC,

My requirements are similar in that I want to read asset tags in photos. The only way I found was to create an image analysis model, and a custom "document processing" model. The flow would analyse the image, and if it found an asset tag in it, I would send the image to the document processing model to read the text in the asset tag.

However I found this to be so unreliable it was not suitable for production use, and there is no way to "correct" or train incorrect reads of the text. So in my view its not really deserving of the name "Artifical Intelligence" as to my view Intelligence implies the ablity to learn over time. Microsoft should just rename it "Have A Guess" or "HAG" for short;-)

Was this reply helpful? Yes No
AVFC 147 on at

Like (0)

Report

Re: Object detection and text recognition together

Thanks @sossie07 for the reply, kinda sounds like what Joe suggested in message 2. Will have another play and see what i can do but as you say is kinda not messy...

Was this reply helpful? Yes No
AVFC 147 on at

Like (0)

Report

Re: Object detection and text recognition together

Would you mind elaborating on how you stored the results of your image analysis? I'm very new to Power platform and can't see which output i can use to store my captured meter readings. I've sent myself an email as a test from the flow but it just sends the coordinate details??

Was this reply helpful? Yes No
sossie07 79 on at

Like (0)

Report

Re: Object detection and text recognition together

If I understand your question correctly...
When document analysis is run in a flow, for each area of text that you tag in the AI Builder Model, there will be a range of items that it gets, in my case the tag is "Asset ID" and "Asset ID Value" is the one you want. "Asset ID Confidence" is useful for testing. e.g. Do if "Assett ID Confidence" > .8

I use Assett ID Value to rename the sharepoint file, and put details in a sharepoint list etc.

Was this reply helpful? Yes No
AVFC 147 on at

Like (0)

Report

Re: Object detection and text recognition together

The "Asset ID Value" sounds exactly what i need! I'm using my trained model (meter reader) in an AI Builder detect and count objects in images module like below. I was trying to see the output in an email but it sounds like if i want to see the output of the tagged meter reading i need to use Asset ID & Value into a SP list or Excel sheet to test?

Was this reply helpful? Yes No