Unanswered

AUto tagging images using Computer Vision AI (Sort on confidence)

(0) Share

Report

Posted on by JimmyW

2,563

So i created i flow to auto tag images when uploaded to a SharePoint library. This all works very good.
What i can see is that some words i just dont want it to tag the image with. For example i upload a image of a car and it will add a tag called "cake"

If i check th Analyze Image results in flow i can see it rates tags based on confidence score.

So i wonder is it possible to filter or sort out words if confidence score is below a certain number? Now i dont see the tag cake have any confidence rating so i wonder why is this even added?

{
 "categories": [
 {
 "name": "trans_car",
 "score": 0.9921875
 }
 ],
 "tags": [
 {
 "name": "sky",
 "confidence": 0.9996563196182251
 },
 {
 "name": "car",
 "confidence": 0.9995741248130798
 },
 {
 "name": "outdoor",
 "confidence": 0.99254310131073
 },
 {
 "name": "blue",
 "confidence": 0.9446885585784912
 },
 {
 "name": "ocean",
 "confidence": 0.9404265880584717
 },
 {
 "name": "beach",
 "confidence": 0.8443412184715271
 },
 {
 "name": "vehicle",
 "confidence": 0.7354080677032471
 },
 {
 "name": "automotive",
 "confidence": 0.6446248292922974
 },
 {
 "name": "wheel",
 "confidence": 0.6275951862335205
 },
 {
 "name": "land vehicle",
 "confidence": 0.5628407001495361
 },
 {
 "name": "sandy",
 "confidence": 0.181476891040802
 },
 {
 "name": "roof",
 "confidence": 0.10028418898582458
 }
 ],
 "description": {
 "tags": [
 "car",
 "outdoor",
 "blue",
 "ocean",
 "beach",
 "surfing",
 "water",
 "sitting",
 "cake",
 "sandy",
 "parked",
 "sand",
 "driving"
 ],
 "captions": [
 {
 "text": "a blue car on a sandy beach next to the ocean",
 "confidence": 0.9698794056125721
 }
 ]
 },
 "requestId": "deleted",
 "metadata": {
 "width": 2000,
 "height": 1500,
 "format": "Jpeg"
 }
}

Categories:

Building flows

I have the same question (0)

All responses (13)

Answers (1)

Verified answer

yashag2255 24,769 Super User 2024 Season 1 on at

Like (1)

Report

Hi @JimmyWork

You can use the filter array action for that:

Here i am checking the confidence score greater than 0.5 and filtering the tags based on that.

Hope this Helps!

If this reply has answered your question or solved your issue, please mark this question as answered. Answered questions helps users in the future who may have the same issue or question quickly find a resolution via search. If you liked my response, please consider giving it a thumbs up. THANKS!

Was this reply helpful? Yes No
RezaDorrani 12,143 on at

Like (1)

Report

Hi @JimmyWork

Use filter array

for the From option point to the tags attribute in your JSON
In my case I hardcoded your JSON response in Compose action
Make sure to filter on the ['tags'] node in the response object

Next for the filter critera
Use item()?['confidence']

--------------------------------------------------------------------------------
If this post helps answer your question, please click on “Accept as Solution” to help other members find it more quickly. If you thought this post was helpful, please give it a Thumbs Up.

Was this reply helpful? Yes No
JimmyW 2,563 on at

Like (0)

Report

So i tested to use the dynamic content from the Analyze Image. Not sure this is possible but im not getting it to filter. Can't we use the dynamic content available without the need to use JSON?

Was this reply helpful? Yes No
yashag2255 24,769 Super User 2024 Season 1 on at

Like (1)

Report

Hey @JimmyWork

You can use the dynamic content too. I did not have the option to run the action using the analyze image so I just took the data you posted originally and then used it through parse json to get the appropriate values.

Hope this Helps!

If this reply has answered your question or solved your issue, please mark this question as answered. Answered questions helps users in the future who may have the same issue or question quickly find a resolution via search. If you liked my response, please consider giving it a thumbs up. THANKS!

Was this reply helpful? Yes No
JimmyW 2,563 on at

Like (0)

Report

@yashag2255 Thank you
I tested filtering using dynamic content and it works.
But im not able to filter Tag names im not really sure how to create the same output that i have today.

My output looks like this.
I filter using dynamic content but this is for Tags and not Tag names, so how can i remove tag names or filter them because i dont get a confidence score on them?

Was this reply helpful? Yes No
JimmyW 2,563 on at

Like (0)

Report

@yashag2255 Basicly i dont know how to create a compose that looks like Tag Names result and in the same time sort on confidence score. SO i want the output to look like my compose so i can write the tags the same way.

Was this reply helpful? Yes No
yashag2255 24,769 Super User 2024 Season 1 on at

Like (1)

Report

Hey @JimmyWork

You can filter that using the apply to each like this:

I have used the variables for reference purpose only. You can replicate using the dynamic content accordingly.

Hope this Helps!

If this reply has answered your question or solved your issue, please mark this question as answered. Answered questions helps users in the future who may have the same issue or question quickly find a resolution via search. If you liked my response, please consider giving it a thumbs up. THANKS!

Was this reply helpful? Yes No
JimmyW 2,563 on at

Like (0)

Report

@yashag2255 Thank you

DO you know why microsoft don't implement confidence score on Tag Names, if they do then this would be done really easy. Right now they only have Confidence score on Tags and not Tag Names, so i dont know how Tag Names are generate really?

Anyway thank you for all the help

Was this reply helpful? Yes No
yashag2255 24,769 Super User 2024 Season 1 on at

Like (0)

Report

Hey @JimmyWork

The tags with the score has the tag names too right. So it is basically providing you with all the output and you can pick from those and use the ones that are required in the scenario you are building. I think it is generalised output and you can customize it the way you want (again depends on the scenario).

Hope this Helps!

If this reply has answered your question or solved your issue, please mark this question as answered. Answered questions helps users in the future who may have the same issue or question quickly find a resolution via search. If you liked my response, please consider giving it a thumbs up. THANKS!

Was this reply helpful? Yes No
JimmyW 2,563 on at

Like (0)

Report
@yashag2255 My Tag Names and my Tags is not the same well mostly but not the same. So i dont know how it generates the Tag Names and from dynamic content i can only Select Tag Name not Tag Names (Multiple) if i filter.

Result from Tags:
[ { "name": "car", "confidence": 0.9996582269668579 }, { "name": "road", "confidence": 0.9992706775665283 }, { "name": "outdoor", "confidence": 0.9959216117858887 }, { "name": "vehicle", "confidence": 0.9325582981109619 }, { "name": "land vehicle", "confidence": 0.9007418751716614 }, { "name": "automotive", "confidence": 0.8912363052368164 }, { "name": "red", "confidence": 0.8797998428344727 }, { "name": "mountain", "confidence": 0.8575743436813354 }, { "name": "wheel", "confidence": 0.6563787460327148 } ]
Results from Tag Names
[ "car", "road", "outdoor", "red", "mountain", "driving", "grass", "small", "riding", "parked", "motorcycle", "street", "black", "side", "parking", "lot", "man", "traveling", "white", "highway", "surfing", "hill", "mirror" ]

Was this reply helpful? Yes No