Answered

AUto tagging images using Computer Vision AI (Sort on confidence)

(0) Share

Report

Posted on by JimmyW

2,563

So i created i flow to auto tag images when uploaded to a SharePoint library. This all works very good.
What i can see is that some words i just dont want it to tag the image with. For example i upload a image of a car and it will add a tag called "cake"

If i check th Analyze Image results in flow i can see it rates tags based on confidence score.

So i wonder is it possible to filter or sort out words if confidence score is below a certain number? Now i dont see the tag cake have any confidence rating so i wonder why is this even added?

{
 "categories": [
 {
 "name": "trans_car",
 "score": 0.9921875
 }
 ],
 "tags": [
 {
 "name": "sky",
 "confidence": 0.9996563196182251
 },
 {
 "name": "car",
 "confidence": 0.9995741248130798
 },
 {
 "name": "outdoor",
 "confidence": 0.99254310131073
 },
 {
 "name": "blue",
 "confidence": 0.9446885585784912
 },
 {
 "name": "ocean",
 "confidence": 0.9404265880584717
 },
 {
 "name": "beach",
 "confidence": 0.8443412184715271
 },
 {
 "name": "vehicle",
 "confidence": 0.7354080677032471
 },
 {
 "name": "automotive",
 "confidence": 0.6446248292922974
 },
 {
 "name": "wheel",
 "confidence": 0.6275951862335205
 },
 {
 "name": "land vehicle",
 "confidence": 0.5628407001495361
 },
 {
 "name": "sandy",
 "confidence": 0.181476891040802
 },
 {
 "name": "roof",
 "confidence": 0.10028418898582458
 }
 ],
 "description": {
 "tags": [
 "car",
 "outdoor",
 "blue",
 "ocean",
 "beach",
 "surfing",
 "water",
 "sitting",
 "cake",
 "sandy",
 "parked",
 "sand",
 "driving"
 ],
 "captions": [
 {
 "text": "a blue car on a sandy beach next to the ocean",
 "confidence": 0.9698794056125721
 }
 ]
 },
 "requestId": "deleted",
 "metadata": {
 "width": 2000,
 "height": 1500,
 "format": "Jpeg"
 }
}

All responses (13)

Answers (1)

yashag2255 24,529 Super User 2024 Season 1 on at

Like (1)

Report

Re: AUto tagging images using Computer Vision AI (Sort on confidence)

Hey @JimmyWork

No worries keep posting until you get the issue resolved.

Now about the tag and tag names, if you see the screenshot, I am first extracting all the arrqay elements of a particular order (tagnames for example), then I already have a tags array that is prepopulated and so, I am using the apply to each and populating a fresh array that exist in both the arrays (makes sense?)

The solution you derived pretty much does it though!

Hope this Helps!

If this reply has answered your question or solved your issue, please mark this question as answered. Answered questions helps users in the future who may have the same issue or question quickly find a resolution via search. If you liked my response, please consider giving it a thumbs up. THANKS!
JimmyW 2,563 on at

Like (0)

Report

Re: AUto tagging images using Computer Vision AI (Sort on confidence)

@yashag2255

I solved it by creating a Html table, not sure if this is a good way to do it?
JimmyW 2,563 on at

Like (0)

Report
Re: AUto tagging images using Computer Vision AI (Sort on confidence)

@yashag2255

I know i accepted the solution so sorry for tagging you and asking again but i tried getting this right and i can't
There is a difference beetwen Tags and Tag Names. Tag Names don't come with a confidence score so i can't sort them.

Tags comes with a confidence score but i can't structure the data output to look like Tag Names. Im new to using JSON and so on.
What i get from my filter looks like this.
[ { "name": "bicycle", "confidence": 0.9999939799308777 }, { "name": "bike", "confidence": 0.9754800796508789 }, { "name": "bicycle wheel", "confidence": 0.9688829183578491 }, { "name": "outdoor", "confidence": 0.9576890468597412 }, { "name": "parked", "confidence": 0.9431015253067017 } ]
I want the output to look like
[ "bicycle", "outdoor", "parked", "leaning", "wheel", "front", "sitting", "side", "standing", "large", "black", "wooden", "man", "laying", "riding" ]
I tried with Initialize variable and so on as you showed but i fail to get this to work. I just want the restructure the data i get out
JimmyW 2,563 on at

Like (0)

Report
Re: AUto tagging images using Computer Vision AI (Sort on confidence)

@yashag2255 My Tag Names and my Tags is not the same well mostly but not the same. So i dont know how it generates the Tag Names and from dynamic content i can only Select Tag Name not Tag Names (Multiple) if i filter.

Result from Tags:
[ { "name": "car", "confidence": 0.9996582269668579 }, { "name": "road", "confidence": 0.9992706775665283 }, { "name": "outdoor", "confidence": 0.9959216117858887 }, { "name": "vehicle", "confidence": 0.9325582981109619 }, { "name": "land vehicle", "confidence": 0.9007418751716614 }, { "name": "automotive", "confidence": 0.8912363052368164 }, { "name": "red", "confidence": 0.8797998428344727 }, { "name": "mountain", "confidence": 0.8575743436813354 }, { "name": "wheel", "confidence": 0.6563787460327148 } ]
Results from Tag Names
[ "car", "road", "outdoor", "red", "mountain", "driving", "grass", "small", "riding", "parked", "motorcycle", "street", "black", "side", "parking", "lot", "man", "traveling", "white", "highway", "surfing", "hill", "mirror" ]
yashag2255 24,529 Super User 2024 Season 1 on at

Like (0)

Report

Re: AUto tagging images using Computer Vision AI (Sort on confidence)

Hey @JimmyWork

The tags with the score has the tag names too right. So it is basically providing you with all the output and you can pick from those and use the ones that are required in the scenario you are building. I think it is generalised output and you can customize it the way you want (again depends on the scenario).

Hope this Helps!

If this reply has answered your question or solved your issue, please mark this question as answered. Answered questions helps users in the future who may have the same issue or question quickly find a resolution via search. If you liked my response, please consider giving it a thumbs up. THANKS!
JimmyW 2,563 on at

Like (0)

Report

Re: AUto tagging images using Computer Vision AI (Sort on confidence)

@yashag2255 Thank you

DO you know why microsoft don't implement confidence score on Tag Names, if they do then this would be done really easy. Right now they only have Confidence score on Tags and not Tag Names, so i dont know how Tag Names are generate really?

Anyway thank you for all the help
yashag2255 24,529 Super User 2024 Season 1 on at

Like (1)

Report

Re: AUto tagging images using Computer Vision AI (Sort on confidence)

Hey @JimmyWork

You can filter that using the apply to each like this:

I have used the variables for reference purpose only. You can replicate using the dynamic content accordingly.

Hope this Helps!

If this reply has answered your question or solved your issue, please mark this question as answered. Answered questions helps users in the future who may have the same issue or question quickly find a resolution via search. If you liked my response, please consider giving it a thumbs up. THANKS!
JimmyW 2,563 on at

Like (0)

Report

Re: AUto tagging images using Computer Vision AI (Sort on confidence)

@yashag2255 Basicly i dont know how to create a compose that looks like Tag Names result and in the same time sort on confidence score. SO i want the output to look like my compose so i can write the tags the same way.
JimmyW 2,563 on at

Like (0)

Report

Re: AUto tagging images using Computer Vision AI (Sort on confidence)

@yashag2255 Thank you
I tested filtering using dynamic content and it works.
But im not able to filter Tag names im not really sure how to create the same output that i have today.

My output looks like this.
I filter using dynamic content but this is for Tags and not Tag names, so how can i remove tag names or filter them because i dont get a confidence score on them?
yashag2255 24,529 Super User 2024 Season 1 on at

Like (1)

Report

Re: AUto tagging images using Computer Vision AI (Sort on confidence)

Hey @JimmyWork

You can use the dynamic content too. I did not have the option to run the action using the analyze image so I just took the data you posted originally and then used it through parse json to get the appropriate values.

Hope this Helps!

If this reply has answered your question or solved your issue, please mark this question as answered. Answered questions helps users in the future who may have the same issue or question quickly find a resolution via search. If you liked my response, please consider giving it a thumbs up. THANKS!