Unanswered

AI Builder Document Processing custom collection shows 98% confidence but only 41% when it actually processes.

(0) Share

Report

Posted on by ford_sopris

I've got one collection that I have trained that shows the confidence score of the collection at 98% on the main AI Builder dashboard but when I take one of the samples I just trained and run it through -- the confidence score is only 41%. I passed 10 of the PDFs through it and all of them have a confidence score of about 41%. The 10 PDFs were actually ones used to do the training.. The collection has 25 samples in it.

It is causing havoc since I am evaluating the PDFs to see if their confidence score is above 60% and if it is not, the Flow processes it using the prebuilt Invoice Processing model.

Any thoughts?

Categories:

AI Builder

I have the same question (0)

All responses (2)

Answers (0)

Sort by

ignaceopsommer 6 on at

Like (0)

Report
Copy link

Link copied!

hello
that is very strange indeed, my first question would be : if you test the model through a power automate flow, did you already publish the last version of your custom AI builder model ? if not, please do so and test again, if yes, give me more info, then i will try to help you out
regards
ignace

Was this reply helpful? Yes No
jakub_mar 14 on at

Like (0)

Report
Copy link

Link copied!

I have the same issue, in my model accuracy score is 83 % but when I run the flow, the AI confidence score is about 40 %, even with the same documents used in the model training.. Any thoughts about that?

Was this reply helpful? Yes No