I've got one collection that I have trained that shows the confidence score of the collection at 98% on the main AI Builder dashboard but when I take one of the samples I just trained and run it through -- the confidence score is only 41%. I passed 10 of the PDFs through it and all of them have a confidence score of about 41%. The 10 PDFs were actually ones used to do the training.. The collection has 25 samples in it.
It is causing havoc since I am evaluating the PDFs to see if their confidence score is above 60% and if it is not, the Flow processes it using the prebuilt Invoice Processing model.
Any thoughts?
I have the same issue, in my model accuracy score is 83 % but when I run the flow, the AI confidence score is about 40 %, even with the same documents used in the model training.. Any thoughts about that?
hello
that is very strange indeed, my first question would be : if you test the model through a power automate flow, did you already publish the last version of your custom AI builder model ? if not, please do so and test again, if yes, give me more info, then i will try to help you out
regards
ignace