Skip to main content

Notifications

Power Apps - AI Builder
Unanswered

AI Builder Document Processing custom collection shows 98% confidence but only 41% when it actually processes.

(0) ShareShare
ReportReport
Posted on by 94

I've got one collection that I have trained that shows the confidence score of the collection at 98% on the main AI Builder dashboard but when I take one of the samples I just trained and run it through -- the confidence score is only 41%.  I passed 10 of the PDFs through it and all of them have a confidence score of about 41%.  The 10 PDFs were actually ones used to do the training..  The collection has 25 samples in it.

 

It is causing havoc since I am evaluating the PDFs to see if their confidence score is above 60% and if it is not, the Flow processes it using the prebuilt Invoice Processing model.  

 

Any thoughts?

Categories:
  • jakub_mar Profile Picture
    jakub_mar 12 on at
    Re: AI Builder Document Processing custom collection shows 98% confidence but only 41% when it actually processes.

    I have the same issue, in my model accuracy score is 83 % but when I run the flow, the AI  confidence score is about 40 %, even with the same documents used in the model training.. Any thoughts about that? 

     

  • ignaceopsommer Profile Picture
    ignaceopsommer 6 on at
    Re: AI Builder Document Processing custom collection shows 98% confidence but only 41% when it actually processes.

    hello

    that is very strange indeed, my first question would be : if you test the model through a power automate flow, did you already publish the last version of your custom AI builder model ? if not, please do so and test again, if yes, give me more info, then i will try to help you out 

    regards

    ignace  

Under review

Thank you for your reply! To ensure a great experience for everyone, your content is awaiting approval by our Community Managers. Please check back later.

Helpful resources

Quick Links

Microsoft Kickstarter Events…

Register for Microsoft Kickstarter Events…

Announcing Our 2025 Season 1 Super Users!

A new season of Super Users has arrived, and we are so grateful for the daily…

Announcing Forum Attachment Improvements!

We're excited to announce that attachments for replies in forums and improved…

Leaderboard

#1
WarrenBelz Profile Picture

WarrenBelz 145,666

#2
RandyHayes Profile Picture

RandyHayes 76,287

#3
Pstork1 Profile Picture

Pstork1 64,996

Leaderboard