web
You’re offline. This is a read only version of the page.
close
Skip to main content

Announcements

News and Announcements icon
Community site session details

Community site session details

Session Id :
Power Platform Community / Forums / Power Automate / Document Processing fi...
Power Automate
Unanswered

Document Processing field detection getting worse as I add more training documents

(0) ShareShare
ReportReport
Posted on by 190

Hey Team,

 

anyone else finding that their per-field scores are going down, as you add more training documents? Even though, in theory, adding more documents should improve those scores...

 

Thanks

Craig

 

Categories:
I have the same question (0)
  • plarrue Profile Picture
    Microsoft Employee on at

    Hi  @Craig_Humphrey 

    Thanks for reaching out. Did you also tag the newly added documents to the training data set after you included them?

    Improve the performance of your document processing model - AI Builder | Microsoft Learn

     

    Thanks

    Regards

     

  • Craig_Humphrey Profile Picture
    190 on at

    Yes, you can't retrain the model without having the documents tagged.

  • Dana_zr Profile Picture
    2 on at

    Hi @Craig_Humphrey 

     

    were your issue resolved? cause i'm facing the same issue right now. i retrained my model for an additional field but a different field got effected. and the accuracy keeps getting worse and worse.

     

    Thanks.

  • Craig_Humphrey Profile Picture
    190 on at

    Hi @Dana_zr ,

     

    No, I was working through a MS support ticket at the time and came to the conclusion that the tech just wasn't ready. There seemed to be so many factors that required separate collections of training documents for the invoicing we were working on:

    1. different vendor (sometimes different stores from the same vendor had sufficiently different layouts)
    2. single page vs multiple pages
    3. grouped items that don't have individual costs
    4. discounts
    5. fields that sometimes wrapped lines (like addresses)

    These all caused us to have to have separate collections of training documents, which was a real pain, as some of them might only occur once every year or two...

     

    We eventually abandoned it.

    It's another case where Power Platform is great and quick for doing quick, simple things, but runs out of steam (or gets way to complex) as the complexity increases.  Probably would look at Azure's AI services if we get asked to tackle this again.

     

    Sorry I can't be more help.

     

    Thanks
    Craig

  • takolota1 Profile Picture
    4,980 Moderator on at

    @Craig_Humphrey @Dana_zr 

    I’ve had people get pretty great results with this alternative set-up that just OCRs document text to create a replica text file & then feed that to GPT for data extraction.

    https://powerusers.microsoft.com/t5/Power-Automate-Cookbook/Extract-Data-From-PDFs-and-Images-With-GPT/td-p/2201345

    It’s especially useful when you have different vendors / file formats & don’t want to train multiple collections of documents since it’s all just GPT without extra training.

     

     

    (Although MS did make this method a little more difficult to share/set-up since they deprecated their, in my opinion better, GPT action for new prompt actions that require creating custom prompts in weird menus)

  • Craig_Humphrey Profile Picture
    190 on at

    Hey @takolota,

     

    that is a really cool solution.  Will definitely keep that in mind next time I need to go down this path.

     

    Thanks

    Craig

     

Under review

Thank you for your reply! To ensure a great experience for everyone, your content is awaiting approval by our Community Managers. Please check back later.

Helpful resources

Quick Links

Introducing the 2026 Season 1 community Super Users

Congratulations to our 2026 Super Users!

Kudos to our 2025 Community Spotlight Honorees

Congratulations to our 2025 community superstars!

Congratulations to the April Top 10 Community Leaders!

These are the community rock stars!

Leaderboard > Power Automate

#1
Vish WR Profile Picture

Vish WR 1,027

#2
Valantis Profile Picture

Valantis 815

#3
Haque Profile Picture

Haque 630

Last 30 days Overall leaderboard