Unanswered

Custom model can no longer handle hyphens ( - ) in text

(0) Share

Report

Posted on by DerekSmith

Hello,

I have a model that I built back in Jan 2022. At that time everything worked fine and I was getting >90% accuracy across the board.

I've got to the point of full scale testing and implementation and I'm running into yet another problem that didn't used to be a problem.

For a specific collection I was having issues with the Amount, so I added more documents to the collection going from 5 -> 22.

That seemed to resolve the amount issue, but I then noticed that the Invoice number went from 99% to 23%.

The invoice number contains a ( - ) hyphen, Both the original and the new. All 17 new documents that I added failed to recognize the new number with a hyphen.

I created a new model with just the 17 new PDF's, tagged everything, and am still getting a 0% score on the Invoice Number.

I'm using a text field for this, as I don't think number is appropriate for the hyphen and in the examples I've seen that's what they are doing.

What happened? when will this be fixed? or any workarounds available?

Categories:

AI Builder

I have the same question (0)

All responses (4)

Answers (0)

Antrod Moderator on at

Like (0)

Report
Dear @DerekSmith ,

Sorry to hear about the downgrade of your model's performance.

Perhaps 2 things you could try to improve it:
If the invoice number always contains a hyphen, try tagging 2 fields (before and after the hyphen) and then combine them in a post processing step within your flow.
If you still get unsatisfying results, try creating a Unstructured doc processing model (instead of a Structured). In many cases, results could be better.
Hope that helps.

Was this reply helpful? Yes No
DerekSmith 51 on at

Like (0)

Report
Thanks for the response...

I'm still having the same issue.
I've only encountered 1 vendor with a hyphen in the invoice number. The rest do not. Making it conditional could be an option in the future. But this used to work with no issue.
Both the original model and the test model I created were unstructured.

I'm still not understanding why I'm seeing the issue, other fields seem to be capturing fine with non letter/number characters.

Was this reply helpful? Yes No
Antrod Moderator on at

Like (0)

Report

Hi @DerekSmith ,

Got it, thanks for the latest update. Yes, the OCR engine may not consider fields as a unique block of text when there are hyphens. Waiting for this behavior to be improved, you could indeed make the logic conditional for this vendor as you are suggesting.

Was this reply helpful? Yes No
takolota1 4,974 Moderator on at

Like (0)

Report

You could also try this new GPT data extraction method: https://powerusers.microsoft.com/t5/Power-Automate-Cookbook/Extract-Data-From-PDFs-and-Images-With-GPT/td-p/2201345

It doesn’t require specifying certain document areas, wordings, styles, data, etc. It just OCRs the file, converts it to a replica text (txt), and passes it to a GPT prompt where you can ask GPT to do whatever you want with the document data.

Even if you have some documents where the original model works better, you could set some criteria for your fields that may indicate a mistake & have it then run this method on it to double check if the custom model results agree with these results.

Was this reply helpful? Yes No