I have been testing building an AP Vendor Invoice processing solution the last week and I wanted to share my experience so far to encourage additional discussion and feedback to the Power Apps and AI Builder teams.
I have previously been part of a team who built invoice automation solutions using Abbey FlexiCapture and Kofax so I am applying those ideas when putting this together. In previous invoice automation solutions I have worked with using OCR it's pretty normal you may have to make corrections to the OCR text captured.
Requirements
- Process 2000-3000 vendor invoices per month.
- Use Custom Built Models for the top vendors
- Use Microsoft Pre-built Invoice Processing (Preview Model) for remaining not covered by Custom Model.
- Invoices are routed to a SharePoint Document library, flow kicks in and runs OCR populating metadata in columns as seen below.
- Power App is used by Finance to review the invoice and correct or add any fields, example Total costs did no populate or is incorrect, etc.
- When Finance Submits changes in Power App Invoice will route to Employee for approval
- When approved creation of a Vendor invoice record in Dynamics Finance and Operations (ERP) including a link back to the invoice in SharePoint.
Overall, using AI Builder and PowerApps for this type of solution has a lot of potential with some gaps that we are hoping to fill before building this out to production level.
Here we can see the first step where invoices are processed using the Model and metadata is automatically populated in SharePoint columns.
Here is a very crude Power App testing out the concept of making corrections to OCR data that did not populate correctly and or adding values that were missed by OCR.
This is where things get interesting.
- There is no concept of training within the Power App, at least from what I can tell. I have seen some solutions that allow you to make training improvements during the validation phase which in this case would be finance reviewing the Invoice. I guess this is understandable as the models are built under the AI Builder however at least consider providing the training experience to view PDF's and make corrections to data.
- From what I can tell there does not seem to be very robust concept for viewing the Invoice (PDF) within a Power App and making corrections to the OCR values populated in SharePoint. I was kind of hoping the same experience used when creating a custom model (The training step) would be exposable in a Power App however this is not the case. Power Apps can't natively view PDF's hosted in Office 365 SharePoint without work arounds. This one I really struggle to understand how this issue has existed this long and we still can't natively view PDF's hosted in SharePoint. We are aware you can open the PDF in a new tab however this creates an inefficient process as you are bouncing between two screens or tabs to make corrections. This is a well known issue and documented. The current workaround is using a flow which runs with every PDF you view adding a few second delay between each view. The more common approach uses premium connectors. We are testing using the following approach.
- The next issue is often when making corrections to your invoice OCR values you want a way to quickly grab the correct value from an OCR PDF. In most solutions I have used you can simply point and click any text from the OCR PDF and it will populate the currently highlighted field very similar to the Microsoft approach when training a custom model. To improve the situation we are performing OCR on the PDF in the flow. You can see in the PowerApp above Amount Due was double Clicked and since this PDF has been processed by the OCR step in the flow you can quickly copy and paste this value into the Invoice Total blank field. This clearly is not as good as the point and click provided in the training however its better than manually typing corrections Strange enough the only way to execute embedded OCR into a PDF using flow is through to 3rd party actions provide by Encodian and Adobe which also means additional costs. Is there really not a Microsoft provided action to create embedded OCR text in a PDF?
- These last two items related to the PowerApps PDF issue and the lack of embedded OCR in a PDF are big ones. feel like both these would be properly addressed if a better control was provided to Power Apps similar to the training in the custom model that properly supported viewing of PDF's in SharePoint and point in click auto-population of text from the OCR PDF.
For reference here is what I was referring to when discussing the point and click or drawing mechanics during the training portion. Sure would be great if we could get a similar experience within the Power App for corrections to invoices.
I really like what I am seeing here and hoping for some feedback in how to address some of our issues.
