We built some heavy process flows that normally at the end of the month. On these days, a specific flow runs many times (at least 3000 records are updated which then creates more list items (doing looks, splits, etc). Those 3000 records would create about 5000 lines
For the second month, and we have run into this issue that the performance of that specific flow is causing problems.
Problems: Most of our flows designed/owned in our business are managed by a one or two generic accounts. Due to flow consolidation e.g. 100 users x 2000 flows a month (200,000 flows a month). Let's just say we have enough flow runs available for us to do this. So why does Microsoft then do the above?
I know the flow in this case is very heavy (lots of conditions , arrays, splits, etc). In normal case, it's take about 6-9s per flow to completely run depending on what the output is.
The source list is SharePoint. The target is a different SharePoint list. A mass change is made on the source sharepoint list (e.g. 100 records) which then triggers 100 flows to start as an example to write to the second sharepoint list.
What is also super frustrating is that when a limit (per 5 minutes or day) are reached, the flows start but then hang. The flow will never complete. We have runtimes of 8-12 hours. So what we do is then have to manually manage each one , by reviewing the flow, and seeing if the output actually took place. Sometimes the flow says hung, but it completed all actions or sometimes it does it partially which means we need to delete the partial records created in the target list and then run the flow. We then need to trace each one to ensure we don't have duplicates etc (for example , if we restart the flow for the specific record).
So how do we actually manage this, because this is very frustrating and I don't want to go through the same pain in the future? 🙂
Note: I just found out about the concurrency control while I was writing this post. It was unlimited before.
Would setting it like this ensure we don't run into limits (when the flow is doing all the heavy things it does, lookups of sharepoint lists, splits, write records) and make it more reliable especially when 100 records are updated at once, which initiates 100 flows but only 3 will run at a time until all 100 are done?
Many thanks
