I have a flow that is trying to insert 75000 records into SharePoint after querying and fetching records from Oracle. The flow is shown below. The flow is essentially inserting contacts into a SharePoint list if it doesn't exist. We split the job up into 40 batches. We used arrays to store the list of existing SharePoint records. We started the flow last night and it created around 47K records before the flows started capitulating. We got the error shown below i.e. "Your flow has exceeded its content throughput (https://aka.ms/pa-contentthroughput) limit in the past 24 hours. If this high usage persists, your flow actions may get throttled and slowed down. Content throughput used: 54.93GB, content throughput transition limit: 50GB." Now we would like to figure out the throughput consumed by each step. Our gut feel is that Standard actions like filter arrays are the culprit but we are not sure. Is there a way to get better insights into each action. Another issue is also that SharePoint started throttling my requests. As you will notice that a single insert took 14 minutes.
If you can’t get all the loop actions into Select & Filter array actions outside a loop, then you may want to use ChatGPT to help you write a Python Azure Function to process JSON array batches of the data the way you need it before then inputting them with the batch SP upsert.
Under review
Thank you for your reply! To ensure a great experience for everyone, your content is awaiting approval by our Community Managers. Please check back later.