Apply to each - limit exceeded for ation Apply_to_each maximum 5000 limitation

(0) Share

Report

Posted on by Community Power Pla... Microsoft Employee

power automate microsoft flow 5000 items limit . How can I just insert 5k a time? is there a example flow for this ?

the set up is

1. grab records from some data source ( have 6k+ records)

2. put them in an array

3. insert those records into SQL server

How can i have a for loop or do until loop that basically inserts 5k records at a time so i don't hit this ridiculous limit.

Categories:

Using flows

I have the same question (0)

All responses (9)

Answers (0)

Sort by

Mira Ghaly 11,415 Moderator on at

Like (0)

Report
Copy link

Link copied!

@Anonymous
I suggest the below:
1. Add a new field to the data source that indicates if the record has been inserted in the SQL Server Database after it is inserted so you can differentiate between inserted records and non - inserted records.
2. The below example assumes your data source is sharepoint
If this post helps you with your problem, please mark your as Accepted solution.
If you like my response, please give it a Thumbs Up.
MG (Naturally Curious)

Was this reply helpful? Yes No
v-bacao-msft Microsoft Employee on at

Like (0)

Report
Copy link

Link copied!

Hi @Anonymous ,

Please check this online doc, it is mentioned here that if you have Office 365 and Free licenses, the maximum number allowed for Apply to each is 5000 times. The maximum allowed number of Plan 1, Plan 2, Per User, and Per Flow licenses is 100,000, and upgrading the license is an option for reference. https://docs.microsoft.com/en-us/power-automate/limits-and-config#looping-and-debatching-limits

Another way worth referring to is to filter records to reduce the number of records that need to be inserted.

Or there are fields in the table that can separate the data into two parts, so you can insert these data separately in Apply to each.

Hope that makes sense.

Best Regards,

Was this reply helpful? Yes No
Community Power Pla... Microsoft Employee on at

Like (0)

Report
Copy link

Link copied!

@Mira_Ghaly - you're confused.

@v-bacao-msft - can you show me an exampe of how I can use a loop to insert 4000 records for each insert? Assume I have more than 4k record total, that is my use case. I need a sample. I am not understanding how I can do that and @Mira_Ghaly did not provide a relevant example

Was this reply helpful? Yes No
Mira Ghaly 11,415 Moderator on at

Like (0)

Report
Copy link

Link copied!

@Anonymous sorry if my example wasn't relevant I was just trying to help... Good luck

Was this reply helpful? Yes No
Community Power Pla... Microsoft Employee on at

Like (0)

Report
Copy link

Link copied!

@Mira_Ghaly anyway you can just give me a example of what I'm asking? lol

1 - retreive 10k records
2 - insert 4000k records at a time so i dont hit 5k limit ?

Was this reply helpful? Yes No
v-bacao-msft Microsoft Employee on at

Like (0)

Report
Copy link

Link copied!

HI @Anonymous ,

For example, when I use the SharePoint list as a data source, I use the ID to divide all items into two parts.

Add a parallel branch to Flow to handle the items contained in these two parts separately.

Image reference:

Hope that makes sense.

Best Regards,

Was this reply helpful? Yes No
Community Power Pla... Microsoft Employee on at

Like (0)

Report
Copy link

Link copied!

@v-bacao-msft - no it doesn't make sense at all. You have a parallel branch , what if i have 20k items, you're telling me I need to make 4 parralell branches? I am receiving records from D365 entities (for context). I have no idea how many items I will have I need a way to loop through the items and delay them once I have hit the limit , wait, and insert another 4k items , rinse and repeat

Was this reply helpful? Yes No
v-bacao-msft Microsoft Employee on at

Like (0)

Report
Copy link

Link copied!

Hi @Anonymous ,

If this is the situation you describe, perhaps you should consider the method provided by @Mira_Ghaly .

Best Regards,

Was this reply helpful? Yes No
takolota1 4,980 Moderator on at

Like (0)

Report
Copy link

Link copied!

@v-bacao-msft @Mira_Ghaly

Anyone with a similar issue can now try using this batch SQL template to create, update, or delete many thousands of SQL rows in a single query:

https://powerusers.microsoft.com/t5/Power-Automate-Cookbook/SQL-Batch-Create-Read-Update-and-Delete/td-p/1715338

Was this reply helpful? Yes No