Answered

Failed to get data more than 5000 when using Recurrence

(1) Share

Report

Posted on by veracruz

Hello, I need help with getting more than 5000 data in one flow.

I use Recurrence as the trigger but then I got this error because I set the pagination to 100000 (my data is up to 60000).

I tried to filter the data in list rows present in the table step by filtering the Delay column which value is greater than or equals -7, but it didn't work.

I can't break the data into a smaller chunk, so is there any way to make the limit bigger so I can get all the data in one go without getting such a limit? So I can use the recurrence for scheduled email.

Thank you!

Categories:

General topics

I have the same question (0)

All responses (4)

Answers (0)

Sort by

Suggested answer

Michael E. Gernaey 53,963 Moderator on at

Like
a
(1)

Report
Copy link

Link copied!

Hi

So helping you is difficult as you didn't share your flow.

But the issue is simple, you need to add a Do Until, and loop through all the iterations of the Pagination.

Your flow almost assuredly has a Get Items, which only gets 5000, but you do this one and exit. You need to wrap your Get Items in a Do Until loop

Do Until

Condition: (make up something that will force your flow to keep looping until this condition is set)

Inside do until

-GetItems(asdfasdf) // here is where you need to have pagination and something to DEFINE what's in a page, so it grabs the values between X and Y, and then the

Apply to Each

Inputs GetItems

-Process items.

End of Apply to each

end of do Until

So think of it, like having two Loops. An outter and an inner. your Get items/Apply to each is the internal, where you update the Filter each "iteration of the Outer loop akka the Do Until", until all rows are processed, then you can set some flag (boolean) that makes the Do until stop looping

Was this reply helpful? Yes No
veracruz 45 on at

Like
a
(0)

Report
Copy link

Link copied!

Hii @FLMike,

So this is a bit of my flow, I use List rows present in table, is that okay to use that one? The one that has a lot of data is in "System".

Also, I'm a bit confused with what kind of condition should I use inside Do Until? And how to get the result of Do Until step for the input of the next step? In this case is for my Filter array Null.

Thanks in advance!

Was this reply helpful? Yes No
Suggested answer

SaiRT14 1,992 Super User 2025 Season 1 on at

Like
a
(0)

Report
Copy link

Link copied!
Steps to Implement Pagination with a Do Until Loop in Power Automate:

Turn on Pagination for Get Items:

Go to the settings of the Get Items action and enable Pagination. Set the Threshold to a high enough number (e.g., 100,000) if your dataset exceeds the default 5,000 limit.

Do Until Loop Structure:

Wrap the Get Items action inside a Do Until loop. This loop will continue running until you've processed all the items (i.e., until no more items are retrieved).

Use SkipToken for Pagination:

If you have many pages of items to process, use the @odata.nextLink (also called SkipToken) to paginate through the results. This is automatically returned by the Get Items action when there are more items to fetch.

Was this reply helpful? Yes No
SB-21040730-0 2 on at

Like
a
(0)

Report
Copy link

Link copied!

Please help me to set up this flow. I have been searching for a solution since months.

Was this reply helpful? Yes No