Unanswered

Strange limitation of Flow at 2048 list items

(1) Share

Report

Posted on by CivilNJ

We have a Flow which is pulling a table from our on-prem SQL with 2550 records yet when the Flow adds them to a clean list, it only imports 2048 records. Has anyone ever seen this or am i missing something obvious? Confirmed count in table, confirmed complete flush of data from list and confirmed final count after run.

We have tried lower counts and they work properly - i.e. 1,000 records works fine. It doesn't seem to be timing out at about 34 minutes to run.

Categories:

Building flows

I have the same question (0)

All responses (6)

Answers (1)

v-yamao-msft on at

Like (0)

Report

Hi CivilNJ,

I am afraid that there might be a limitation for the records that action “get rows” could get. We are able to get the rows up to 2048.

Please consider to create a request at Flow Ideas Forum to expand the total number that action “get rows” can retrieve.
https://powerusers.microsoft.com/t5/Flow-Ideas/idb-p/FlowIdeas

Best regards,
Mabel Mao

Was this reply helpful? Yes No
Verified answer

CivilNJ 59 on at

Like (9)

Report

I believe I have resolved it. In the SQL Get_rows part of my flow, I went to the 3 dots and Settings. I forced PAGINATION to ON and set to max 5000. I am now getting all of my content to pull.

Was this reply helpful? Yes No
CivilNJ 59 on at

Like (0)

Report

I think 2048 is a default.

Was this reply helpful? Yes No
Community Power Pla... on at

Like (0)

Report

Hi CivilNJ,
I have made pagination and set limitation to 10000 even tough not able to get all records

Was this reply helpful? Yes No
twhite-cfauk 9 on at

Like (1)

Report

I've tried this and found that with pagination on and threshold set at 5000, the connector gets the correct number of records, but ends up missing some from the original source and duplicating others instead. Very odd. After a fair bit of tinkering I've surrendered and split the data on the SQL server side into 2 views for separate import. If anyone else has a robust solution though, would love to hear it

Was this reply helpful? Yes No
RPGpro 41 on at

Like (1)

Report
If anyone else has a robust solution though, would love to hear it
Make sure pagination is off, it sets a minimum record count which will mess with this process:
Create 2 variables, 1 is a boolean the other is an integer, initialise them as 0 and false.
Create a while loop which runs while the boolean is false.
Inside the loop, run get rows, set the "skip count" field as the integer you created, make sure pagination is turned off (it sets a MINIMUM return qty that will mess this up, this is why you got duplicate data when setting it to 5000).
count the number of records returned by the get rows connector, if it's 0 set the boolean to true and do nothing else. if > 0 then:
Run w/e process you want with the rows or store the data to process outside of the while loop later on.
Add the number of records to the integer, this means the next loop iteration will skip the first x rows and return the next set of data.
The loop repeats until the last record is returned.

Was this reply helpful? Yes No