Unanswered

Fetching 400k rows from PostgreSQL

(1) Share

Report

Posted on by CU30082123-0

Hello! I'm trying to speed up the process of fetching more than 400,000 rows from a PostgreSQL table. Currently, I'm using the collections method, which takes around 10 minutes to load, and sometimes it just breaks. I also tried using Power Automate, but it has a limitation of 100,000 rows and takes about 8 minutes to fetch those.

Is there any way to speed up this process?

All responses (2)

Answers (0)

CU30082123-0 12 on at

Like (0)

Report

Fetching 400k rows from PostgreSQL

Hi FLMike,

I have Collections code that fetches batches of 2,000 records using the index column until all records are retrieved. The goal is to use these rows as a template for users to write into another table.

For example, imagine we have 20 types of shoes, where the only difference between them is color. We want to use these rows as a template to bulk update the price across all 20 shoes. This process will be done on a monthly basis.
Michael E. Gernaey 40,296 Super User 2025 Season 1 on at

Like (1)

Report

Fetching 400k rows from PostgreSQL

Hi

Technically, you absolutely can make Power Automate work for your 400K, but speeding it up, that takes a lot more thought than we can give just off this little information.

But can you give more details on why you want to load 400K rows into a Canvas or Model Driven App?

It is possible to load them all, but its going to cause the users PC and or Phone to explode as its way too memory intensive,. Also collections can only hold ( by default 2k) so I am not sure how you are using Collections to pull 400K, no way,.