
Hello all,
I wonder if anyone could help.
I have built a simple flow that moves files placed in a folder on a server to a OneDrive account, before deleting the file from its original location on the server.
This works well, there is 100% accuracy and files are moved without issue.
However, I have been informed that large volumes of data will soon be placed into the folder on our server (up to 5,000 files) and that we will need to find a way to move the files to OneDrive much quicker (the aim is to be able to move all 5,000 files within half an hour).
The flow I have built is shown below – does anyone have any suggestions as to how I can speed up the process? At the moment, with the concurrency rate turned off and the interval set at 1 minute, its taking around 21 minutes to transfer 1,000 files.
I have tried a number of different concurrency settings and intervals, but unfortunately I cannot seem to speed up the process.
Is there anything that anyone could suggest that I could try? I am sure there is a much quicker way to move the files, and that I may need to rewrite the flow. If anyone could give me any guidance on whether it is possible to reach this kind of level of speed, that would also be much appreciated.
Thanks,
Hi @365user9 ,
Please refer to below link for throttling limits of File system connector:
File System - Connectors | Microsoft Learn
It means each connection can call File system service 1000 times every 60 seconds. As it's connector limit, we can not raise it any more.
In each run of "For each" loop, it calls 2 times. So handling1000 files within 21 minutes is almost at its best performance.
Advice is dividing the 5000 files into like 5 subfolders, and creating 5 flows with separate connection to move them.
So that you can finish the work around 20 minutes as what your flow works now.
Community Support Team _ Wenjuan Zou
If this post helps, then please consider Accept it as the solution to help the other members find it.