Unanswered

How to import large amount of rows (data) efficiently into dataverse on a daily basis

(0) Share

Report

Posted on by EugeneW

I have a scenario where I have to import approximately 500K rows of transactional data into dataverse on a daily basis. I have tried importing from excel or csv file through dataflow, which took the the system around 4 hours to complete. Ideally, I would like to cut this time to under 10 min.

Data structure is very simple with a single table, about 10 columns which includes date, numbers and text strings.

When I have imported a csv file with the same data format of 1M rows to SQL, it took around 8 min. If there is a solution to migrate this data in SQL to dataverse in a short amount of time, that is also acceptable.

The reason we need to import the data to dataverse is to use the model driven app as a user interface. Please let me know if there are any efficient way to import data into dataverse, or useful workaround. Thank you.

Categories:

Microsoft Dataverse with Power Apps

I have the same question (0)

All responses (3)

Answers (0)

Ram Prakash Duraisamy 5,593 Super User 2025 Season 2 on at

Like (0)

Report

Hello @EugeneW

You can try using DATAFLOW, but for sure it will also take much time less than the Dataverse import.

if you are using SQL to import the record, then you can simply write a query to pull the data from SQL and push it to Dataverse.

Note:
Why Import is taking much time ?

1. The System will read all the data first
2. if any error or mismatch available it will throw error first
3. Without errored records, the system will start importing.

Please mark as Answer if it is helpful and provide Kudos

Subscribe : https://www.youtube.com/channel/UCnGNN3hdlKBOr6PXotskNLA
Blog : https://microsoftcrmtechie.blogspot.com

Was this reply helpful? Yes No
a33ik 3,306 Most Valuable Professional on at

Like (0)

Report

Hello,
I would highly recommend checking out SSIS with KingsWaySoft adapters - https://www.kingswaysoft.com/products/ssis-integration-toolkit-for-microsoft-dynamics-365

Was this reply helpful? Yes No
maxpower45255 223 on at

Like (0)

Report

@EugeneW What did you end up doing? what was your solution? thanks.

Was this reply helpful? Yes No