Skip to main content

Notifications

Community site session details

Community site session details

Session Id : v85b2IRYw7MG9uwu97htlh
Power Apps - Microsoft Dataverse
Unanswered

How to import large amount of rows (data) efficiently into dataverse on a daily basis

Like (0) ShareShare
ReportReport
Posted on 1 Feb 2022 18:39:10 by

I have a scenario where I have to import approximately 500K rows of transactional data into dataverse on a daily basis.  I have tried importing from excel or csv file through dataflow, which took the the system around 4 hours to complete.  Ideally, I would like to cut this time to under 10 min.

Data structure is very simple with a single table, about 10 columns which includes date, numbers and text strings.

When I have imported a csv file with the same data format of 1M rows to SQL, it took around 8 min.  If there is a solution to migrate this data in SQL to dataverse in a short amount of time, that is also acceptable.

The reason we need to import the data to dataverse is to use the model driven app as a user interface.  Please let me know if there are any efficient way to import data into dataverse, or useful workaround. Thank you.

  • maxpower45255 Profile Picture
    223 on 08 Feb 2024 at 22:00:09
    Re: How to import large amount of rows (data) efficiently into dataverse on a daily basis

    @EugeneW What did you end up doing? what was your solution? thanks.

  • a33ik Profile Picture
    3,304 Most Valuable Professional on 01 Feb 2022 at 21:33:39
    Re: How to import large amount of rows (data) efficiently into dataverse on a daily basis

    Hello,

    I would highly recommend checking out SSIS with KingsWaySoft adapters - https://www.kingswaysoft.com/products/ssis-integration-toolkit-for-microsoft-dynamics-365

  • Ram Prakash Profile Picture
    5,179 Super User 2025 Season 1 on 01 Feb 2022 at 19:06:47
    Re: How to import large amount of rows (data) efficiently into dataverse on a daily basis

    Hello @EugeneW 

     

    You can try using DATAFLOW, but for sure it will also take much time less than the Dataverse import. 

     

    if you are using SQL to import the record, then you can simply write a query to pull the data from SQL and push it to Dataverse.

     

    Note:

    Why Import is taking much time ?

     

    1. The System will read all the data first

    2. if any error or mismatch available it will throw error first

    3. Without errored records, the system will start importing.

     

    Please mark as Answer if it is helpful and provide Kudos


    Subscribe : https://www.youtube.com/channel/UCnGNN3hdlKBOr6PXotskNLA
    Blog : https://microsoftcrmtechie.blogspot.com

     

Under review

Thank you for your reply! To ensure a great experience for everyone, your content is awaiting approval by our Community Managers. Please check back later.

Helpful resources

Quick Links

Announcing the Engage with the Community forum!

This forum is your space to connect, share, and grow!

🌸 Community Spring Festival 2025 Challenge Winners! 🌸

Congratulations to all our community participants!

Warren Belz – Community Spotlight

We are honored to recognize Warren Belz as our May 2025 Community…

Leaderboard > Power Apps - Microsoft Dataverse

#1
mmbr1606 Profile Picture

mmbr1606 22 Super User 2025 Season 1

#2
stampcoin Profile Picture

stampcoin 17

#3
ankit_singhal Profile Picture

ankit_singhal 11 Super User 2025 Season 1

Overall leaderboard

Featured topics

Loading started