Incremental refreshing in the Dataverse

(0) Share

Report

Posted on by nleuck_101

462

Hello All,

I need some help in understanding the Incremental refresh in the Dataverse. I tried using that to import a SharePoint list to my Dataverse. I set the refresh schedule for every hour. I thought it would only add new records or update existing records but instead it duplicated the records every hour. What am I not understanding or missing when setting this up? Any help would be greatly appreciated.

Thanks,

Categories:

Microsoft Dataverse with Power Apps

All responses (7)

Answers (0)

Sort by

ronaldwalcott 3,914 Moderator on at

Like
a
(0)

Report
Copy link

Link copied!

There should be a datetime field in the record which you use in configuring the incremental refresh which identifies the age of the record.
nleuck_101 462 on at

Like
a
(0)

Report
Copy link

Link copied!

I was using the import option within Dataverse. I don't see there being an option on setting the refresh to look at a certain column. Is there a different approach I need to take.
ronaldwalcott 3,914 Moderator on at

Like
a
(0)

Report
Copy link

Link copied!

Import doesn't understand if a record is new or existing so it will just create a new primary key for anything that you import.

I believe that you have to set up Duplicate Detection Rules. What makes the records unique other than the key generated within Dataverse.

Set up duplicate detection rules to keep your data clean - Power Platform | Microsoft Learn

If you want to merge records then you'll have to check the documentation at Import Data (Microsoft Dataverse) - Power Apps | Microsoft Learn as I am unsure how to configure that except to know it needs some way to identify that the record has changed.
nleuck_101 462 on at

Like
a
(0)

Report
Copy link

Link copied!

Setting up a Duplicate detection rule didn't seem to work. Maybe I set it up incorrectly. I need to import the data first and then add new or update old records when changes are made in SharePoint. I really wanted to use the virtual tables but you can't connect to them in Power BI (at least I couldn't find a way).
ronaldwalcott 3,914 Moderator on at

Like
a
(0)

Report
Copy link

Link copied!

Since you need to update data you could take two approaches.

Use Dataflows

Create and use dataflows in Power Apps - Power Apps | Microsoft Learn

or import the data into a separate table and use Power Automate or Power Apps to merge the records.
Suggested answer

Garima_PowerPlatform 170 on at

Like
a
(2)

Report
Copy link

Link copied!

Hi,

If you want more control over how the data is merged, especially when records in SharePoint are updated or added, you could import the data into a separate table in Dataverse and then use Power Automate or Power Apps to merge the records. This approach might be more suitable if you want to run complex logic or need to handle merging records manually.
Suggested answer

nleuck_101 462 on at

Like
a
(0)

Report
Copy link

Link copied!

I ended it up creating a flow that when an Item is created or modified in SharePoint to add or update the record in Dataverse.