Answered

Incremental Refresh with Dataflow in PowerApps to load Dataverse Table loads rows again rather than editing existing ones

(0) Share

Report

Posted on by Community Power Pla...

Hi All,

I am new to Dataverse and had set up the Dataflow to load the Dataverse Table from SQL Server. I have set up the Incremental refresh on that Dataflow for 1 day using the Date field that is updated in the SQL Server. So , any records that are updated in SQL server in last 1 day are loaded but it creates the new record rather than editing the existing one. e.g. In My_Customer Table if the Address of the customer_100 is updated in SQL server yesterday, then it should update the existing record for that customer in Dataverse My_Customer Table not adding a new record.

Is it possible in Dataverse table to just update the records and avoiding the duplicate entries ?

NOTE: this Dataflow is in PowerApps not Power BI

Thank you in advance for any ideas/solutions.

Categories:

Microsoft Dataverse with Power Apps

Dataverse Duplica...

Incremental Refre...

I have the same question (0)

All responses (6)

Answers (1)

EricRegnier 8,720 Most Valuable Professional on at

Like (0)

Report

Hi @Anonymous, yes that's possible, there is by setting the "Delete rows that no longer exist in the query output" to true as per image below. This will truncate the rows and re-import all thus removing duplicates.

Hope this helps!

Was this reply helpful? Yes No
Community Power Pla... on at

Like (0)

Report

HI @EricRegnier ,
I have just tried that setting again and created a new record manually in Dataverse Table. After the refresh with new setting "Delete the rows that no longer exists in Query Output" , it wiped my newly created records and just loaded new records.
What it does is that it removes all the existing records from the Dataverse Table and reinsert new records. If this setting is not checked then it would Append all the new records again to existing records in Dataverse Table.
Now, what if we have created some records directly into Dataverse Table through Power Apps ? Those records would be deleted and new records will be loaded.

Somehow , if we can keep at least the records created in Dataverse direclty then it would solve the problem !
Please suggest if any workaround comes to mind.
Thanks,
KV

Was this reply helpful? Yes No
Verified answer

EricRegnier 8,720 Most Valuable Professional on at

Like (1)

Report

HI @Anonymous, apologies for late reply, unfortunately "Delete the rows that no longer exists in Query Output" is by design. I do hope as well it will be better in the future. You can set alternate keys on the destination table and update your Dataflows to map to these alternate keys so it can update existing records if the alt key is found. More info: https://docs.microsoft.com/power-query/dataflows/get-best-of-standard-dataflows?WT.mc_id=DX-MVP-5004271

Cheers

Was this reply helpful? Yes No
Krishna09 234 on at

Like (0)

Report

@Anonymous Can you please share with me how are you able to do incremental refresh?
I have csv files in sharepoint folder. where every week a new csv file is uploaded to it. I created a data flow to save one of these csv file data in dataverse using dataflow in dataverse. Now in order to update following weeks csv files, I want to use incremental refresh. to do that we create RangeStart and RangeEnd in dataflow of dataverse. Now how to go to incremental refresh in dataflow in dataverse?

Was this reply helpful? Yes No
Krishna09 234 on at

Like (0)

Report

@EricRegnier will this action also append the rows or only delete which does not exist in query?

Was this reply helpful? Yes No
EricRegnier 8,720 Most Valuable Professional on at

Like (0)

Report

@Krishna09 it will delete and re-import everything again

Was this reply helpful? Yes No