Power Platform Community Forum Thread Details

Power Platform Dataflow incremental refresh inconsistent behavior and duplicates the records.

Dear all,

I am having an issue that is now frustrating me. I am building some insights for financial application and the amount of data will be in billions. So, we have selected the Power Platform Dataflows. Currently, In order to test/demo the functionality, I have created a dataflow and import contact entity from Dynamics 365 CRM and apply incremental refresh on the entity.

However, I have found inconsistent behavior, whenever I make some changes in my data (i.e. also the field which is in my incremental refresh condition) the data gets duplicated. Please see the configuration of incremental refresh and duplicate data in attached snapshots.

Please see the records in contact entity. Following steps I have made to check either incremental refresh working fine or not and I found inconsistency.

1. I renamed the last names of Edward and Sandy to Owen-updated first and their ID Expiry Dates were Sandy - 1/14/2021 and Edward - 1/26/2021 and refresh my dataflow. It brings the correct data this time.

2. Then I renamed the last names of Edward and Sandy to Owen and the ID Expiry of Edward to 1/28/2021 and refresh the dataflow. This time you can see Sandy was not updated (that is fine) because it does not match the incremental refresh criteria but it did not update the Edward but bring the previous and latest record as a separate row.

Can you please help me out where I am wrong and why is this happening?

Categories:

Dataflows

Quick Links

Subscribe to this forum!