Skip to main content

Notifications

Announcements

Welcome to the Power Platform Communities - Lets Get Started!
Announcing new navigation menus and the Engage with the Community forum!
News and Announcements icon
Community site session details

Community site session details

Session Id : W2T8j4KuHRROf0rIaoh1sZ
Power Platform Community / Forums / Power Apps - Microsoft Dataverse / Run Data Integration P...
Power Apps - Microsoft Dataverse
Answered

Run Data Integration Project keep getting duplicates data

Like (0) ShareShare
ReportReport
Posted on 16 Sep 2019 04:13:23 by PytByt 943

Hi all..

 

I'm new with this data integration, so i made a test project to make sure it works perfectly. But i keep getting duplication data every time i hit Run Project. I'm using D365 to CDS connection to sync the service number.

 

This is the connection i use:
I want to sync the Service NumberI want to sync the Service Number

 

I keep getting the same data (duplicates data) everytime the project run.

mapping2.png

The same data is always getting pulled by the system every time i hit the Run Project Button.

 

It seems somehow the system not checking if i already have the service number or not, because the Upsert always shows the same number (not update, just inserting the same data over again).
I always get duplicate data with the same 246 data totalI always get duplicate data with the same 246 data total

 

What i do wrong?
What should i do to stop the duplicates?

Please help.

Br.

pytbyt

 

 

Categories:
Dataflows
Business Rules
Entities
  • Verified answer
    v-xida-msft Profile Picture
    v-xida-msft on 17 Sep 2019 at 04:45:47
    Re: Run Data Integration Project keep getting duplicates data

    Hi @PytByt ,

    The "Alternate Key" in CDS Entity is used to identify the duplicate records. More details about creating an "Alternate Key" in CDS, please check the following article:

    https://community.dynamics.com/crm/b/microsoftdynamicscrmsolutions/posts/define-alternate-keys-for-an-entity

    Please also check the following article for more details about working with Alternate Key in CDS Entity:

    https://docs.microsoft.com/en-us/powerapps/developer/common-data-service/define-alternate-keys-entity#targetText=With%20alternate%20keys%20you%20can,place%20of%20the%20primary%20key.

     

    More details about creatig a duplicate rule in CDS Entity, please check the following video:

    https://www.youtube.com/watch?v=hFZNRmtti5g

     

    Best regards,

  • PytByt Profile Picture
    PytByt 943 on 17 Sep 2019 at 03:04:50
    Re: Run Data Integration Project keep getting duplicates data

    Hi @v-xida-msft 

    Could you please give a brief description explaination of how alternate key works with my case? and how i create the alternate key  and duplicate rule in CDS?

    Also, i confirm that  the "Duplicate Detection" option for my CDS Entity is already enabled by default. So, this option is not my exit.

    Thanks, 

    pytbyt

     

     

  • PytByt Profile Picture
    PytByt 943 on 17 Sep 2019 at 02:02:54
    Re: Run Data Integration Project keep getting duplicates data

    Hi @v-xida-msft 

    Its nice to see your reply.

    I will read the articles and see if i get it right, also i will try to enable the "Duplicate Detection" option on my CDS Entity.

     

    BR.

    pytbyt

  • v-xida-msft Profile Picture
    v-xida-msft on 17 Sep 2019 at 01:59:53
    Re: Run Data Integration Project keep getting duplicates data

    Hi @PytByt ,

    Have you taken a try to define an alternate key or duplicate detection rules for your CDS Entity?

     

    The user @Dudditz has faced similar issue with you, please check the response within the following thread:

    https://powerusers.microsoft.com/t5/Common-Data-Service-for-Apps/Integration-Job-A-Record-Already-exists-and-creating-duplicates/td-p/263540

     

    Please also check and see if the following article would help in your scenario:

    https://community.dynamics.com/crm/b/microsoftdynamicscrmsolutions/posts/define-alternate-keys-for-an-entity

     

    You could also consider enable the "Duplicate Detection" option for your CDS Entity as below:1.JPG

    then create a duplicate rule for your Entity based on the following article:

    https://docs.microsoft.com/en-us/powerapps/developer/common-data-service/detect-duplicate-data-with-code

     

    Please consider take a try with above solution, then check if the issue is solved.

     

    Best regards,

Under review

Thank you for your reply! To ensure a great experience for everyone, your content is awaiting approval by our Community Managers. Please check back later.

Helpful resources

News and Announcements

Welcome to the Power Platform Communities - Lets Get Started!
Announcing new navigation menus and the Engage with the Community forum!

Quick Links

Announcing the Engage with the Community forum!

This forum is your space to connect, share, and grow!

🌸 Community Spring Festival 2025 Challenge Winners! 🌸

Congratulations to all our community participants!

Warren Belz – Community Spotlight

We are honored to recognize Warren Belz as our May 2025 Community…

Leaderboard > Power Apps - Microsoft Dataverse

#1
mmbr1606 Profile Picture

mmbr1606 22 Super User 2025 Season 1

#2
stampcoin Profile Picture

stampcoin 17

#3
ankit_singhal Profile Picture

ankit_singhal 11 Super User 2025 Season 1

Overall leaderboard

Featured topics

Product updates

Microsoft Power Platform Community release plans