Skip to main content

Notifications

Announcements

🌸You're Invited: Community Spring Festival Awards Celebration 🌸
Replay: Power Platform Community AMA - Copilot Studio | April
Welcome to the Power Platform Communities
News and Announcements icon
Community site session details

Community site session details

Session Id : OF/hSeB1GQrCnDPZx/1k5y
Power Platform Community / Forums / Power Apps - Microsoft Dataverse / Run Data Integration P...
Power Apps - Microsoft Dataverse
Answered

Run Data Integration Project keep getting duplicates data

Like (0) ShareShare
ReportReport
Posted on 16 Sep 2019 04:13:23 by PytByt 943

Hi all..

 

I'm new with this data integration, so i made a test project to make sure it works perfectly. But i keep getting duplication data every time i hit Run Project. I'm using D365 to CDS connection to sync the service number.

 

This is the connection i use:
I want to sync the Service NumberI want to sync the Service Number

 

I keep getting the same data (duplicates data) everytime the project run.

mapping2.png

The same data is always getting pulled by the system every time i hit the Run Project Button.

 

It seems somehow the system not checking if i already have the service number or not, because the Upsert always shows the same number (not update, just inserting the same data over again).
I always get duplicate data with the same 246 data totalI always get duplicate data with the same 246 data total

 

What i do wrong?
What should i do to stop the duplicates?

Please help.

Br.

pytbyt

 

 

Categories:
Dataflows
Business Rules
Entities
  • Verified answer
    v-xida-msft Profile Picture
    v-xida-msft on 17 Sep 2019 at 04:45:47
    Re: Run Data Integration Project keep getting duplicates data

    Hi @PytByt ,

    The "Alternate Key" in CDS Entity is used to identify the duplicate records. More details about creating an "Alternate Key" in CDS, please check the following article:

    https://community.dynamics.com/crm/b/microsoftdynamicscrmsolutions/posts/define-alternate-keys-for-an-entity

    Please also check the following article for more details about working with Alternate Key in CDS Entity:

    https://docs.microsoft.com/en-us/powerapps/developer/common-data-service/define-alternate-keys-entity#targetText=With%20alternate%20keys%20you%20can,place%20of%20the%20primary%20key.

     

    More details about creatig a duplicate rule in CDS Entity, please check the following video:

    https://www.youtube.com/watch?v=hFZNRmtti5g

     

    Best regards,

  • PytByt Profile Picture
    PytByt 943 on 17 Sep 2019 at 03:04:50
    Re: Run Data Integration Project keep getting duplicates data

    Hi @v-xida-msft 

    Could you please give a brief description explaination of how alternate key works with my case? and how i create the alternate key  and duplicate rule in CDS?

    Also, i confirm that  the "Duplicate Detection" option for my CDS Entity is already enabled by default. So, this option is not my exit.

    Thanks, 

    pytbyt

     

     

  • PytByt Profile Picture
    PytByt 943 on 17 Sep 2019 at 02:02:54
    Re: Run Data Integration Project keep getting duplicates data

    Hi @v-xida-msft 

    Its nice to see your reply.

    I will read the articles and see if i get it right, also i will try to enable the "Duplicate Detection" option on my CDS Entity.

     

    BR.

    pytbyt

  • v-xida-msft Profile Picture
    v-xida-msft on 17 Sep 2019 at 01:59:53
    Re: Run Data Integration Project keep getting duplicates data

    Hi @PytByt ,

    Have you taken a try to define an alternate key or duplicate detection rules for your CDS Entity?

     

    The user @Dudditz has faced similar issue with you, please check the response within the following thread:

    https://powerusers.microsoft.com/t5/Common-Data-Service-for-Apps/Integration-Job-A-Record-Already-exists-and-creating-duplicates/td-p/263540

     

    Please also check and see if the following article would help in your scenario:

    https://community.dynamics.com/crm/b/microsoftdynamicscrmsolutions/posts/define-alternate-keys-for-an-entity

     

    You could also consider enable the "Duplicate Detection" option for your CDS Entity as below:1.JPG

    then create a duplicate rule for your Entity based on the following article:

    https://docs.microsoft.com/en-us/powerapps/developer/common-data-service/detect-duplicate-data-with-code

     

    Please consider take a try with above solution, then check if the issue is solved.

     

    Best regards,

Under review

Thank you for your reply! To ensure a great experience for everyone, your content is awaiting approval by our Community Managers. Please check back later.

Helpful resources

News and Announcements

🌸You're Invited: Community Spring Festival Awards Celebration 🌸
Replay: Power Platform Community AMA - Copilot Studio | April
Welcome to the Power Platform Communities

Quick Links

Understanding Microsoft Agents - Introductory Session

Confused about how agents work across the Microsoft ecosystem? Register today!

Warren Belz – Community Spotlight

We are honored to recognize Warren Belz as our May 2025 Community…

Congratulations to the April Top 10 Community Stars!

Thanks for all your good work in the Community!

Leaderboard

#1
WarrenBelz Profile Picture

WarrenBelz 146,745 Most Valuable Professional

#2
RandyHayes Profile Picture

RandyHayes 76,287 Super User 2024 Season 1

#3
Pstork1 Profile Picture

Pstork1 66,091 Most Valuable Professional

Leaderboard

Featured topics

Product updates

Microsoft Power Platform Community release plans