web
You’re offline. This is a read only version of the page.
close
Skip to main content
Community site session details

Community site session details

Session Id : t/xZOgy/gunYGNNTFPcFpU
Power Automate - Building Flows
Answered

Remove dublicate Data with Power Automate

Like (0) ShareShare
ReportReport
Posted on 2 Nov 2021 09:08:50 by 41

Hey Fellows πŸ™‚

 

I have a problem building a Flow. Maybe some of you can give me an inspiration.

 

So what i have build is a Custom Connector, importing Data from a hybrid Business Central into a custom Dataverse Table.

The Problem is, that i have to run the flow every 24 hours, so it get "new" Data from BC, but when i do that, it importing all the data again, without checking dublicates. Now i want something, detecting the dublicates and removing it.

 

My first thought was, i make a automated flow, with a automated trigger in dataverse "if a new item is created" it will look through the other items and if its a duplicate, it will delete it.
But no idea how i start there. Can somebody give me a hint? Or maybe there is another better way to check for duplicates?

 

Cheers

  • Berwynn Profile Picture
    2 on 07 Nov 2023 at 11:52:05
    Re: Remove dublicate Data with Power Automate

    Hey, 

     

    I know I am a bit late to this, but though I'd leave a comment in case someone else comes across this thread.

    Originally in my flow I had a very similar logic to this, however record stopped coming in a week ago (not sure if MS has changed something in the background). 
    Regardless, what we need to do now is to check if there is more than 1 record and not more than 0.

    Berwynn_0-1699357793901.png

    Here is my logic behind it:

    1. The record comes in and gets created.

    2. We fetch the table with a given criteria.

    3. The criteria will always return at least one record (the newly created one)

    4. The newly created record is flagged as a duplicate and therefore is deleted.

     

    So we can keep the existing logic in, it's just the case of changing the 0 (or in my case null) to a 1, ensuring the original record will exist at least once.

     

    I hope this is helpful!

  • thegreatdensby Profile Picture
    41 on 04 Nov 2021 at 09:14:27
    Re: Remove dublicate Data with Power Automate

    Thank you very much @Mira_Ghaly !

    Its working now and its running fast.

     

     

  • Verified answer
    Mira Ghaly Profile Picture
    11,409 Moderator on 02 Nov 2021 at 20:23:07
    Re: Remove dublicate Data with Power Automate

    @TheGreatDensby

    • So on your second flow:

    First step should be List rows with Row Filter say your column where you want to verify duplicate against called AccountNumber, in other words account number is the column where you link BC and Dataverser

    Note in the left side of your Row Filter you  should be using column schema name(name) and not the display name:

     

    Mira_Ghaly_0-1635884104505.png

    The above step List Rows its intention to check if this row already exists in DataVerse or No and by a check on the length of the List Rows output you can decide delete existing rows:

    Expression at 1:

    length(outputs('List_rows')?['body/value'])

     

    Mira_Ghaly_0-1635884535235.png

     

     

  • thegreatdensby Profile Picture
    41 on 02 Nov 2021 at 10:41:05
    Re: Remove dublicate Data with Power Automate

    @Mira_Ghaly 
    Hey Mira, thanks for your fast reply πŸ™‚

     

    So your custom connector gets data and insert in dataverse? Can you share how it currently structured?

     

    -> I just build an automated flow, that called the custom connector, extraced the body and for every line, it created a new Dataverse Item with the dynamic properties. Thats working fine.

    What i than did, i build another flow and everytime a new line in the dataverse table is added it list all lines in that table. After that, he do a check if the added row no. is equal to each of the row numbers in the table.
    IF YES than he try to delete the line with the ODATA ID. But thats not really working . He dont get the right row ID to delete the line. 

  • Mira Ghaly Profile Picture
    11,409 Moderator on 02 Nov 2021 at 09:16:33
    Re: Remove dublicate Data with Power Automate

    @TheGreatDensby

    So your custom connector gets data and insert in dataverse? Can you share how it currently structured?

    According to how it is currently structured you may want to do a list rows to check if the row already exists in dataverse prior to create a new record or your idea still works.

    In either way you need list rows with OData filter and then a condition to check on the length of the List Rows and create new item if length of List Rows is 0.

     

Under review

Thank you for your reply! To ensure a great experience for everyone, your content is awaiting approval by our Community Managers. Please check back later.

Helpful resources

Quick Links

Telen Wang – Community Spotlight

We are honored to recognize Telen Wang as our August 2025 Community…

Announcing our 2025 Season 2 Super Users!

A new season of Super Users has arrived, and we are so grateful for…

Congratulations to the July Top 10 Community Leaders!

These are the community rock stars!

Leaderboard > Power Automate

#1
Tomac Profile Picture

Tomac 986 Moderator

#2
stampcoin Profile Picture

stampcoin 699 Super User 2025 Season 2

#3
Riyaz_riz11 Profile Picture

Riyaz_riz11 577 Super User 2025 Season 2

Loading complete