Power Platform Community Forum Thread Details

I found this flow to check for duplicates and delete them, the problem is the flow deletes all entries from the list not just one of the duplicates. Any help with tweaking this?

Categories:

General topics

This does not work because you have a full list in "Get Items" and when you are deleting the duplicates using ID from "Get items 2" that is fine, but then, the first "Apply to each" still contains the duplicates, so when they are processed and you "Get Items 2" you'll get the only item left, and the title and status will match but IDs will be diffrent, so you removed your "surviving" item.

For example:

Items

1
2
3 - Dup of 1
4 - Dup of 1
5
6 - Dup of 5
7

Item 1 is in the first apply to each, and then you "get items 2" which will remove Items 3 & 4.

Your list is like this

Items

1
2
5
6 - Dup of 5
7

the next item to proces is ID 2 and will find no duplicates and your list will remain the same.

Next, remember that your "Apply to each" still has the full list since it downloaded the data at the beginning of the flow, meaning that it will process item 3 (which was already deleted in SharePoint). When you "Get items 2" it will retrieve the up to date list, but item 3, is a duplicate of item 1, so this cycle will remove item 1. After all that your final output should be only the items that did not have duplicates

Items

2
7

If you want to use this approach you could:

Keep track of the "Get item" IDs that removed duplicates, so that if you can check and if it is in your array, do not delete it because it is the surviving item.
Delete the ID of "Get items" instead and catch the error or track the deleted IDs
Instead of retrieving the full Task list, filter is with the columns you need to check for.

Another issue with this, is that performance is going to be really bad, this other solution is more complex, but will be faster

Find and Remove Duplicates - Power Platform Community (microsoft.com)

Last thing to consider is your list size, wish might cause memory or limit issues, maybe?