How to? Remove duplicate rows from a Sharepoint List.... Easily or simply?

(0) Share

Report

Posted on by SuperDude123

Hi all,

Id like to be able to create a Flow that would remove rows where a single column entry (a unique ID) is a duplicate.

In this case its an Incident number, so that would be INC123456 for example.

How can this be done?

Ive googled until all the hyperlinks have turned purple. This seems to be the closest I have come to some success.

https://tomriha.com/how-to-delete-duplicate-items-in-a-sharepoint-list-with-power-automate/#:~:text=Remove%20only%20the%20duplicates&text=Use%20the%20skip(%E2%80%A6),another%20'Apply%20to%20each'.&text=Then%20just%20delete%20all%20the%20remaining%20items%20using%20their%20ID.&text=At%20the%20end%20of%20the,unique%20items%20in%20the%20list.

Still, this doesn't work for me, not a single item is being deleted from my list.

Could someone help with a fool proof way for automating this? I don't want to concatenate any of the columns or anything like that. You would think that something like this should be simple ? Why would we want to complicate things further like this?

https://www.youtube.com/watch?v=SGXAqAzYUSM

Categories:

Building flows

I have the same question (2)

All responses (21)

Answers (2)

Verified answer

takolota1 4,974 Moderator on at

Like (0)

Report

@SuperDude123

Here is a template that finds & removes the smallest/oldest or largest/most recent duplicates from a list that contains up to 100,000 items.

https://powerusers.microsoft.com/t5/Power-Automate-Cookbook/Find-and-Remove-Duplicates/td-p/2191403

Note: Version 1.2 is much faster.

Was this reply helpful? Yes No
takolota1 4,974 Moderator on at

Like (0)

Report

@SuperDude123

Here is a template that finds & removes the smallest/oldest or largest/most recent duplicates from a list that contains up to 100,000 items.

https://powerusers.microsoft.com/t5/Power-Automate-Cookbook/Find-and-Remove-Duplicates/td-p/2191403

Was this reply helpful? Yes No
SuperDude123 47 on at

Like (0)

Report

Hello Takolata and thanks for replying. I have already tried this template.
At step "The middle actions will process the records, identify the duplicate records, & return a JSON array of only duplicate records." I am unable to follow the instructions. How do I create the JSON at this stage? ITs not part of any defined variables. Are you able to clarify ?
Thanks

Was this reply helpful? Yes No
SuperDude123 47 on at

Like (0)

Report

Please see the snip here, for example I understand that I need to type that expression at the bottom of the snip. How do I create the top two expressions?

Was this reply helpful? Yes No
takolota1 4,974 Moderator on at

Like (0)

Report
It’s

item()?['RecordJSON']

and

removeProperty(item(), 'RecordJSON')

But you could also have just downloaded the zip file at the end of the post & imported that into Power Automate.

Was this reply helpful? Yes No
SuperDude123 47 on at

Like (1)

Report

Thank you Takolota. I will look at this template again.
I had already tried to download and upload the template as you suggested.
I tried this a few times over the past couple of days, and once uploaded I couldnt fine the template I had uploaded.
I tried to search cloud templates for example, but it just wasnt obvious to me where the template had gone once it had been uploaded.
Many thanks again for your response, I will try to complete the template again using your comments above.

Was this reply helpful? Yes No
SuperDude123 47 on at

Like (0)

Report

Hello again!
Please see the attached screenshot.
Body is empty and it doesnt look like Delete Item is being triggered.
I assumed that isDuplicate (Filter array Get duplicate records) should be item()?['isDuplicate'] (correct?)
This is pretty much the same problem I had when I tried to follow the below link. Delete is not being triggered.
https://tomriha.com/how-to-delete-duplicate-items-in-a-sharepoint-list-with-power-automate/#:~:text=....
Should Body be empty?
Thanks in advance!

Screenshot 2023-0...

Was this reply helpful? Yes No
takolota1 4,974 Moderator on at

Like (0)

Report

@SuperDude123

I used IsDuplicate with a capital I. Does your creation expression & the reference from your picture match? Because these are case sensitive.

Was this reply helpful? Yes No
SuperDude123 47 on at

Like (0)

Report

Thanks, I have changed to upper case capital I.
Still not working.sadly.
Do I need to map additional properties? I am only mapping one property which is the incident number.

Screenshot.png

Was this reply helpful? Yes No
SuperDude123 47 on at

Like (0)

Report

BTW the screenshot shows Apply to each as nested twice. I have confirmed this is not the case, please ignore previous screenshot.

Screenshot.png

Was this reply helpful? Yes No