Skip to main content

Notifications

Community site session details

Community site session details

Session Id :
Power Apps - Building Power Apps
Answered

ClearCollect pulling duplicates from SharePoint list when using filter function

(1) ShareShare
ReportReport
Posted on by 143

I'm pulling my datasource into a temporary collection so that I can patch all changes in one go when the user submits the form. I've based my design on Reza Dorani's Editable Grid in Power Apps video: YouTube

 

When loading the data (either editing an existing record or after saving a new one), the collection is pulling duplicates. The only issues I can find online that seem similar is this one Android: ClearCollect resulting in duplicate entri... - Power Platform Community (microsoft.com)

My problem is occurring in the browser as well though.

 

I've altered Reza Dorani's setup slightly, because I need to fetch my records based on a value, as I'm filling this data source with 'child' items of the main list. The button loading my data contains the following OnSelect:

ClearCollect(colGridData,Filter(TC_commitment,checklistID.Id = lastForm));
 
lastForm is filled based on the form that's force submitted in a previous step by using: TransactionChecklistForm.LastSubmit.ID

 

You can see the item ID in the comment column (added it for troubleshooting purposes). All are duplicated except for #6 somehow.

Maarten_DIF_0-1719829314338.png

 

Any insight on how to remedy this is greatly appreciated.

 

  • Verified answer
    Maarten_DIF Profile Picture
    143 on at
    Re: ClearCollect pulling duplicates from SharePoint list when using filter function

    I followed a different tutorial which does things slightly differently and the issue did not occur there:

    Mastering SharePoint List Relationships in Power Apps | Create Responsive Repeating Tables (youtube.com)

  • DBO_DV Profile Picture
    4,530 Super User 2025 Season 1 on at
    Re: ClearCollect pulling duplicates from SharePoint list when using filter function

    You did It like this because of Performance gaining? 

    You don't really need to do that here since for every new search you would collect again from the Sharepoint list. It doesn't really make a difference if you use a collection or directly the table from performance standpoint, in this scenario. 

    Collections are useful if you need just a few columns (Not records) and you need to reuse them over and over in different screens. But since you are only editing them or deleting them and it's not a couple Hundred rows you should be perfectly fine with the direct query. 

     

    otherwise with a collection you would need to use a for all at the end to ensure your changes are also in Sharepoint. ForAll is performance wise a bigger problem.

     

  • Maarten_DIF Profile Picture
    143 on at
    Re: ClearCollect pulling duplicates from SharePoint list when using filter function

    Wouldn't I have to make a lookup for each column separately then? That seems to defeat the purpose of the performance gain with the temporary collection.

     

    I'm contemplating to go back to the usual 'direct edit/query' on the SharePoint list.

     

    To understand your solution better, what would I put in my columns' Default(SelectedItems) and how would I fill the collection that is input for the patch function?

     

    Many thanks for your help so far!

  • DBO_DV Profile Picture
    4,530 Super User 2025 Season 1 on at
    Re: ClearCollect pulling duplicates from SharePoint list when using filter function

    Let me explain: so you can Put this in a Gallery as Items: 

    Distinct(Filter(TC_commitment,checklistID.Id = lastForm),Unique Identifier)

    you 'll  have a Table with one column Named Value. 

     

    with this one column you can make changes on other columns of the same record.

     

    This way could help you solve the error since Distinct will always hold back any duplicates.

     

     

  • Maarten_DIF Profile Picture
    143 on at
    Re: ClearCollect pulling duplicates from SharePoint list when using filter function

    Not sure if I follow. I tried to build something in the Load Data button based on your comment but it won't compile:

     

    With({FilteredData:Distinct(TC_commitment,ID)},
    LookUp(TC_commitment,ID = ThisItem.Value,FilteredData)

     

     

    Also tried this, but it gives incompatible data type (table vs number):

     

    With({FilteredData:Distinct(TC_commitment,ID)},
    ClearCollect(colGridData,LookUp(TC_commitment, FilteredData.Value = ThisRecord.ID)

     

     

    EDIT:

    Changed it to this and that did compile, but it's not pulling in any results:

    Distinct(Filter(TC_commitment,checklistID.Id = lastForm),ID);
    ClearCollect(colGridData, LookUp(TC_commitment, ID = ThisRecord.ID)

     

  • DBO_DV Profile Picture
    4,530 Super User 2025 Season 1 on at
    Re: ClearCollect pulling duplicates from SharePoint list when using filter function

    I have an idea. 

     

    You could use the distinct function. Distinct on ID . This will give you a table with just one column called Value. 

    Distinct(Filter(TC_commitment,checklistID.Id = lastForm),Unique Identifier)

    Then you can use a LookUp to get the rest:

    LookUp(TC_commitment, UniqueIdentifier=THisItem.Value)

     

     

     

  • Maarten_DIF Profile Picture
    143 on at
    Re: ClearCollect pulling duplicates from SharePoint list when using filter function

    @DBO_DV didn't get any results with that.

     

    Did some more testing with the following code in ClearCollect:

    Filter(TC_commitment,checklistID.Id = lastForm)

     

    To ensure no funny business with the editor, I published my version and went in as 'user'.

     

    The issue seems to occur consistently when there are less than 6 rows in the child list. Seems like an arbitrary number, but the behavior is consistent. When there's 6 or more rows, the app behaves as it should. I'm really stumped on this one.

  • DBO_DV Profile Picture
    4,530 Super User 2025 Season 1 on at
    Re: ClearCollect pulling duplicates from SharePoint list when using filter function

    could you try something else? 

    use the Distinct function. This will display only one time the same Value :

    With({FilteredData:Filter(TC_commitment,checklistID.Id = lastForm)},
    ClearCollect(colGridData,Distinct(FilteredData,Id))
    )

     

  • Maarten_DIF Profile Picture
    143 on at
    Re: ClearCollect pulling duplicates from SharePoint list when using filter function

    One thing I noticed on the duplicate records is that one seems to be from SharePoint, because it contains all the data like created time/by, whilst the other seems to be only created in the temporary collection (colGridData) and it only contains the data points that are added by the app (ID, amount, category in this snip):

    Maarten_DIF_0-1719912078279.png

     

    Some additional information, which might help troubleshooting:

    The columns in the gallery template all have the following in their OnChange: Select(Parent)

    The Gallery OnSelect is as follows:

    Patch(colGridData,ThisItem,
     {
     ID: ThisItem.ID,
     checklistID:{'@odata.type':"#Microsoft.Azure.Connectors.SharePoint.SPListExpandedReference",
     Id:ThisItem.checklistID.Id,
     Value:ThisItem.checklistID.Value},
     rowNumber: Value(FundCommitments_rowNumber.Text),
     category: {Value: "Fund Commitments"},
     commitmentType: {Value: FundCommitments_commitmentType.Selected.Value},
     amount: Value(FundCommitments_amount.Text),
     comment: FundCommitments_comment.Text
     }
    );

     

    The (later invisible) button btnLoadDataTCCommitment has the following OnSelect:

    ClearCollect(colGridData,Filter(TC_commitment,checklistID.Id = lastForm));

    The (later invisible) button btnSaveDataTCCommitment has the following OnSelect:

    Patch(
     TC_commitment,
     UpdateIf(
     colGridData,
     Created = Blank(),
     {ID: Blank(),checklistID: {Id:lastForm,Value:lastForm}}
     )
    );
    Remove(TC_commitment,colDelete);
    Clear(colDelete);
    Select(btnLoadDataTCCommitment);

    The trash icon in the gallery has the following OnSelect:

    Collect(colDelete,ThisItem);
    Remove(colGridData,ThisItem);

    Explanation of the SaveData:

    It patches the SharePoint list (TC_commitment) with the temporary collection colGridData from the gallery, but first updates the ID and checklistID if it's an item that does not yet exist in SharePoint.

    It removes any items that were added to the colDelete collection by the user clicking the trash icon.

    After patching, it loads the data fresh from the SharePoint list by selecting the load button.

  • Maarten_DIF Profile Picture
    143 on at
    Re: ClearCollect pulling duplicates from SharePoint list when using filter function

    Hi @DBO_DV, still the same unfortunately.

     

    Tried to create a new entry entirely and behavior is also the same. The filtering by checklistID.Id = lastForm works as expected with any of the expressions listed in this thread, it's just the entries for the specific checklistID that are duplicating.

    I do notice that sometimes it duplicates all of them, and sometimes the first ID is not duplicated (like ID 6 in the initial screenshot).

Under review

Thank you for your reply! To ensure a great experience for everyone, your content is awaiting approval by our Community Managers. Please check back later.

Helpful resources

Quick Links

🌸 Community Spring Festival 2025 Challenge 🌸

WIN Power Platform Community Conference 2025 tickets!

Markus Franz – Community Spotlight

We are honored to recognize Markus Franz as our April 2025 Community…

Kudos to the March Top 10 Community Stars!

Thanks for all your good work in the Community!

Leaderboard

#1
WarrenBelz Profile Picture

WarrenBelz 146,524 Most Valuable Professional

#2
RandyHayes Profile Picture

RandyHayes 76,287 Super User 2024 Season 1

#3
Pstork1 Profile Picture

Pstork1 65,906 Most Valuable Professional

Leaderboard