web
You’re offline. This is a read only version of the page.
close
Skip to main content

Announcements

Welcome to the Power Platform Communities
News and Announcements icon
Community site session details

Community site session details

Session Id :
Power Platform Community / Forums / Power Apps / Collections maxes out ...
Power Apps
Answered

Collections maxes out at 2000 when populating with SharePoint list datasource

(0) ShareShare
ReportReport
Posted on by CLS720 49

I have an app where on the OnView property for a screen, a collection is created from a SharePoint list datasource.

ClearCollect(CompletedRequestsCollection, CompletedRequests)

I then try to work around the delegation issue  within a gallery by sorting/filtering the collection that is created from OnView. However, even though there is no filtering on this datasource and there is no indication of non-delegation, the collection that is created OnView always maxes out with the first 2000 items in the list. Is this expected? Are there any workarounds?

Categories:
Building Power Apps
  • Verified answer
    Drrickryp Profile Picture
    Drrickryp Super User 2024 Season 1 on at

    HI @CLS720,

    Collect() and ClearCollect() are not delegatable so they max out at 2000k items.  However, once a collection is created, all powerapps functions can be delegated to the collection including Search() and all numeric functions.   @martinav and I (well mostly him) came up with a way to rapidly collect a maximum of 4000 items from a Sharepoint list.  As long as the list doesn't exceed 4000 items it will work and it works by collecting the first 2k items then sorting the list descending and collecting the bottom 2k items.  Then merging the lists and then removing duplicates.  It works this way.: In the OnVisible property of the screen:

    Concurrent(
ClearCollect(List1,Filter(SortByColumns(List,"ID",Ascending),ID>0)),
ClearCollect(LIst2,Filter(SortByColumns(List,"ID",Descending),ID>0))
);

    So two collections (List1 and List2) are created concurrently.  This greatly increases the speed of the process.

    Then the two two collections are merged, elimininating duplicates, into ListAll and the two collections, List1 and List2 are emptied.  Just add this formula to follow the semicolon in the upper formula box. 

    ClearCollect(ListAll,List1,Filter(List2, Not(ID in List1.ID)));Clear(List1);Clear(List2)

     ListAll now contains up to 4000 unique rows and can be used with all Powerapps functions without regard to delegation,  and as an items property for galleries,  datatables, and Comboboxes (but not Dropdowns as these are hardcode limited to a maximum number of items, no one wants to scroll through thousands of items in a dropdown anyway.).    The execution speed of the formulas like Search() or Lookup() will astonish you. 

  • martinav Profile Picture
    martinav 3,347 on at

    Hey  @Drrickryp,  THanks for the tag!  It was a combined effort, but I thought you came up with the first/last scenario... anyhwo, it is working very well.  I have a list with 2,400 items, and it does indeed work well.  Also, by using the ShowColumns() command, you can pull in only the columns you wish to have in the collection.  If you dont need it, no sense in wasting time and memory space with data you dont need.  

     

    Here are my actual commands, which I put in the "OnStart" control for the app.  

    Concurrent(
ClearCollect(MDL_List1,Filter(ShowColumns(SortByColumns(Master_Drawing_List,"ID",Ascending),"ID","MD_PartNumber","MD_Project","MD_LatestRevNumber","MD_DrawingRevDate","MD_EngrgRelDate","Title","MD_PDM_Link","MD_LinkToDrawing","MD_IsActive","MD_Data_Flagged"),ID>0)),
    
ClearCollect(MDL_List2,Filter(ShowColumns(SortByColumns(Master_Drawing_List,"ID",Descending),"ID","MD_PartNumber","MD_Project","MD_LatestRevNumber","MD_DrawingRevDate","MD_EngrgRelDate","Title","MD_PDM_Link","MD_LinkToDrawing","MD_IsActive","MD_Data_Flagged"),ID>0))
);

ClearCollect(MDL_All,MDL_List1,Filter(MDL_List2, Not(ID in MDL_List1.ID)));Clear(MDL_List1);Clear(MDL_List2)

    So, everything is pulled one time.  I do have a data refresh button that executes the exact same commands if the user thinks his data is stale.  I also have a timer that runs, and reloads the data every 30min.  If you dont do that, since data in your collection does not update automatically, you might want to recollect the data from sharepoint regularly.  It will depend on your specific use.  Also, note that I cleared the two collections used to pull in the two separate lists.  To minimize memory usage.

     

    This really did work well.  Much better than I could have hoped!  I did have another idea..  To perhaps get more items.  I was wondering if can collect 2,000 items at a time using the ID of the sharepoint list.  (if(ID>2000 AND ID<4000) etc etc. It might take using a Flow, though, because I think that using gt or lt functions are not delegable.  However, flow allows you to to get items, and use ID.  I think you can do get items using the get items filter.  Then sending back to powerapps using the Response action.  I have no idea how long it would take.   

     

  • CLS720 Profile Picture
    CLS720 49 on at

    Thank you both! I applied this solution and it worked beautifully.

  • martinav Profile Picture
    martinav 3,347 on at

    Sweet!  Another happy customer!

  • Community Power Platform Member Profile Picture
    Community Power Pla... Microsoft Employee on at

    Hey Guys, can you help me please?

     

    I have a excel spreadsheet wich cointains 37k rows and i want to bring it to powerapps.

     

    the data does not change, so i uploaded the excel via import via excel, and when i collect it brings me 15k rows.

     

    but how can i separate in 15k15k and another one with 8k to bring all the datas?

  • martinav Profile Picture
    martinav 3,347 on at

    Hold that thought...  while you are waiting... what commands/objects are you using?

  • Community Power Platform Member Profile Picture
    Community Power Pla... Microsoft Employee on at

    actually @timl figured out a way to do this with.

     

    spread the excel table in 3 parts then collect all of them in powerapps.

     

    worked 😉

     

    thanks!  

  • ubikwitous Profile Picture
    ubikwitous Microsoft Employee on at

    Seems to work @Drrickryp and @martinav , but more fun!

     

    I have a SharePoint list with thousands of items, but I only want to pull items where the Start Date is greater than today. New items will come in over time. Is there a way to do this? I tried adding the Filter in, but with no change.

     

    Concurrent(
    ClearCollect(List1,Filter(SortByColumns('Events','Start Time'>Now(),"ID",Ascending),ID>0)),
    ClearCollect(List2,Filter(SortByColumns('Events','Start Time'>Now(),"ID",Descending),ID>0))
    );
    ClearCollect(ListAll,List1,Filter(List2, Not(ID in List1.ID)));Clear(List1);Clear(List2)

     

    I have a feeling I'm going to have to somehow create a Flow to keep a 2nd copy of the SharePoint List that is cleaned up. Any advice would be appreciated - thanks!

  • martinav Profile Picture
    martinav 3,347 on at

    @ubikwitous ,

     

    Filter first, then sort.  The parenthesis work from the inside out.  

     

    Concurrent(
ClearCollect(List1,SortByColumns(Filter('Events','Start Time'>Now()),"ID",Ascending),ID>0)),
ClearCollect(List2,SortByColumns(Filter('Events','Start Time'>Now()),"ID",Descending),ID>0))
);
ClearCollect(ListAll,List1,Filter(List2, Not(ID in List1.ID)));Clear(List1);Clear(List2)

    I didnt actually try this, I think the syntax is all ok.  But, this may not work because the ">" will be a delegation issue itself. This may not work.

     

    Getting 2,000 rows first, and then filtering would look like this... (I think)

     

    Concurrent(
ClearCollect(List1,Filter(SortByColumns('Events',"ID",Ascending),'Start Time'>Now()),ID>0)),
ClearCollect(List2,Filter(SortByColumns('Events',"ID",Descending),'Start Time'>Now()),ID>0))
);
ClearCollect(ListAll,List1,Filter(List2, Not(ID in List1.ID)));Clear(List1);Clear(List2)

    I'm not sure about your syntax for 'Start Time'.  You also will need to be sure you use time zone correction to get your time adjusted to UTC().

  • ubikwitous Profile Picture
    ubikwitous Microsoft Employee on at

    Thanks for the reply @martinav !

     

    I still can't get it working. I've tried going back to the original, but still no luck. Is there a way to refresh the data set post updating? No matter what I do, it is only showing the older events and nothing newer, which leads me to believe it just isn't pulling them. Thanks!

     

    For onVisible:

    Concurrent(
    ClearCollect(List1,Filter(SortByColumns('Events',"ID",Ascending),ID>0)),
    ClearCollect(List2,Filter(SortByColumns('Events',"ID",Descending),ID>0))
    );
    ClearCollect(CollectionNew,List1,Filter(List2, Not(ID in List1.ID)));Clear(List1);Clear(List2)

     

    On the BrowseGallery, I have:

    Collect(CollectionNew, BrowseGallery1.AllItems)

     

    And for Items:

    SortByColumns(
        Search('Events',
            TextSearchBox1.Text,
            "Location"
        ),
        "Title",
        If(
            SortDescending1,
            SortOrder.Descending,
            SortOrder.Ascending
        )
    )

Under review

Thank you for your reply! To ensure a great experience for everyone, your content is awaiting approval by our Community Managers. Please check back later.

Helpful resources

News and Announcements

Welcome to the Power Platform Communities

Quick Links

Season of Sharing Community Challenge Winners!

Congratulations to our community stars!

Kudos to our 2025 Community Spotlight Honorees

Expanding mentorship, skilling, and AI innovation

Congratulations to the July Top 10 Community Leaders!

These are the community rock stars!

Leaderboard > Power Apps

#1
11manish Profile Picture

11manish 394 Super User 2026 Season 2

#2
Mohsin Ali Profile Picture

Mohsin Ali 328

#3
WarrenBelz Profile Picture

WarrenBelz 324 Most Valuable Professional

Last 30 days Overall leaderboard

Featured topics

Auto-Populate field Power Pages
Gallery Tabs - Show or Hide Tabs based on Field Value
How to patch multiple records at once
Enable Copilot for end users in model-driven apps
Question for everyone : How are you using Create Text GPT in AI Builder?
Community content - sample components, blogs etc. - Link to this page (http://aka.ms/PCFdemos)
Welcome to the Power Apps forum!

Product updates

Microsoft Power Platform Community release plans