Skip to main content

Notifications

Community site session details

Community site session details

Session Id : v2tlsaiqCgTvZE9SMpl1pB
Power Apps - Power Apps Experimental Features
Unanswered

The performance of collecting multiple batches of 2000 rows from sharepoint is seriously impacted

Like (0) ShareShare
ReportReport
Posted on 5 Feb 2025 16:58:20 by 101
I am using this code to create a collection for my works CMS tool , activity has 60,000 rows , and when I went to switch on the New analysis engine , I had to switch off Collection access in component scope.
The impact is that the performance is seriously impacted , with users complaining about how slow the tool had become.
I have switched off New analysis engine and turned the Collection access in component scope back on , and the performance is back to the normal speeds. 
I am concerned that this feature is in Retired section and the impact it will have on my works CMS tool if removed.
 
Concurrent(
        ClearCollect( activity_00, Filter( Activity_List, ID_Code < 1 )),
        ClearCollect( activity_0, Filter( Activity_List, ID_Code >= 1 && ID_Code < 2000 )),
        ClearCollect( activity_1, Filter( Activity_List, ID_Code >= 2000 && ID_Code < 4000 )),
        ClearCollect( activity_2, Filter( Activity_List, ID_Code >= 4000 && ID_Code < 6000 )),
        ClearCollect( activity_3, Filter( Activity_List, ID_Code >= 6000 && ID_Code < 8000 )),
        ClearCollect( activity_4, Filter( Activity_List, ID_Code >= 8000 && ID_Code < 10000 )),
        ClearCollect( activity_5, Filter( Activity_List, ID_Code >= 10000 && ID_Code < 12000 )),
        ClearCollect( activity_6, Filter( Activity_List, ID_Code >= 12000 && ID_Code < 14000 )),
        ClearCollect( activity_7, Filter( Activity_List, ID_Code >= 14000 && ID_Code < 16000 )),
        ClearCollect( activity_8, Filter( Activity_List, ID_Code >= 16000 && ID_Code < 18000 )),
        ClearCollect( activity_9, Filter( Activity_List, ID_Code >= 18000 && ID_Code < 20000 )),
        ClearCollect( activity_10, Filter( Activity_List, ID_Code >= 20000 && ID_Code < 22000 )),
        ClearCollect( activity_11, Filter( Activity_List, ID_Code >= 22000 && ID_Code < 24000 )),
        ClearCollect( activity_12, Filter( Activity_List, ID_Code >= 24000 && ID_Code < 26000 )),
        ClearCollect( activity_13, Filter( Activity_List, ID_Code >= 26000 && ID_Code < 28000 )),
        ClearCollect( activity_14, Filter( Activity_List, ID_Code >= 28000 && ID_Code < 30000 )),
        ClearCollect( activity_15, Filter( Activity_List, ID_Code >= 30000 && ID_Code < 32000 )),
        ClearCollect( activity_16, Filter( Activity_List, ID_Code >= 32000 && ID_Code < 34000 )),
        ClearCollect( activity_17, Filter( Activity_List, ID_Code >= 34000 && ID_Code < 36000 )),
        ClearCollect( activity_18, Filter( Activity_List, ID_Code >= 36000 && ID_Code < 38000 )),
        ClearCollect( activity_19, Filter( Activity_List, ID_Code >= 38000 && ID_Code < 40000 )),
        ClearCollect( activity_20, Filter( Activity_List, ID_Code >= 40000 && ID_Code < 42000 )),
        ClearCollect( activity_21, Filter( Activity_List, ID_Code >= 42000 && ID_Code < 44000 )),
        ClearCollect( activity_22, Filter( Activity_List, ID_Code >= 44000 && ID_Code < 46000 )),
        ClearCollect( activity_23, Filter( Activity_List, ID_Code >= 46000 && ID_Code < 48000 )),
        ClearCollect( activity_24, Filter( Activity_List, ID_Code >= 48000 && ID_Code < 50000 )),
        ClearCollect( activity_25, Filter( Activity_List, ID_Code >= 50000 && ID_Code < 52000 )),
        ClearCollect( activity_26, Filter( Activity_List, ID_Code >= 52000 && ID_Code < 54000 )),
        ClearCollect( activity_27, Filter( Activity_List, ID_Code >= 54000 && ID_Code < 56000 )),
        ClearCollect( activity_28, Filter( Activity_List, ID_Code >= 56000 && ID_Code < 58000 )),
        ClearCollect( activity_29, Filter( Activity_List, ID_Code >= 58000 && ID_Code < 60000 )),
        ClearCollect( activity_30, Filter( Activity_List, ID_Code >= 60000 && ID_Code < 62000 ))
    );
 
    // Clear and collect all filtered data into a single 'activity' collection
    ClearCollect(activity,activity_00, activity_0, activity_1, activity_2, activity_3, activity_4, activity_5, activity_6, activity_7, activity_8, activity_9, activity_10, activity_11, activity_12, activity_13, activity_14, activity_15, activity_16, activity_17, activity_18, activity_19, activity_20, activity_21, activity_22, activity_23, activity_24, activity_25, activity_26, activity_27, activity_28, activity_29, activity_30);
  • iAm_ManCat Profile Picture
    18,206 Most Valuable Professional on 06 Feb 2025 at 14:32:15
    The performance of collecting multiple batches of 2000 rows from sharepoint is seriously impacted
    As I said, I think it would be better to re-architect. Resolve the delegation issues and that should mean you can refer to the datasources directly instead of trying to cache large volumes of data.
     
    Can you share which parts are showing as non-delegable and what the field types are in each case?
  • JD_26 Profile Picture
    101 on 06 Feb 2025 at 08:17:59
    The performance of collecting multiple batches of 2000 rows from sharepoint is seriously impacted
    Thanks for your response.
    I have to create collections , due to the delegation issues.
    My issue is that I have CMS built from 4 different sharepoint list
    Master , where a new organization is added , the ID from that is then used
    to connect the other 3 list
    I have a supplier list , which can have up to 15 different suppliers per organization
     
    The large list is the activity list , which shows activity for each of the suppliers
    The CMS consist of galleries and forms , with filters.
    When a user selects an organization it then populates all the matched suppliers in a gallery
    Then when one of those is selected then the activity is shown in another gallery. 
    This is why there are 60,000 rows of activity 1200 orgs , 6000 suppliers , and 60,000 activity records.
     
    the galleries are populated with this code
    suppliers
    Filter(
        Master_Profiles,
        account_id = master_search.Selected.ID,
        (combo_sm_rep.Selected.Result = Blank() || combo_sm_rep.Selected.Result = "All" || 'SM Rep' = combo_sm_rep.Selected.Result),
        (combo_dataset.Selected.Value = "All" || Dataset = combo_dataset.Selected.Value),
        (combo_status.Selected.Value = "All" || Status = combo_status.Selected.Value)
    )
    activity
    SortByColumns(
            Filter(
                activity,
                'Dataset Subtype' = supplier_search.Selected.'Dataset Subtype' &&
                DupeID = supplier_search.Selected.'Dupe ID' &&
                AccountID = supplier_search.Selected.account_id
            ),
            "ID",
            SortOrder.Descending
        )
       
  • iAm_ManCat Profile Picture
    18,206 Most Valuable Professional on 06 Feb 2025 at 07:51:25
    The performance of collecting multiple batches of 2000 rows from sharepoint is seriously impacted
    Hiya,
     
    Regardless of whether or not the new analysis engine is slowing down your data retrieval, I would strongly suggest re-architecting this as soon as you can - you should not be loading more than a small number of records in locally, as that's performant design practice.
     
    Could you explain the reason that 60k records are needing to be pulled in instead of referring to the datasource directly - is it because of offline usage?
     
    Cheers,
    Sancho

Under review

Thank you for your reply! To ensure a great experience for everyone, your content is awaiting approval by our Community Managers. Please check back later.

Helpful resources

Quick Links

Announcing the Engage with the Community forum!

This forum is your space to connect, share, and grow!

🌸 Community Spring Festival 2025 Challenge Winners! 🌸

Congratulations to all our community participants!

Warren Belz – Community Spotlight

We are honored to recognize Warren Belz as our May 2025 Community…

Leaderboard > Power Apps - Power Apps Experimental Features

#1
mmbr1606 Profile Picture

mmbr1606 51 Super User 2025 Season 1

#2
MS.Ragavendar Profile Picture

MS.Ragavendar 28

#3
WarrenBelz Profile Picture

WarrenBelz 21 Most Valuable Professional

Overall leaderboard

Featured topics

Loading started