Unanswered

Read a big sharepoint list database

(1) Share

Report

Posted on by Community Power Pla...

Hello

I have an aplication developed in power apps, but the database is very very long, contains more than 1 millions rows. The database is in sharepoint list.

My difficulty is that within the application I need to display this data within a gallery, I created a collection but it takes a long time to read the data.
What do I need to do to make the application faster and more efficient.

Thanks

Categories:

Building Power Apps

I have the same question (0)

All responses (4)

Answers (0)

JayMagness 1,663 on at

Like (0)

Report

Hi,
this is a large dataset, but SharePoint can handle it.

Make sure you have the filter fields indexed... Also, may help to gather the data form the million in a Clearcollect.

Was this reply helpful? Yes No
pstork 62 on at

Like (0)

Report

The answer is to find a way to let users see the records they want to see using filtering. Its simply not practical to show that large a list all at one time for a couple of reasons.
1) As you've seen creating a local collection takes a long time. It also leads to data corruption since any changes made aren't reflected until you save the data back to the data source. So if someone else makes a change while you are working off the collection you won't see it and may overwrite it with your own change.
2) Galleries load records in pages of 100 to improve performance. You have to scroll to the bottom to get the next 100. So if your data source has 1,000,000 rows the user will need to scroll down in the gallery 10,000 times to see it all. That's simply not practical.

Was this reply helpful? Yes No
JayMagness 1,663 on at

Like (0)

Report

HI all,c

While I have never seen a situation as in Item #1 above, and number 2 is correct initially, but there are many ways. As an example, to do things. I have a system running here at a customer that has over 400,000 records and is indexed on a field with 47 characters. and I get a single record retrieval of less than 1 second. If you use clearcollect with your filter? you will get a page size in the gallery of 2,000 records or less. You can also chunk up data into sets by Week, Day, Month, Year that will also allow your table and system to run faster but would need to be in the use of ClearCollect. Also, you can clearcollect on the dataset and gather more than one data list in memory, once they are there you can splice them together, but it is all up to you... I have system running in many places in the world that are using all of the processes above. Also, if you are doing one then one clearcollect on a data set they process in random order so if you are putting them together after this process needs to be outside the Concurrent process.

Was this reply helpful? Yes No
takolota1 4,974 Moderator on at

Like (0)

Report

@JayMagness

You could also use something like this solution to quickly access, filter, & search large SP lists for Power Apps: https://powerusers.microsoft.com/t5/Community-App-Samples/Delegable-Large-SharePoint-List-App/td-p/2330721

Was this reply helpful? Yes No