Unanswered

Searching Large Document Library in SharePoint

(0) Share

Report

Posted on by GerryBrimacombe

I am using Flow to update a large SharePoint list (20,000 items) with links to a large document library (30,000 folders and files). I am having some strange behaviour with odata filtering with "startswith". It seems to work inconsistently, or maybe I am missing something.

I am using the action "Get Files (properties only" and using the odata filter "startswith(FileLeafRef,'Full-Report-Number')" where Full-Report-Number is a variable set earlier. As mentioned, this works inconsistently. I suspect that the 5,000 view limit may be affecting me, but in the past when I've bumped into that I've got an explicit error message. This time, it just returns a success code "200" and no items. I thought about setting a view or filtering by an indexed column, but haven't come up with a way to do that yet (I can't add new indexes now bevuase the library is too big, and the files I am looking for could be anywhere and modified anytime, so my view options are limited.)

Here's the action:

And here's what it looks like after running:

And here's the Report library search, showing that there are files (and in some cases folders) starting with NS102:

filter 3.jpg

Really scratching my head on this one. Thanks for your help.

Gerry

Categories:

Building flows

I have the same question (0)

All responses (5)

Answers (0)

v-yamao-msft on at

Like (0)

Report

Hi @GerryBrimacombe ,

What’s the error message returned before?

Please make sure the variable Full-Report-number return matched value.

I have made a simple test on my side, which is working successfully.

Here I have seen a blog, which could be a reference for you:

https://www.chakkaradeep.com/2018/05/01/deep-dive-into-get-items-and-get-files-sharepoint-actions-in-microsoft-flow/

Best regards,

Mabel

Was this reply helpful? Yes No
GerryBrimacombe 12 on at

Like (0)

Report

Thank you for testing and commenting, Mabel.

Yes, I have read that article - very useful.

I am not surprised your example works - as mentioned, I am getting inconsistent results which is driving me mad. 🙂 Yes, the variable "Full-Report-Number" is being set correctly, so I don't think that's the issue.
I am not getting any error message, just a return code of "200" and and an empty result set. Is there a method or place I should check to get more detailed error messages or logging?

Thank you,
Gerry

Was this reply helpful? Yes No
GerryBrimacombe 12 on at

Like (0)

Report

A bit more testing and debugging...

I'm not sure if this means anything more, but here is the returned Outputs section of the Get Files (properties only) action:

Here's the full text of the BODY section above:
{
"@odata.nextLink": "https://flow-apim-msmanaged-na-eastus2-01.azure-apim.net/apim/sharepointonline/shared-sharepointonl-fabf0e50-484e-4907-901f-123dfd118c33/datasets/https%253A%252F%252Fdrcoell.sharepoint.com%252F/tables/d144e7b2-84bd-4818-b5e6-8bad50a53e28/getfileitems?$filter=startswith(FileLeafRef%2c%27NS102%27)&$skiptoken=Paged%3dTRUE%26ix_Paged%3dTRUE%26ix_ID%3d5424",
"value": []
}

Was this reply helpful? Yes No
v-yamao-msft on at

Like (0)

Report

Hi @GerryBrimacombe ,

Not sure if it is caused by the large amount files contained in the library.

Could you please try getting files (properties only) from a library which doesn’t have too many files?

Best regards,

Mabel

Was this reply helpful? Yes No
AlanPD 2 on at

Like (0)

Report

I am running into exactly the same issue. I have a list and two document libraries. And I am trying to add links to the list to the related folders in the document libraries. I noticed this behaviour, where the Get Files returns successful but does not return results (when there folders matching the query parameters.) The scripts running on the 2 document libraries are identical and while both have suffered from this issue it is far worse in the larger document library!

Was this reply helpful? Yes No