PnP-Tools icon indicating copy to clipboard operation
PnP-Tools copied to clipboard

Sharepoint Search Query Tool Pagination of results

Open ITMonkey78 opened this issue 1 year ago • 3 comments

Sharepoint Search Query Tool v2.10.0 Results in the tool are limited to 500 regardless of row limit or rows per page entered. Pagination doesnt seen to work. Is this a limitation of the REST query or is the tool not paginating the results correctly?

Case Scenario: I have almost 10000 sites to crawl. I'm trying to return a list of sites that an account has access to through the Everyone permission (instructions are on the Microsoft Support site here)

The query: https://example.sharepoint.com/sites/Example_Site_SPO/_api/search/query?querytext='path%3ahttps%3a%2f%2fexample.sharepoint.com'&enableinterleaving=false&rowsperpage=500&rowlimit=500&clienttype='ContentSearchRegular'

My query returns Elapsed Time (ms): 1355

Primary Query Results:
	Total Rows: 74631
	Total Rows Including Duplicates: 97912
	

But the Raw tab contains only an xml or json object with 500 entries and the Primary Results lists only 500

I can modify the Start position to skip the first 500 and collect the next 500, but for 10000 results this is not ideal.

ITMonkey78 avatar Aug 10 '23 16:08 ITMonkey78

See https://learn.microsoft.com/en-us/sharepoint/dev/general-development/pagination-for-large-result-sets on how to use search to page large result sets. Or see https://www.techmikael.com/2023/08/how-to-paginate-large-results-sets-for.html to do the same using Graph API.

And page size is limited to 500, and has been for a long time.

wobba avatar Aug 10 '23 18:08 wobba

Im aware of how to use startrow to page through large data sets, as I stated in my last comment. The issue i have is I need to manually edit the startrow value in the Query Tool and re-run the query every time and for thousands of sites this means manually changing the value dozens of times.

With Graph API I can script the use of the @odata.nextLink returned in the results and loop the requests to collect them all, but this doesn't work in this tool.

What would be nice would be a Next Page button in the tool to page through the results automatically instead of needing to manually set the startrow value every time i need the next page of results.

On Thu, 10 Aug 2023, 19:59 Mikael Svenson, @.***> wrote:

See https://learn.microsoft.com/en-us/sharepoint/dev/general-development/pagination-for-large-result-sets on how to use search to page large result sets. Or see https://www.techmikael.com/2023/08/how-to-paginate-large-results-sets-for.html to do the same using Graph API.

— Reply to this email directly, view it on GitHub https://github.com/pnp/PnP-Tools/issues/333#issuecomment-1673748861, or unsubscribe https://github.com/notifications/unsubscribe-auth/AENCHN5EKSL5DJYLYPG7TD3XUUVRZANCNFSM6AAAAAA3LYTSWY . You are receiving this because you authored the thread.Message ID: @.***>

ITMonkey78 avatar Aug 13 '23 18:08 ITMonkey78

Happy to accept a PR for this and makes sense :)

wobba avatar Aug 14 '23 11:08 wobba