powershell icon indicating copy to clipboard operation
powershell copied to clipboard

[FEATURE] Add PageSize option to more cmdlets

Open sg1888 opened this issue 1 year ago • 2 comments

Is your feature request related to a problem? Please describe. I would like to have -PageSize as an option for the following cmdlets: Get-PnPFileInFolder, Get-PnPFolderItem, and Get-PnPFolderInFolder. Right now, PageSize is not supported.

I have a large Sharepoint library with > 4 million files. Right now I have to use Get-PnPListItem, which loads the entire site. This is incredibly slow and memory intensive. I tried using Get-PnPListItem -List "Documents" -FolderServerRelativeURL "/Sites/Url/", but it doesn't seem to work when PageSize is specified.

Describe the solution you'd like I would like to be able to specify PageSize for the aforementioned cmdlets so that they can be used on large folders and subfolders.

Describe alternatives you've considered I've considered buying 256GB of RAM and carving out 2-3 weeks to let the script work, but there has to be a better way.

sg1888 avatar Aug 07 '24 23:08 sg1888

To avoid loading the whole list/library in memory, use the -ScriptBlock parameter of Get-PnPListItem

The script block to run after every page request.

It accepts script block like { Param($items) } where $items contains as many items as specified in -PageSize . I was able to process 1.5 M files in a library this way without any RAM issues.

jackpoz avatar Aug 08 '24 20:08 jackpoz

To avoid loading the whole list/library in memory, use the -ScriptBlock parameter of Get-PnPListItem

The script block to run after every page request.

It accepts script block like { Param($items) } where $items contains as many items as specified in -PageSize . I was able to process 1.5 M files in a library this way without any RAM issues.

Thanks. I didn't think of using ScriptBlock - I will try that. I do think that adding PageSize to more cmdlets would be helpful as scriptblocks are a bit kludgy, especially if you need to reference other variables.

sg1888 avatar Aug 09 '24 23:08 sg1888

hi @sg1888 , this has been fixed/added in the latest nightly builds. Closing this.

gautamdsheth avatar Dec 25 '24 14:12 gautamdsheth