Stream too long when trying to download very large file

[fazleskhan]

I am attempting to host a very large file for HTTP download via Pode and am encountering the following exception

`PS E:\git\navision-docker-container\navlive-backup-copied> pode start VERBOSE: Adding Route: [Static] /backup[/]{0,1}(?.*) Listening on the following 1 endpoint(s) [1 thread(s)]: - http://xxx.xx.x.xxx:xxxx/ Date: 2022-09-13 12:34:39 Level: Error ThreadId: 1 Server: xxxxxxx Category: NotSpecified: (:) [Set-PodeResponseAttachment], MethodInvocationException Message: Exception calling "Write" with "3" argument(s): "Stream was too long." StackTrace: at Set-PodeResponseAttachment, C:\Program Files\WindowsPowerShell\Modules\Pode\2.7.1\Public\Responses.ps1: line 115 at <ScriptBlock>, <No file>: line 97

Date: 2022-09-13 12:34:39 Level: Error ThreadId: 1 Server: xxxxxx Category: System.Management.Automation Message: Exception calling "Write" with "3" argument(s): "Stream was too long." StackTrace: at System.Management.Automation.ExceptionHandlingOps.CheckActionPreference(FunctionContext funcContext, Exception exception) at System.Management.Automation.Interpreter.ActionCallInstruction2.Run(InterpretedFrame frame) at System.Management.Automation.Interpreter.EnterTryCatchFinallyInstruction.Run(InterpretedFrame frame) at System.Management.Automation.Interpreter.EnterTryCatchFinallyInstruction.Run(InterpretedFrame frame) at System.Management.Automation.Interpreter.Interpreter.Run(InterpretedFrame frame) at System.Management.Automation.Interpreter.LightLambda.RunVoid1[T0](T0 arg0) at System.Management.Automation.PSScriptCmdlet.RunClause(Action1 clause, Object dollarUnderbar, Object inputToProcess) at System.Management.Automation.PSScriptCmdlet.DoEndProcessing() at System.Management.Automation.CommandProcessorBase.Complete()

Date: 2022-09-13 12:34:39 Level: Error ThreadId: 1 Server: xxxxx Category: mscorlib Message: Stream was too long. StackTrace: at System.IO.MemoryStream.Write(Byte[] buffer, Int32 offset, Int32 count) at CallSite.Target(Closure , CallSite , Object , Object , Int32 , Object )

172.26.4.26 - - [13/Sep/2022:12:34:39 -04:00] "GET /backup/NavLive-Daily-Full.bak HTTP/1.1" 500 1148 "-" "-"`

The file I am trying to serve is huge +10GB. Based on the stacktrace this issue may be related to the open PowerShell issue

https://github.com/PowerShell/Microsoft.PowerShell.Archive/issues/19

I've tried disabling compression but that does not seem to help. Any suggestions on what to try next?

server.ps1 ` $HostIPAddress = (Get-NetIPAddress -InterfaceIndex 10).IPAddress Write-Host $HostIPAddress

$Port = 9898 Write-Host $Port

Start-PodeServer {

Add-PodeEndpoint -Address $HostIPAddress -Port $Port -Protocol Http

Add-PodeRoute -Method Get -Path '/' -ScriptBlock {
    Write-PodeJsonResponse -Value @{ 'value' = 'Hello, world!' }
}

Add-PodeStaticRoute -Path '/backup' -Source './backup' -DownloadOnly -Verbose

Use-PodeLogging -Path './log'

New-PodeLoggingMethod -Terminal | Enable-PodeRequestLogging

New-PodeLoggingMethod -Terminal | Enable-PodeErrorLogging

} `

server.psd1 @{ Server = @{ Request = @{ Timeout = 86400 } } Web = @{ Compression = @{ Enable = $false } } }

[Badgerati]

Hi @fazleskhan,

The issue comes down to MemoryStream using a Integer, which limits the max size to ~2GB. I believe it was at the beginning of the year, I actually fixed the opposite issue, whereby file uploads to the server were capped at 2GB. I might be able to use similar logic that fixed that issue here 🤔

One other workaround for now could be to split the file up into multiple parts, and then reform the file on the client side?

[fazleskhan]

Thank you for letting me know. I've been pulled into another project for the next couple of weeks. When I get a chance I'll give the splitting up the file try.