zhp icon indicating copy to clipboard operation
zhp copied to clipboard

Support for Handling Larger Files for Forms submission

Open wesleyParriott opened this issue 2 years ago • 3 comments

when trying to submit a file larger than 50MB I get the following stack trace

warning: Write to temp file
Segmentation fault at address 0x8
/home/vvesley/workspace/zig/build/lib/zig/std/crypto/tlcsprng.zig:62:1: 0x22ad04 in std.crypto.tlcsprng.tlsCsprngFill (zhttpd)
fn tlsCsprngFill(_: *anyopaque, buffer: []u8) void {
^
/home/vvesley/workspace/zig/build/lib/zig/std/rand.zig:60:17: 0x2c536c in std.rand.Random.bytes (zhttpd)
        r.fillFn(r.ptr, buf);
                ^
/home/vvesley/workspace/zig/build/lib/zig/std/fs.zig:156:32: 0x2c45c6 in std.fs.AtomicFile.init (zhttpd)
            crypto.random.bytes(rand_buf[0..]);
                               ^
/home/vvesley/workspace/zig/build/lib/zig/std/event/loop.zig:1420:25: 0x2965fd in std.event.loop.Loop.workerRun (zhttpd)
                        resume handle;
                        ^
/home/vvesley/workspace/zig/build/lib/zig/std/Thread.zig:395:13: 0x2db23f in std.Thread.callFn (zhttpd)
            @call(.{}, f, args);
            ^
/home/vvesley/workspace/zig/build/lib/zig/std/Thread.zig:911:30: 0x2db1c6 in std.Thread.Instance.entryFn (zhttpd)
                return callFn(f, self.fn_args);
                             ^
???:?:?: 0x3dd86e in ??? (???)
Aborted (core dumped)

wesleyParriott avatar May 24 '22 23:05 wesleyParriott

I notice that forms.parse will return an error.NotImplemented when the content.type (which it gets from requests) is of type TempfFile.

I would like to help implement handling large files because in a current project I just upped the buffersize to be like 500 mbs (which will have problems later in the project). So, what all needs to be done to get this implemented and how can I help?

wesleyParriott avatar May 24 '22 23:05 wesleyParriott

I'm not sure why it's not in the trace but the error is coming from https://github.com/frmdstryr/zhp/blob/master/src/request.zig#L481.

The form parser needs refactored, I just don't have the time/need at the moment. Ideally it should require the user to pass in a comptime known struct to hold everything expected in the form and for large files write them to their destination location that way it can handle large streams.

You can always write your own code handle large file uploads if needed by just reading from the iostream (eg https://github.com/frmdstryr/zhp/blob/master/src/request.zig#L492-L501).

frmdstryr avatar May 25 '22 13:05 frmdstryr

When writing large files to their destination would that destination be in that comptime struct that the user would then setup?

wesleyParriott avatar Jun 22 '22 18:06 wesleyParriott