jitsu icon indicating copy to clipboard operation
jitsu copied to clipboard

Improve HTTP event processing

Open jfk9w opened this issue 3 years ago • 0 comments

Problem

Events submitted via HTTP (/api/v1/s2s/event, for example) currently are buffered in memory (as in "the whole request body is read into a byte slice"). This can cause memory overflow.

Solution

Implement parsing the request body without buffering. It would also be nice to log the first 1000 symbols of the request body in case of errors – see server/middleware/logging.go:22 (some kind of io.Reader implementation is needed which would buffer the first 1000 symbols for logging and concatenate the buffer with the rest of the body when parsing the body). Maybe even write it to a separate log file.

jfk9w avatar Feb 17 '22 17:02 jfk9w