LocalAI
LocalAI copied to clipboard
feat: Centralized Request Processing middleware
This PR reworks how our endpoints parse and process user input, in order to centralize and commonize that code as much as possible. This will drastically cut down the endpoint-specific code to a minimum.
Additionally, the VAD subsystem was untested - making it hard to validate if this changeset was safe or not. Therefore, this PR also adds both manual testing tools (via bruno) and automated testing in the AIO image.
Note to @mudler - this PR looks worse than it is - here's some reviewing notes to help
-
core/http/middleware/request.go
should be where you start - its the heart of the change and replacescore/http/ctx/fiber.go
- configuration of defaults is now a responsibility of the route function - seems a bit weird at first, but allows endpoints to be easily reused with different defaults on different paths.
- while all the endpoint files were updated, the changes all follow the same pattern:
- A new standard header to the function was added which pulls the relevant input and config from the earlier middleware stage
- A bunch of code is deleted as its now performed earlier by those middlewares
- The functional code specific to the endpoint is unchanged except for variable references
- Specific to the AIO CI test: I was seeing very unpredictable behavior on CI versus locally reproducing this test. It turns out that setting
LOCALAI_SINGLE_ACTIVE_BACKEND=true
helps with a memory usage issue!