jan
jan copied to clipboard
feat: Decouple `model.json` from model binaries
Problem
- Currently, we force users to create a model folder
- We force users to create a
model.json
- In order to "import and use" any model binary
- This is cumbersome & breaks all the time.
Big Picture My opinion is that we should build around the worst case (i.e. user drags model into /models) Building for worst-case allows us to handle any type of scenario, e.g. the "Add Model Folder" that users have been requesting We should see our model.json as a UX improvement, vs. a necessity.
- Jan should be able to detect any .gguf or .bin file in /models
- Jan should fail gracefully if model is malformed, half-downloaded, or wrong format
- Jan should display model in somewhat acceptable format (i.e. filename) in UX
- Decouple
model.json
from Model, i.e. the model config file is not a requirement; We should instead see /modelname/model.json as a UX improvement - Jan should allow the user to see the ugly imported model in Hub, and add structure + metadata
- e.g. Renaming model = creates folder, moves Model into folder, creates model.json
- Alternatively, "orphaned" model is moved automatically into /modelname folder?
Open questions
- Support symlink?
- How to handle multi-model without model.json? (if we support it?)
- How to handle model deletions of binaries but not model.jsons?
- How to handle complete model deletions?
Specs
- Jan runs model binaries in
/models
- Jan doesn't require
model.json
files for models to run -
model.json
assumes the existence of model binary in/models
Specs TBA
Additional context Add any other context or screenshots about the feature request here.