jan icon indicating copy to clipboard operation
jan copied to clipboard

feat: Decouple `model.json` from model binaries

Open 0xSage opened this issue 6 months ago • 0 comments

Problem

  • Currently, we force users to create a model folder
  • We force users to create a model.json
  • In order to "import and use" any model binary
  • This is cumbersome & breaks all the time.

Big Picture My opinion is that we should build around the worst case (i.e. user drags model into /models) Building for worst-case allows us to handle any type of scenario, e.g. the "Add Model Folder" that users have been requesting We should see our model.json as a UX improvement, vs. a necessity.

  • Jan should be able to detect any .gguf or .bin file in /models
  • Jan should fail gracefully if model is malformed, half-downloaded, or wrong format
  • Jan should display model in somewhat acceptable format (i.e. filename) in UX
  • Decouple model.json from Model, i.e. the model config file is not a requirement; We should instead see /modelname/model.json as a UX improvement
  • Jan should allow the user to see the ugly imported model in Hub, and add structure + metadata
    • e.g. Renaming model = creates folder, moves Model into folder, creates model.json
    • Alternatively, "orphaned" model is moved automatically into /modelname folder?

Open questions

  • Support symlink?
  • How to handle multi-model without model.json? (if we support it?)
  • How to handle model deletions of binaries but not model.jsons?
  • How to handle complete model deletions?

Specs

  • Jan runs model binaries in /models
  • Jan doesn't require model.json files for models to run
  • model.json assumes the existence of model binary in /models

Specs TBA

Additional context Add any other context or screenshots about the feature request here.

0xSage avatar Dec 27 '23 05:12 0xSage