llamafile
llamafile copied to clipboard
API endpoint to return llamafile version + other metadata
It would be useful to have an API endpoint like /info
that would return a json dict containing 1) the version of llamafile and 2) model metadata/configuration.
Llamafile version allows me to adapt client code to work with each release. E.g. I think the output of the /embedding
endpoint changed between release v0.6.2 and what's currently on master. This will be a breaking change for my LlamaIndex integration -- v0.6.2 llamafiles will work but v0.6.3 llamafiles will not (in certain cases).
It would also be useful to have a way to get model metadata via an endpoint. E.g. user systems need to know the model's max sequence length in order to truncate/batch prompts appropriately. Currently there's no way to get this info.