Mechanism for recording a different model ID from the one requested

[simonw]

Inspired by this comment in https://simonwillison.net/2024/Apr/22/llama-3/#local-llama-3-70b-instruct-with-llamafile

One warning about this approach: if you use LLM like this then every prompt you run through llamafile will be stored under the same model name in your SQLite logs, even if you try out different llamafile models at different times. You could work around this by registering them with different model_id values in the YAML file.

This made me think: there are other cases where the model that executed the prompt may not be the exact model you specified. Calling OpenAI and asking for gpt-4-turbo and getting a response run against the more recent dated model for example.

It would be useful to have a mechanism for recording the actual model used (as opposed to the model_id requested) in the SQLite logs.

[simonw]

Maybe this is a nullable column in the responses table which is only sometimes populated?