distilabel
distilabel copied to clipboard
[FEATURE] Review `setup_logging` and add control flags
Description
Currently the logging
handler created for the LLMs is named as distilabel.llm.{llm.model_name}
, but the llm.model_name
property shouldn't be used since it can be confusing or even expose a private URL when we're using either Inference Endpoints or an LLM subclass without a name other than the URL for example. So on, the logging
handler for the LLMs
should be improved.
Additionally, it would be nice to add a control flag named verbose
for the steps that we want to silence or make less verbose as e.g. the data loading step which will work in most of the cases and adding logs per batch whenever there are multiple batches can overload the logs with no relevant information.
Other than that, an in detail review of the current logging
approach could be done too.