[CODE IMPROVEMENT] Redirect stdout logging from huggingface download to the logging module

[pascal-pfeiffer]

trafficstars

🔧 Proposed code refactoring

Redirect stdout logging from huggingface download to the logging module

Something on these line could solve it (untested):

import logging
from transformers import logging as transformers_logging

logger = logging.getLogger(__name__)
transformers_logging.set_verbosity_info()
transformers_logging.enable_default_handler()

# Redirect the download progress bar to the logger
transformers_logging.install_verbosity_handler(logger=logger, log_level=logging.INFO)

# Now, any download done by Hugging Face will be logged to the logger

Motivation

Improve logs that are shown in the web app

[maxjeblick]

Like the idea! Tricky part may be progress bars (model downloads etc.) that could cause logging a new line at each update.

[aress31]

I am in the same situation than @pascal-pfeiffer, I need to redirect the tf_logging more specifically the progressbar for downloading models to other streams, e.g., the logging module. Is there a workaround?