DeepSpeed-MII
DeepSpeed-MII copied to clipboard
Configure server log level
Please add one or more params to control logs from RESTful API server - namely in mii.serve() function.
You can see as reference -log- config params in vLLM: https://docs.vllm.ai/en/latest/serving/openai_compatible_server.html#command-line-arguments-for-the-server
It is especially important at startup, when there is a huge amount of logs (from several processes).
Hi @sedletsky-f5 - we will try to prioritize this, in the meantime, if you are able or willing, would you consider making a PR and we can review and get it merged?
thanks @loadams , I will wait for you implementation (I didn't get into the code to make this change)