Lev Kurilenko

Hi @satpalsr, Apologies for the delayed response. I think we'd still like to merge this change, but it would be nice to merge the latest master and rerun tests again....

> We tried to use this last night and the universal checkpoint conversion script failed because our DS checkpoint was missing universal _checkpoint_info. We commented out all references to that...

Hi @molly-smith, Can you please try the setup instruction in the automatic-speech-recogintion [README](https://github.com/microsoft/DeepSpeedExamples/tree/master/inference/huggingface/automatic-speech-recognition#setup) and see if that fixes it? I was able to get it to run on my end,...

Tested again w/ latest transformers version and this example is working okay now. Closing the issue.

Hi @Chevolier, DeepSpeed has monitoring functionality built in and the monitor can be selected by specifying the corresponding configuration (TensorBoard, WandB, csv). The documentation can be found here: https://www.deepspeed.ai/docs/config-json/#monitoring-module-tensorboard-wandb-csv For...

Hi @Chevolier, Just wanted to update you that we have a PR to add various instrumentation across all the DS Chat steps, including tensorboard logging (GH-624) Feel free to give...

> @loadams @lekurile it is fixed now, can you please trigger the unit tests? Hi @oelayan7, kicked off the unit tests.

Hello @delock, Thank you for raising this issue. I ran a local vllm benchmark with the `microsoft/Phi-3-mini-4k-instruct` model using the following code: ```python # Run benchmark python ./run_benchmark.py \ --model...

Hi @delock, I'm using `transformers==4.40.1` as well. After https://github.com/microsoft/DeepSpeedExamples/pull/895 was committed to the repo, I'm seeing the same error on my end as well. ```bash File "/lib/python3.8/site-packages/transformers/tokenization_utils_fast.py", line 504, in...

@delock, here's the PR fixing the `tokens_per_sec` metric to work for both the streaming and non-streaming cases: https://github.com/microsoft/DeepSpeedExamples/pull/897 You should be able to get past your error above with this...