transformers-bloom-inference icon indicating copy to clipboard operation
transformers-bloom-inference copied to clipboard

Published 20 hours ago • verified publisher icon huggingface

Incorrectly benchmarking

Open JoeyTPChou opened this issue 2 years ago • 0 comments

All 3 scripts under bloom-inference-scripts incorrectly benchmark the t_generate_span time. The t_generate_span is got from the first generate() call at here https://github.com/huggingface/transformers-bloom-inference/blob/main/bloom-inference-scripts/bloom-ds-inference.py#L257 instead of in the benchmark cycle.

JoeyTPChou avatar Apr 01 '23 00:04 JoeyTPChou