ByteMLPerf

ByteMLPerf copied to clipboard

【ILUVATAR】add general model infer case.

1

Add bert、albert、roberta、deberta、videobert、swin-transformer、widedeep、resnet50、yolov5、conformer.

yudefu

【Iluvatar】add iluvatar operator && llm && bert/resnet50/widedeep/yolov5 ixrt infer backend.

1

1. add iluvatar operator backend. 2. add iluvatar llm infer backend. 3. add bert/resnet50/widedeep/yolov5 ixrt infer backend.

forestlee95

Modify torchrun parameter

1

Modify the torchrun parameter _--nproc_per_node_ (Number of workers per node) to correct format. Thanks~

davidshtian

运行gpt2-torch-fp32 task 遇到的2个问题

1

问题1: general_perf/prepare_model_and_dataset.sh 脚本无法下载到正确的位置： ``` wget -O general_perf/download/traced_gpt2.tar https://lf-bytemlperf.17mh.cn/obj/bytemlperf-zoo/traced_gpt2.tar tar xf general_perf/download/gpt2.tar -C general_perf/model_zoo/sota/ ``` 下载的tar 包和解压的tar 包名称不一样，需要修改为： ``` wget -O general_perf/download/traced_gpt2.tar -c https://lf-bytemlperf.17mh.cn/obj/bytemlperf-zoo/traced_gpt2.tar mkdir general_perf/model_zoo/sota/traced_gpt2 tar xf general_perf/download/traced_gpt2.tar -C general_perf/model_zoo/sota/traced_gpt2/...

longerzone

请教下目前 model_zoo 中提供的 fp32 的模型，有提供工具或者方法快速转换为 fp

2

eric-yq

请问llm perf有没有可能支持llam3测试， 8B 版本。这个版本现在用的人比较多。

incomingflyingbrick

add batch_gemm, group_gemm; add int8 dtype to gemm ops; fix situation that world_size exceeds available devices.

suisiyuan

[llm_perf] fix wrong return value type of CoreInferencer.Task.get_result

1

update `async def get_result(self) -> GenerateResult` to `async def get_result(self) -> GenerateResult`

qinhj

【Issue Help】 chatglm2-6b has some cases dismatch with golden

2

https://github.com/bytedance/ByteMLPerf/blob/main/byte_infer_perf/llm_perf/workloads/chatglm2-torch-fp16-6b.json We run on A100-40G to get output logits with the below configuration： ```json { "model": "chatglm2-torch-fp16-6b", "test_accuracy": true, "test_perf": true, "min_new_tokens": 128, "max_new_tokens": 256, "tp_sizes": [1, 2], "batch_sizes":[1, 2,...

DeepTecher

add MTGPU backends for operators

1

add MTGPU backends for operators

qiyulei-mt