SovereignRemedy

> Anybody able to reproduce(重现) the deterministic results?有人能够重现确定性结果吗？ The same problem was encountered at least two weeks ago, but no logical explanation has been seen so far

Some problems that output few tokens may have a high probability of stable output，But the odds are less than 50%

> This PR adds a simple fastapi server to serve the llama model. > > Thank you for your time(时间) on reviewing this PR :) hi . Will the http...

[GCHeap.heap-3.20230131112906.390.log](https://github.com/mosn/holmes/files/10554811/GCHeap.heap-3.20230131112906.390.log) [mem.20230131112506.312.log](https://github.com/mosn/holmes/files/10554815/mem.20230131112506.312.log) 提供两个dumpdump出来的文件 如果需要别的可以再提供

> > [GCHeap.heap-3.20230131112906.390.log](https://github.com/mosn/holmes/files/10554811/GCHeap.heap-3.20230131112906.390.log) [mem.20230131112506.312.log](https://github.com/mosn/holmes/files/10554815/mem.20230131112506.312.log) 提供两个dumpdump出来的文件 如果需要别的可以再提供 > > 两个profile里的metric指向的函数都是，应该是add2line的时候找不到函数名和地址。能提供下demo吗 因为是公司的服务感觉不太好直接全贴出来..需要哪些关键的地方我可以节选出来。 ` func systemInit() { err := service.Init() if err != nil { glog.Fatalf("system fatal err:%s", err) } configs.InitDb() appstore.Init() appdev.Init()...

> > > > [GCHeap.heap-3.20230131112906.390.log](https://github.com/mosn/holmes/files/10554811/GCHeap.heap-3.20230131112906.390.log) [mem.20230131112506.312.log](https://github.com/mosn/holmes/files/10554815/mem.20230131112506.312.log) 提供两个dumpdump出来的文件 如果需要别的可以再提供 > > > > > > > > > 两个profile里的metric指向的函数都是，应该是add2line的时候找不到函数名和地址。能提供下demo吗 > > > > > > 因为是公司的服务感觉不太好直接全贴出来..需要哪些关键的地方我可以节选出来。 ` func systemInit() { err...

> I agree with your observation. > > Since prefill stage is compute bound, GPU is achieving the highest efficiency, so increasing the workload (batched tokens in your case) will...