glowwormX
glowwormX
sdk最好支持此种写法，未来有计划支持么
@samadejacobs I'm glad to see this pr will be merged soon. When are you going to support sdpa in the future? It's useful for me.
> @samadejacobs I'm glad to see this pr will be merged soon. When are you going to support sdpa in the future? It's useful for me. Yes, but I want...
@zeyugao I carefully read your pull requests for transformers and accelerate, and pulled your code to try training. Now I have encountered a problem: when entering DistributedAttention, the q, k,...
@zeyugao My God, I missed it, I thought there was this code in pr. Thank you for replying.
@zeyugao Have you compared the loss of sequence parallel? After a fixed seed is added to DistributedSampler, the training data is the same. Modify the trainer.py: ``` if is_accelerate_available() and...
> Hi everyone, I’m encountering the following error when trying to run the image vllm/vllm-openai:v0.6.6.post1 on a node equipped with 8x H100 SMX GPUs: > > ``` > ValueError: Error...
@samadejacobs Hello, I'd like to ask why z3 is not supported currently. I modified certain code and successfully executed z3. The difference between loss and loss is almost 4%. The...
@samadejacobs Thank you for replying. If I want to support z3 on the HF client, where do I need to start? Where can I learn from the code of the...
Same issue