Jinn

cc @zhyncs . Sorry to bother, but could u have a check on this EAGLE issue?

You’ll need to modify the forward function so that—as it generates each token—it saves it (for example, by appending it to a list) and then returns that list. In many...

According to the log, it appears that the process termination routine is happening over and over. More specifically, the `sigquit_handler` (in engine.py at line 333) is repeatedly calling `kill_process_tree`(`os.getpid()`), and...

So what's your current issue? Do you evaluate it on any benchmark since you mentioned the performance? You are welcome to report any accuracy issue and we will take a...

Sure! Thank you for your help~ cc @Fridge003

This error stems from a bug in our custom Triton kernel (in the decode_attention module) used for data-parallel attention. With the configuration `–tp 2 –dp 2 –enable-dp-attention`, the kernel ends...

cc @zhaochenyang20 @SunMarc I tried to rerun the testing script and facing with this issue: ``` Writing report to /tmp/mmlu_meta-llama_Llama-3.2-1B-Instruct.html {'other': 0.1875, 'other:std': 0.3903123748998999, 'score:std': 0.40232478717449166, 'stem': 0.2727272727272727, 'stem:std': 0.4453617714151233,...

I also tried to create a new testing script to especially test opt model: [](https://gist.github.com/jhinpan/7fbd8fecceb475ad96c9b4ad5a8ba2ef) Only to face with the issues below: ``` [2025-05-01 21:52:18] Received sigquit from a child...

Sure, I was testing on a H100. I will check the rouge-l score and let chenyang know with running CI tests as well. cc @zhaochenyang20

cc @XuehaiPan @zhyncs. Can u guys take some final look and check whether it can be merged? LGTM rn.