Chengyuan Li

Questions about fields identifiers

5

I have 3 questions about DCGM, I noticed that there are field identifiers like memory utilization and gpu utilization. 1. How these methods are calculated? 2. What if I want...

As mentioned [here](https://github.com/SafeAILab/EAGLE/issues/31#issuecomment-1927696940), any updates? Btw, the [checkpoint](https://huggingface.co/yuhuili/EAGLE-mixtral-instruct-8x7B) you released on huggingface is also just one transformer layer, right?

Question on GQA implementation

1

In GQA, only one copy of kv cache will be saved for each group, but snapKV saves kv cache with `num_key_value_heads * num_key_value_groups` heads. Indeed in kv cache eviction, the...