Ebubekir

Can EAGLE-3 be used across different machines?

6

Can EAGLE-3 be used across multiple GPUs on different machines/nodes, or between a GPU on one machine, and a CPU on another?

Bug Fix For "matplotlib eventplot not shows all the binary data for big number of events" - Draft PR

10

** DRAFT PULL REQUEST ** closes #20243 This PR is a fix to the bug from Issue 20243. The bug concerns events being missing in graphs outputted by eventplot, when...

Can you summarise the purpose of these functions in eagenerate: - initialize_tree() - tree_decoding() - evaluate_posterior() - update_inference_inputs() - Also, model.ea_layer.topK_genrate()

EAGLE-3 Training and Test Data

21

In the newly added code for EAGLE-3 training, there is references to the training data of "/home/lyh/code/nlp/developing/vllmbase/vllm/gedata/l318b.jsonl" and the testing data of "/home/lyh/code/nlp/developing/vllmbase/vllm/gedata/0318.json"? Do paths refer to the training and...

Calculating Average Acceptance Length on EAGLE-3

2

Is this correct to calculate average acceptance length and the number of new tokens generated? output_ids = model.eagenerate(input_ids, temperature=0.0, max_new_tokens=256, log=True) new_tokens = int(output_ids[1]) steps = int(output_ids[2]) avg_accept_length = new_tokens...