whcao

First of all, thank you for sharing your valuable code. As shown in fig2(c) of the paper, the softmax function is used to convert the matrix product to the dot-product...

# Usage ```diff custom_hooks = [ + dict(type=HFCheckpointHook), ] ``` The HF model will be saved at `work_dir/timestamp/hf_model` # Notes 1. As the priority of `HFCheckpointHook` is 95 which is...

[Feature] Add qwen1.5 110b cfgs

2


Visualize layer activations and weights to simplify the quantization process.

5

## Usage 1. Draw the first type of plot which shows the absmax/absmean/max/mean/min value of a linear layer at different layers. For instance, the code below presents the visualized results...