ZhiYuanZeng

能否像shadowsocks那样自定义要过滤的域名

could you please add the hyper-parameters in the readme, e.g. bias. it will make it easier for others to reimplement the result in the paper without looking for these hyper-parameters...

Why the rope params are ignored while converting hf checkpoint to composer checkpoint?

3

I found that the rope params are ignored in composer_to_hf.py and that the base of rope in composer_llama.py is set to be 10000 constantly. However, it is normal to tune...

In `tokenize_single_file.py` (line 61), the dtype of data saved in .npy file is set to be uint16. However it is not correct for the case where vocabulary size is large...

[BUG] Unreasonable memory comsumption

1

### 🐛 Describe the bug Creating an TransformerEncoder causes memory overflow, but the same config works with the huggingface `transformers` module. ```python # config.py from colossalai.amp import AMP_TYPE fp16=dict( mode=AMP_TYPE.TORCH...