amulil

[WIP][Feature] DPO

10

@pppppM 佬，按你说的，初步想法是在 dataset 目录下实现 `DPODataset`，在 model 目录下实现 `DPO`，其他 hook 暂时和 sft 一致的，不用修改，但是有一个疑问，DPO 里有 model 和 ref_model 两个 model，deepspeed 相关的部分用修改嘛？

[BUG] Can't `xtuner check-custom-dataset` use?

1

I use the command `xtuner check-custom-dataset $CONFIG`, and have a error. ```python Traceback (most recent call last): File "/xtuner/xtuner/tools/check_custom_dataset.py", line 157, in main() File "/xtuner/xtuner/tools/check_custom_dataset.py", line 51, in main dataset...

[bug] The datasets can't load successfully when using two nodes in slurm.

2

```python # reproduce srun -p debug --job-name=xtuner --nodes=2 --gres=gpu:8 --ntasks-per-node=8 --kill-on-bad-exit=1 xtuner train yi_34b_qlora_oasst1_e3_gpu16 --deepspeed deepspeed_zero2 --launcher slurm ``` ```python # loginfo File "/data/miniconda3/lib/python3.10/site-packages/torch/distributed/distributed_c10d.py", line 2261, in broadcast_object_list object_tensor =...

[Feature] how to open window attention in qwen-14B?

4

### Motivation I know I can change `/path/to/turbomind-style/triton_models/weights/config.ini` to open NTK-aware interpolation and LogN attention scaling.But where I can open window attention? If I use NTK-aware interpolation and LogN attention...

The sequence parallel is open when I don't use it.

4

## version `05/09 21:16:21 - mmengine - INFO - 0.1.18` ## how to reproduce `CUDA_VISIBLE_DEVICES=4,5,6,7 NPROC_PER_NODE=4 xtuner train qwen1_5_0_5b_chat_qlora_alpaca_e3` ## log I only change the batch_size to 4 in config...

Does it support Yi-6B/34B-chat?

1

I try to use it like llama2, but it failed.