MrPanch

### Motivation Deepspeed provide out of box tensor parallelism. However, when I modify config, for example, internvl_chat/zero_stage3_config.json adding "model_parallelism" parameters to fine 26B model: "model_parallel": { "enabled": true, "dp_world_size": 6,...