Chi Zhang issues

Results 17 issues of


                                            Chi Zhang

Is there anyway to export the grid structure as a graph (e.g. networkX)?

### Versions - [ ] Python version: - [ ] Python architecture: - [ ] Operating system and version: - [ ] OpenDSSDirect.py version number: ### Feature Request ### Bug...

Question about Pipeline parallelism

Just wonder does the current PipelineStage API supports variable length input shapes like in Megatron? https://github.com/NVIDIA/Megatron-LM/blob/e33c8f78a35765d5aa37475a144da60e8a2349d1/megatron/core/model_parallel_config.py#L212 This is particular useful for packed inputs where all the paddings are removed.

question

Question about custom cuda operators for tensor parallelism

We are currently trying to apply torchtitan to MoE models. MoE models require using grouped_gemm https://github.com/fanshiqing/grouped_gemm. GroupedGemm ops basically follow the same rule as in ColumnLinear and RowLinear. Is there...

question

kl is applied twice if kl_loss_coef >0 and kl_coef > 0 for GRPO

We should make them mutually exclusive by using assertion in config

Chi Zhang

Is there anyway to export the grid structure as a graph (e.g. networkX)?

Question about Pipeline parallelism

Question about custom cuda operators for tensor parallelism

kl is applied twice if kl_loss_coef >0 and kl_coef > 0 for GRPO

Add a format issue template

Add nightly ci to ensure accuracy

Add assertion to ensure the reward in GRPO is generated by ORM