ShijieZZZZ

To address this issue: [[BUG] DeepSpeed zero_to_fp32.py script ignores some layers while creating FP32 checkpoints from DS ZeRO checkpoints. ](https://github.com/microsoft/DeepSpeed/issues/1896) Shared parameters that hold reference are missing when extracts fp32...

Fix redundant shared_params in zero_to_fp32.py

2

state_dict["module"] has redundant params that were mistakenly recorded in shared_params Related: (1) https://github.com/microsoft/DeepSpeed/issues/3291 (2) https://github.com/microsoft/DeepSpeed/pull/3295

Request to add 'sparse' in BaseEmbeddingConfig

2

Trying to make [nn.EmbeddingBag/nn.Embedding](https://github.com/pytorch/torchrec/blob/2bc324cd56849046ff042e4ad40ca540e349b282/torchrec/modules/embedding_modules.py#L150) in EmbeddingBagCollection `sparse=True` through `EmbeddingBagConfig/EmbeddingConfig`. However, these config classes do not have `sparse ` attribute now. self.embedding_bags[embedding_config.name] = nn.EmbeddingBag( num_embeddings=embedding_config.num_embeddings, embedding_dim=embedding_config.embedding_dim, mode=pooling_type_to_str(embedding_config.pooling), device=device, include_last_offset=True, dtype=dtype, sparse=embedding_config.sparse,...