torchrec

torchrec copied to clipboard

register a custom op to keep the PEA module unflattened when torch.export

6

Summary: reference: * D54009459 Differential Revision: D56282744

TroyGarden

embedding vector init norm by embedding dim instead of num_embeddings

3

di-wnd

Remove import HeteroPlanner to fix torch package issues

1

Differential Revision: D56358725

IvanKobzarev

Adjust default options for HeteroEmbeddingShardingPlanner

3

Summary: (1) Let HeteroEmbeddingShardingPlanner use MemoryBalancedPartitioner instead of GreedyPerfPartitioner. Using MemoryBalancedPartitioner makes more sense due to the nature of diffrerent DDR/HBM sizes (2) Let HeteroEmbeddingShardingPlanner use EmbeddingEnumerator with exact emumerate...

gnahzg

gpu version `int_nbit_split_embedding_codegen_lookup_function` not support `indices` and `offsets` with uint64 dtype

3

This [PR](https://github.com/pytorch/torchrec/pull/1487) removes uint32 indices protection in quant/embedding_modules.py. Nevertheless, when `indices` and `offsets` inputs of uint64 dtype are provided, int_nbit_split_embedding_codegen_lookup_function triggers a `RuntimeError: expected scalar type Int but found Long`...

tiankongdeguiji

quantize_embeddings + KeyedJaggedTensor+ vbe cannot work

3

``` import torch from torchrec import KeyedJaggedTensor from torchrec import EmbeddingBagConfig,EmbeddingConfig from torchrec import EmbeddingBagCollection,EmbeddingCollection kt2 = KeyedJaggedTensor( keys=['user_id', 'item_id', 'id_3', 'id_4', 'id_5', 'raw_1', 'raw_4', 'combo_1', 'lookup_2', 'lookup_3', 'lookup_4', 'match_2',...

yjjinjie

EmbeddingCollection+KeyedJaggedTensor+vbe the inverse_indices don't work

2

``` import torch from torchrec import KeyedJaggedTensor from torchrec import EmbeddingBagConfig,EmbeddingConfig from torchrec import EmbeddingBagCollection,EmbeddingCollection kt = KeyedJaggedTensor( keys=['t1', 't2'], values=torch.tensor([0,0,0,0,2]), lengths=torch.tensor([1,1,1,1,0,1], dtype=torch.int64), ) kt2 = KeyedJaggedTensor( keys=['t1', 't2'], values=torch.tensor([0,0,2]),...

yjjinjie

`AttributeError: 'NoneType' object has no attribute '_dynamo_weak_dynamic_indices'` when using row-wise sharding

3

# Description I’m using torch.compile with DistributedModelParallel. Running below code result in AttributeError: 'NoneType' object has no attribute '_dynamo_weak_dynamic_indices'. Note that this seems to only happen when using row-wise sharding....

jiannanWang

`ValueError: Tensors must be contiguous` when running a specific model in DistributedModelParallel with world size equaling 2

1

# Description I’m using torch.compile with DistributedModelParallel. Running the below code results in a ValueError: Tensors must be contiguous. This error seems to be specific to the model and the...

jiannanWang

Long running time when using torch.compile with DistributedModelParallel and dynamo errors

1

# Description I’m using torch.compile with DistributedModelParallel. Given torch.compile is able to speed up pytorch distributed models, I would expect to see faster inference time. However, it takes 50 seconds...

jiannanWang