torchrec icon indicating copy to clipboard operation
torchrec copied to clipboard

gpu version `int_nbit_split_embedding_codegen_lookup_function` not support `indices` and `offsets` with uint64 dtype

Open tiankongdeguiji opened this issue 1 year ago • 3 comments

This PR removes uint32 indices protection in quant/embedding_modules.py. Nevertheless, when indices and offsets inputs of uint64 dtype are provided, int_nbit_split_embedding_codegen_lookup_function triggers a RuntimeError: expected scalar type Int but found Long on the GPU. Would it be possible to add support for the uint64 dtype on GPU devices?

tiankongdeguiji avatar Mar 27 '24 11:03 tiankongdeguiji

It what cases would it be necessary to use uint64?

PaulZhang12 avatar Apr 16 '24 15:04 PaulZhang12

It what cases would it be necessary to use uint64?

In certain scenarios, the uint32 data type is insufficient to contain the indices for user_id and several combo feature.

tiankongdeguiji avatar Apr 18 '24 11:04 tiankongdeguiji

@tiankongdeguiji Hi, let me bring this to the FBGEMM team's attention and get back to you.

spcyppt avatar May 13 '24 22:05 spcyppt