[QST] what the purpose to reimplement torch.unique as UniqueOp

[westfly]

What is your question? what the purpose to reimplement torch.unique as UniqueOp ? I noted that the functional of UniqueOp is same as torch.unique in testsuite corelib/dynamicemb/test/test_unique_op.py

---

By submitting this issue, you agree to follow our code of conduct and our contributing guidelines.

[jiashuy]

Hi @westfly ,thanks for you question.

• UniqueOp is hash-based which performance is better.
• And for multi-table case, table's bias is needed to generate inverse index, we can use UniqueOp to fuse bias-adding. Therefore, for performance and functional requirement, we think UniqueOp is better.

[westfly]

Hi @westfly ,thanks for you question.

• UniqueOp is hash-based which performance is better.
• And for multi-table case, table's bias is needed to generate inverse index, we can use UniqueOp to fuse bias-adding. Therefore, for performance and functional requirement, we think UniqueOp is better.

Is there any code related to benchmarking? It seems that the open-source part only includes regression tests for correctness.

[jiashuy]

Hi @westfly , we still not have benchmark code about UniqueOp in this repo. Hash-based UniqueOp's performance is verified in other framework. We will keep this issue opened and update the benchmark code once it's ready, thanks!