Ryan Spring comments

Results 5 comments of


                                            Ryan Spring

Python Implementations?

Not yet. This repo reproduces the LSH implementation in the paper. We're working on GPU version with support for python.

Batched Computation

In a batched setting, I would combine the weight vectors for all of the examples. Say you have a very large hidden layer with 10K neurons and a batch size...

Batched Computation

SimHash or Signed Random Projection (SRP) uses a standard matrix-multiplication. You will need a GPU kernel to turn the signed bits into integers. The tricky part depends on the size...

Transpose scheduler plan

Here is Ivan's PR to enable transpose with primTorch: https://github.com/pytorch/pytorch/pull/84629

Fix `_broadcast_in_dim_prim_grad` when `reduce_dims` is empty

I discovered it when updating https://github.com/Lightning-AI/lightning-thunder/pull/260. ```python input shape = (5,) target shape = () weight = True reduction = mean label_smoothing = 0.5 ``` In this ^^^ single example...