Liger-Kernel

Liger-Kernel copied to clipboard

ValueError: Pointer argument (at 0) cannot be accessed from Triton (cpu tensor?)

1

### 🐛 Describe the bug Getting `ValueError: Pointer argument (at 0) cannot be accessed from Triton (cpu tensor?)` when doing inference using HF `from_pretrained()` with `device_map="auto"`. ### Error ``` File...

shivam15s

cross entropy kernel: inplace modification of input

2

Hello, Regarding inplace modification of PyTorch tensors, there are already multiple (#254, #262, #272) issues. I would also like to point out that according to PyTorch docs for [mark_dirty()](https://pytorch.org/docs/stable/generated/torch.autograd.function.FunctionCtx.mark_dirty.html#torch-autograd-function-functionctx-mark-dirty): ```...

mgrabban

### 🚀 The feature, motivation and pitch Hi, great job on the FusedLinearCrossEntropy kernel! I’ve found it very efficient for model training. However, it seems to lack support for custom...

ChenlongDeng

mllama patch modifies nn.LayerNorm globally

3

### 🐛 Describe the bug Instead of only patching the transformers mllama module (`transformers.models.mllama.modeling_mllama`), `apply_liger_kernel_to_mllama` modifies `torch.nn.LayerNorm` globally. The issue is [here](https://github.com/linkedin/Liger-Kernel/commit/6ab3b9febc29f5045e6d2e27ba6bacaa4f041d91#diff-376c16dd1328612cf488158c6ce9805b044773659c8459cdcb2c6ec35dac346bR163). The fix would be to: (1) Not patch...

tyler-romero