Manil Shrestha

icon indicating an email [email protected]

icon company Drexel University icon location Philadelphia icon location Grad Student

Results 2 issues of Manil Shrestha

RuntimeError with Tensors on Different Devices When Using Outlines

1 comment

### Describe the issue as clearly as possible: I've encountered an issue when attempting to use the outlines library with a model downloaded from the Hugging Face Hub (turboderp/Llama-3-8B-Instruct-exl2) and...

bug

Make sure that logits are always on the device where the model weights are

1 comment

The issue was due to the logits always being in device=0, I made it such that the logits generated by the model should always be in the model's device.