Joshua Rosenkranz

Python lambda much slower than java class implementing same interface when calling from Java

2

I am trying to create a python class that implements a Java Interface using py4j with a callback server. I have been successfully able to do that, but I am...

Since switching to version `0.10.9`, while using the multiprocessing module, I have seen programs hang, and no value be returned. For example, the following code works in `0.10.8.1`, but hangs...

[WIP] MLPSpeculator speculative decoding support

1

# What does this PR do? **_1. What is the motivation behind this feature? Is it related to a problem or frustration with the library? Is it a feature related...

# What does this PR do? Currently MLPSpeculator does not support `tie_weights`. Many newer trained speculators with MLPSpeculator architecture are using this feature, which makes the speculator much smaller. -...

Currently, we require an fms variant of a model, this should make it simpler to load models that have variants that fms does not support natively

Currently, you would need to add chat templates manually, we should have a way to add them into the training in a more streamlined fashion.