Paolo Rechia

First, I'd like to thank you for the hard work on this repository, amazing job here! I wanted to share that I've successfully put a fork of this repository to...

Feedback: some missing features / bugs?

3

Hi, everyone. First, thanks for this project, it looks very promising. Unfortunately, I still found that some features missing (and maybe bugs?). I'm just wondering if these things have been...

Supports multi line input by adding a new tool

2

Hi, I've recently opened an issue (#3287) and added a potential solution, a new tool for multi line human input. The code is very simple. I tested locally with my...

Hi, I'm trying to add a code snippet in the Human input, but I can't seem to paste it correctly (it seems to breaks on newline). Is there a way...

Add integration to guidance library

19

Thanks a lot for implementing this ToT library, nice work! I think it could benefit a lot from using Microsoft Guidance (https://github.com/microsoft/guidance), specially with smaller local models that have a...

Adds example with text generation web ui for local models

4

Tested with WizardLM 7B, did not work super well, but could be iterated on. I think combining this with the guidance library could be really nice BTW Not sure if...

Integrating with Guidance: adding a positive bias to certain tokens

5

Hi, thanks a lot for this project, really interesting! I'm interested in trying to hook it up with the guidance library (https://github.com/microsoft/guidance). Before I attempt any coding though, would be...

Adds the possibility to influence prediction with bias

3

Related issue (created by me): https://github.com/turboderp/exllama/issues/103

Setup package

1

Supports installing `exllama` as a package. Example usage: ``` pip install 'exllama_lib @ git+https://github.com/paolorechia/exllama@setup-package' ``` EDIT: Worth explaining how to use the installed package. Since the installation setup creates a...