MLTQ

Same here- doing a lambda function on a large dataframe hangs at seemingly random points, some threads run to completion and others fail. try-catching in the lambda function doesn't seem...

Hi, I wanted to do this too and figured out a way- pretty rudimentary but it works! Add this function to cli.py then run it and it will pop up...

M1 Pro 32GB, 30B model: main: mem per token = 43387780 bytes main: load time = 10701.85 ms main: sample time = 279.92 ms main: predict time = 37065.80 ms...

Sounds like the ideal structure of this would be to load the model into memory in interactive mode, listen for input on some port, then wait for initial prompt &...

Thanks @nschulzke, I have been doing this, but I would ideally want to do something like: `./main` and it look for a config file, and if it finds it, use...

You can run the model in chat mode and suppress diagnostic output, seems like it wouldn't be too difficult to integrate.

Adding on here, I need some sort of GUI to connect with the IBKR gateway which we want to run on AL2023. MATE, TIGERVNC, doesn't matter, just need something for...

This shouldn't contain a hardcoded model, but should reference something in `utils.py`, and `max_token` counts should be similarly parameterizable.