Justin Waugh

Hey @marella, any update on this? I like the idea of having a transformers compatible model and tokenizer object for using ctransformers. I'd love to try it out, if you...

Hmm... is "asynchronous" flag on the filesystem meant to mean 1:1 with streaming? That feels like an over-specification / a mismatch from what I would expect when trying to use...

here's a verison where I don't set the `asynchronous=True` flag, but still get behavior that is unexpected. ``` import fsspec import asyncio import random async def read_bytes(url, start, end): my_id...

Based on your comment: > The sync file-like object does, of course, call down into the async code, so it is possible to get a true async and random-access file,...

I did, haha. The command above includes `--kt-threadpool-count 1`. My issue wasn't the NUMA, it was that the output tokens were incoherent. When you got it running witht he single...

Coming back to this a few days later: My guess is that using an AMD processor is the main issue, as the AMXINT4 is an Intel specific instruction (?). That...

I have run into this in Jupyter in vscode -- took me so long to pin down to specifically the 7.4.9 -> 8.0.0 jump as the source of the issue....

I found a solution to launch claude via a small wrapper that removes the env vars it uses to detect VScode. Adding the following alias/command works for me as a...

I don't know if this is real, but it seems like in the last ~48 hours, this has gotten a lot worse for me. I now have to start every...