Oleg Klimov

I actually tried that! On my 4·2048·1536 float16 batch, Dense attention built on top of blocksparse matmul and softmax: 196.8ms + 264.1ms = 460.9ms (forw and back, no checkpointing, 12...

Yes, broken for me. I have MacBook Air M1, and LG UltraFine. Thank you for this great piece of software BTW.

Aha thank you, clear now. I think a fix for this is to mention "you need to rebuild with cuda support" on the front page.

I'm happy to report this bug disappeared, after updating to Ventura 13.2.1

live streaming -n 1000 is a good idea for plain file 50000 maybe is a good idea too

Thanks for reporting!

We have docs repository https://github.com/smallcloudai/web_docs_refact_ai

Interesting! But it works in nightly without any changes 🤔 Let's ask what @JegernOUTT and @mitya52 think.

@ChinnYu awesome that you are trying this! You might need a change in `refact-lsp`, just add the model there by analogy like the other models. There was this idea to...

Code completion doesn't work very well on apple silicon or CPUs. The context is just too big for completion to be fast enough. Chat without much context is of course...