Mimocro

@TingTingin, yep, have some of it. But even with much more black magic, tiny colab RAM is still a main issue. Other issues are colab specific and i think they...

When i say load model from VRAM to RAM i mean model loads from cuda to cpu, It can be checked if crash model by setting precision to bf16 (google's...

@1blackbar, i suppose, the limit RAM still the main problem. If colab pro with P100 can give more than 12.6 GB RAM, it is possible with replacing the git clone...

@robbsaber, maybe, paths to something are incorrect?

> @1blackbar, i suppose, the limit RAM still the main problem. If colab pro with P100 can give more than 12.6 GB RAM, it is possible with replacing the git...

For me, changing the lines from `from modules.grammar.grammar_utils import initialize_grammar from modules.grammar.logits_process import GrammarConstrainedLogitsProcessor` to `from modules.grammar_utils import initialize_grammar from modules.logits_process import GrammarConstrainedLogitsProcessor` in "modules/text_generation.py" after copying both files from...

> > I'm trying to use my local model but i can't save without providing a VLM API Key > >  > > Which llm client are you using?...

Still missing every single click for me. Win11, LM studio, ui-tars-desktop 0.1.2, cant even click at search box. In settings, i have set the VLM provider to "Hugging Face for...