aici
aici copied to clipboard
microsoft
AICI: Prompts as (Wasm) Programs
logits tensor is float16, we use -100 to ban a token. Temperature setting below around `0.0003` causes overflow and the following crash: ``` File "/workspaces/aici/vllm/vllm/model_executor/layers/sampler.py", line 409, in _sample parent_seq_ids,...
both in ModuleRegistry and Stepper - if unused for too long, just delete them
turn LLM output to string, encode and compare token ids
We currently fail here, for simple rx like '\d*' ```rust impl FunctionalRecognizer for RecRx { fn initial(&self) -> RecRxState { self.dfa .universal_start_state(regex_automata::Anchored::Yes) .expect("dfa has no universal start state; make sure...
Can't build it with the command: ../aici.sh build . --tag pyctrl-latest Error below: abridged version OpenSSL include directory does not exist: /usr/include/x86_64-linux-gnu/include I have tried changing OPENSSL_DIR to many things,...
For example, the llama tokenizer has "" as 35 and "▁" (space) as 29871, as well as "" as 36 and "!" as 29991, etc. We need to: * pick...
Hello 👋, When I try to generate JSON using the example in the `basics_tutorial` notebook with the `phi2` model, the JSON generated doesn't conform to the structure specified: ``` Please...
This promptlib program when run with `phi2` seems to consistently crash the server with the following stacktrace. ```python program = pl.PromptProgram(aici) # load a grammar file c_grammar = open("../../../controllers/aici_abi/grammars/c.y", "r").read()...
pyctrl has `log_level`. in js we also need the setter (as you cannot set variables in other modules from the outside)
I already downloaded phi3 instruct gguf from: https://huggingface.co/microsoft/Phi-3-mini-4k-instruct-gguf/resolve/main/Phi-3-mini-4k-instruct-q4.gguf and placed it at: ~/models/ jawad@desktoper:~/models$ ls Phi-3-mini-4k-instruct-q4.gguf Yet, when I give command to choose model with -m or --model I get...