tc-wolf

Feature request: on-the-fly masking for pretraining (MLM + RTD)

3

**Is your feature request related to a problem? Please describe.** In the [BERT data creation script](https://github.com/keras-team/keras-nlp/blob/master/examples/bert_pretraining/bert_create_pretraining_data.py) the masking is all done statically (before training, then write out to a TF...

Bug: LLAMAFILE = 0 in `llama_print_system_info` even though compiled with `-DGGML_LLAMAFILE=ON` and can see set during compilation

1

### What happened? In `llama_print_system_info` in llama.cpp, the output is `LLAMAFILE = 0`, even though can see that it's enabled (from the output `Using llamafile` when building). ``` $ cmake...

**Is your feature request related to a problem? Please describe.** I have a dataset that I'm using for RAG. The user's question determines lookup for the top N most relevant...

Would like to train with https://huggingface.co/docs/peft/main/en/package_reference/shira as the peft method. See also: - https://github.com/Qualcomm-AI-research/SHiRA - https://github.com/huggingface/peft/tree/main/examples/shira_finetuning