Fred Bliss

I think it's still just going to optimize for completion of the full text field and doesn't differentiate between the input/output? At least based on the lora code in mlx-llm....

Looking at the mlx-llm code, I think we need to adjust the Dataset class to account for various dataset types (instruct, chat, etc). Which starts to turn into an axolotl...

I believe this fork handles it correctly: https://github.com/chimezie/mlx-tuning-fork/blob/main/src/mlx_tuning_fork/training.py (https://github.com/ml-explore/mlx-examples/pull/235) edit: saw this PR (https://github.com/ml-explore/mlx-examples/pull/213), looks like the goal is to keep the lora purely as an example, but i do...

Same issue here on Mac (arc, chrome, firefox). But seems to be for remote only (running on a different machine on the local network). No issue with Safari for some...

I think GPTQ would be where lora support gets added, no? Given this looks like the key addition from the alpaca lora code - model = LLaMAForCausalLM.from_pretrained( "decapoda-research/llama-7b-hf", load_in_8bit=True, device_map="auto",...

I think this is helpful because it indeed would cover the issue I raised that you referenced, but also allows for extensibility by other tooling without reinventing the wheel every...

> @fblissjr I'm not sure if you've seen [Chat Templates](https://huggingface.co/blog/chat-templates), but if you haven't, I strongly suggest you take a look because I think it can answer your question (whether...

One thing I keep thinking back to - as soon as it can run code or a repl arbitrarily, locally, there's potential for problems down the line. Anyone found any...

> I mean, we could make it run things in a docker container for some kind of sandboxing to be safe. Almost sure that for now no LLM would break...

@Wingie didn't know about baby booga - will test it out after work. :)