Chime Ogbuji

Extension of lora.py for supervised ML of configurable dataset formats with YAML-based configuration of parameters

1

Based on lora.py and meant to perform supervised instruction fine-tuning for the various supported models, separating the model-specific logic from the common training and supporting logic (and management of training...

Need more documentation on using a custom dataset for fine-tuning with LoRA

20

The [Custom Data](https://github.com/ml-explore/mlx-examples/tree/main/lora#Custom-Data) section of the LoRa readme has helpful information about how to specify a subdirectory where train.jsonl, valid.jsonl, and test.json files are expected and the dataset in data/...

LoRA on all linear transformer block layers

4

In general, it would be useful for the LoRa tuner and linear_to_lora_layers in mlx_lm.tuner.utils in particular to allow more control over how LoRa is applied (the ability to specify the...

Configurable LR schedulers

1

Updates the mlx_lm LoRA module's YAML configuration to include sections like the following: ```yaml schedule: join: boundaries: [101] schedules: - { name: linear_schedule, arguments: [0.0, 1e-5, 100] } - {...

_pythonToRdf is manhandling rdflib.Identifier instances

2

_pythonToRdf is assuming the object is a python data representation when it is not. The tuple of terms yielded from rdflib.Graph.Graph.triple, for instance is a tuple of rdflib.Identifier instances. So...

Schema introspection (_isUniqueObject) should be lazy or flagged

1

Currently, any attribute access will triger a call to schema introspection to determine the cardinality of the set returned. This is not efficient for large schema graphs. Ideally, either schema...

Configuration-based use of HF hub-hosted datasets for training

1

Per the title, allow a structured `hf_dataset` YAML configuration parameter for specifying an HF hub-hosted dataset (via `name`) to use with training and the ability to use [datasets](https://huggingface.co/docs/datasets/en/index)' local file...

When doing low-level finetuning (without the aid of HF's [SFTtrainer](https://huggingface.co/docs/trl/sft_trainer) library, for example), you may need to be able to tokenize a string with the model's prompting format but without...

Some systems (such as [mlx](https://github.com/ml-explore/mlx)) don't yet work with HF's safetensors ( see [LLama example doesn't work with HF format models? #65](https://github.com/ml-explore/mlx-examples/issues/65) ) and require access to the PyTorch files...

Iteration over results from oapi.wrap_for_multiproc raising KeyError: '__getstate__'

1

I originally got this error in my code while iterating over the results gathered asynchronously from a call to oapi.wrap_for_multiproc(prompt_to_chat('.. prompt ..'), **model_params). I was able to successfully reproduce it...