Load FastLanguageModel from specific checkpoint

[Ben-Pfirsich]

I would like to fine-tune a local model checkpoint with the DPO Trainer from TRL. The folder structure of the model directory looks as follows:

drwxr-xr-x. 1        458                       3. Apr 21:17 checkpoint-1291
drwxr-xr-x. 1         458                      3. Apr 21:45 checkpoint-2583
drwxr-xr-x. 1      458                         3. Apr 22:14 checkpoint-3873
-rw-r--r--. 1         909                       3. Apr 22:14 config.json
-rw-r--r--. 1       183                         3. Apr 22:14 generation_config.json
-rw-r--r--. 1  4939050888                3. Apr 22:14 model-00001-of-00003.safetensors
-rw-r--r--. 1  4947390880                3. Apr 22:14 model-00002-of-00003.safetensors
-rw-r--r--. 1  3590554352                3. Apr 22:14 model-00003-of-00003.safetensors
-rw-r--r--. 1            23950                3. Apr 22:14 model.safetensors.index.json
-rw-r--r--. 1                694                3. Apr 22:14 special_tokens_map.json
-rw-r--r--. 1              1336                3. Apr 22:14 tokenizer_config.json
-rw-r--r--. 1        1843133                3. Apr 22:14 tokenizer.json

The following code does work. But i can not tell which checkpoint is loaded. How can i find this out?

  MODEL_PATH = 'GAMMA_1/leo-hessianai-7b_true_parallel/checkpoint-3873' // This is not used
  TOKENIZER_PATH = 'GAMMA_1/leo-hessianai-7b_true_parallel'

  model, tokenizer = FastLanguageModel.from_pretrained(
      model_name=TOKENIZER_PATH,
      max_seq_length=max_seq_length,
      device_map='auto',
      dtype=None, 
      load_in_4bit=False, 
  )

[danielhanchen]

@Ben-Pfirsich See https://github.com/unslothai/unsloth/wiki#finetuning-from-your-last-checkpoint

[Ben-Pfirsich]

Thank you.