Directory structure and files to avoid model downloads.

[mattopeerenboom]

To avoid large model downloads if I might already have the required files, what directory structure and files should be in place?

[VTSTech]

I notice the md5 for the largest file is present in checklist.chk

I may put in a pull request to md5 existing files and skip if they are already valid.

This is where I find the files for one of the models

C:\Users\%USERNAME%\dalai\llama\models\7B

[mattopeerenboom]

Thanks! Can you also say what the file names are? The available models have a lot of different formats.

[VTSTech]

C:\Users\VTSTech\dalai\llama\models>dir /s
 Volume in drive C is HITACHI-1TB
 Volume Serial Number is 62C9-A8A2

 Directory of C:\Users\VTSTech\dalai\llama\models

2023-03-19  02:13 PM    <DIR>          .
2023-03-19  02:13 PM    <DIR>          ..
2023-03-19  02:22 PM    <DIR>          7B
2023-03-19  02:13 PM           499,723 tokenizer.model
2023-03-19  02:13 PM                50 tokenizer_checklist.chk
               2 File(s)        499,773 bytes

 Directory of C:\Users\VTSTech\dalai\llama\models\7B

2023-03-19  02:22 PM    <DIR>          .
2023-03-19  02:22 PM    <DIR>          ..
2023-03-19  02:04 PM               100 checklist.chk
2023-03-18  04:37 PM    13,476,939,516 consolidated.00.pth
2023-03-19  12:36 PM    13,477,682,409 ggml-model-f16.bin
2023-03-19  02:07 PM     4,212,727,017 ggml-model-q4_0.bin
2023-03-19  02:04 PM               101 params.json
               5 File(s) 31,167,349,143 bytes

     Total Files Listed:
               7 File(s) 31,167,848,916 bytes
               5 Dir(s)  306,201,251,840 bytes free