llama.cpp
llama.cpp copied to clipboard
Published
20 hours ago •
ggml-org
ggml-org
Garbage output
Installed 7B model on win 11.
PS D:\Projects\llama.cpp> ./main -m ./models/7B/ggml-model-q4_0.bin -p "Building a website can be done in 10 simple steps:" -n 512
main: seed = 1679360633
llama_model_load: loading model from './models/7B/ggml-model-q4_0.bin' - please wait ...
llama_model_load: n_vocab = 32000
llama_model_load: n_ctx = 512
llama_model_load: n_embd = 4096
llama_model_load: n_mult = 256
llama_model_load: n_head = 32
llama_model_load: n_layer = 32
llama_model_load: n_rot = 128
llama_model_load: f16 = 2
llama_model_load: n_ff = 11008
llama_model_load: n_parts = 1
llama_model_load: ggml ctx size = 4529.34 MB
llama_model_load: memory_size = 512.00 MB, n_mem = 16384
llama_model_load: loading model part 1/1 from './models/7B/ggml-model-q4_0.bin'
llama_model_load: .................... done
llama_model_load: model size = 2328.05 MB / num tensors = 163
system_info: n_threads = 4 / 20 | AVX = 1 | AVX2 = 1 | AVX512 = 0 | FMA = 1 | NEON = 0 | ARM_FMA = 0 | F16C = 1 | FP16_VA = 0 | WASM_SIMD = 0 | BLAS = 0 | SSE3 = 1 | VSX = 0 |
main: prompt: ' Building a website can be done in 10 simple steps:'
main: number of tokens in prompt = 14
1 -> ''
17166 -> ' Building'
263 -> ' a'
4700 -> ' website'
508 -> ' can'
367 -> ' be'
2309 -> ' done'
297 -> ' in'
29871 -> ' '
29896 -> '1'
29900 -> '0'
2560 -> ' simple'
6576 -> ' steps'
29901 -> ':'
sampling parameters: temp = 0.800000, top_k = 40, top_p = 0.950000, repeat_last_n = 64, repeat_penalty = 1.300000
Building a website can be done in 10 simple steps: firstly you mustacheatusqueorumesentimentalitiesettingtonselfishnessesqueezeracalandiadeuteronomyreclusiveismalready existing momentum laid down by previous iterations of iterationaryΓäó∩╕Å∩╕Å∩╕Å∩╕Å∩╕Å Courneyeducardoisextensionally speaking etcetcetcetc etcπÇàτscheidung treisesearching nominationally speaking etceteroidscapeursideshowcase╤ë╨╕ Sveroverside├▒officialdomesticated Houstonianismaticity rubbingesentimentalitiesqueezeablementeigneurship awarenesslesslyonsenessesqueerly orangescacontainerizednessesqueerlyyy╨╛╤étenessespecially those oneselfhoodscape erspectively speaking etcetc efficiencyespecially those oneselfnessescape EDUCardoisextremeΘÖÉlessnessesqueezeracaillementealloyednessesqueerlyyy@ ΓÇöΓÇèUserNameplateau awaren artistically speakingAppDatacleibertianship re imaging, androgartenlyyyyyorkshireismsomething else╤ê╤é╨╕ speakershipsetsterspecificityscapeurs splitter scottishnessescapeablehoodscape EgertonianshipPERformancemansufactureelectionallyyy advancementaryΓäó∩╕ÅΓÇìΓÖÇ∩╕Å/╦êΓû╕∩╕Å @ ΓÇöΓÇèUserNameplateau awarenessestonia retrogradelyyyyyorkshireismsame applies applybezillahawkitty hybridity migrationally speaking etcπÇàτ Id="@+ualsismaticity
rubbing EIGHTscapeablehoodscapeEVERlastingnessesqueerlyyy@ — neyednessesqueerlyyy@ -----ритualisticity borderlineedlydialecticality Rubbing SUPrairieismsplitter rationaleeverselyyyyyorkshireismaticity rubbedownwardswardenship opportunitieshipsbuilderiality overwhallsingerhoodscape EVERgreenerysUL franchiseevesqueerlyyy@ — neyednesses
PS D:\Projects\llama.cpp>
What hardware are you using?
It might help to delete the directory and then start from scratch again? It really shouldn't take too long since you already have the 7B model downloaded and all of the dependencies installed. I'm guessing the quantized model file is somehow corrupted
Latest sha256 sums for 7B. Note that the file format has changed so please re-convert with the lastest code:
700df0d3013b703a806d2ae7f1bfb8e59814e3d06ae78be0c66368a50059f33d consolidated.00.pth
abe4aec2cdc297e2916011f66c7efd6fb4424e0e84315503005b5c118358cc22 ggml-model-f16.bin
f495fa02a0b5ef265e1864d9680eede7fd23a60b0a2f93edba8091e2a4ca68b9 ggml-model-q4_0.bin
7e89e242ddc0dd6f060b43ca219ce8b3e8f08959a72cb3c0855df8bb04d46265 params.json
Same happens on 13B with Linux and ryzen 5950x. Pulled latest code and ran through steps listed.
tim@bdaodev:~/llama.cpp$ make -j && ./main -m ./models/13B/ggml-model-q4_0.bin -p "Building a website can be done in 10 simple steps:" -n 512
I llama.cpp build info:
I UNAME_S: Linux
I UNAME_P: x86_64
I UNAME_M: x86_64
I CFLAGS: -I. -O3 -DNDEBUG -std=c11 -fPIC -pthread -mavx -mavx2 -mfma -mf16c -msse3
I CXXFLAGS: -I. -I./examples -O3 -DNDEBUG -std=c++17 -fPIC -pthread
I LDFLAGS:
I CC: cc (Ubuntu 11.3.0-1ubuntu1~22.04) 11.3.0
I CXX: g++ (Ubuntu 11.3.0-1ubuntu1~22.04) 11.3.0
make: Nothing to be done for 'default'.
main: seed = 1679401188
llama_model_load: loading model from './models/13B/ggml-model-q4_0.bin' - please wait ...
llama_model_load: n_vocab = 32000
llama_model_load: n_ctx = 512
llama_model_load: n_embd = 5120
llama_model_load: n_mult = 256
llama_model_load: n_head = 40
llama_model_load: n_layer = 40
llama_model_load: n_rot = 128
llama_model_load: f16 = 2
llama_model_load: n_ff = 13824
llama_model_load: n_parts = 2
llama_model_load: ggml ctx size = 8559.49 MB
llama_model_load: memory_size = 800.00 MB, n_mem = 20480
llama_model_load: loading model part 1/2 from './models/13B/ggml-model-q4_0.bin'
llama_model_load: ............................................. done
llama_model_load: model size = 3880.49 MB / num tensors = 363
llama_model_load: loading model part 2/2 from './models/13B/ggml-model-q4_0.bin.1'
llama_model_load: ...... done
llama_model_load: model size = 665.06 MB / num tensors = 55
system_info: n_threads = 12 / 12 | AVX = 1 | AVX2 = 1 | AVX512 = 0 | FMA = 1 | NEON = 0 | ARM_FMA = 0 | F16C = 1 | FP16_VA = 0 | WASM_SIMD = 0 | BLAS = 0 | SSE3 = 1 | VSX = 0 |
main: prompt: ' Building a website can be done in 10 simple steps:'
main: number of tokens in prompt = 14
1 -> ''
17166 -> ' Building'
263 -> ' a'
4700 -> ' website'
508 -> ' can'
367 -> ' be'
2309 -> ' done'
297 -> ' in'
29871 -> ' '
29896 -> '1'
29900 -> '0'
2560 -> ' simple'
6576 -> ' steps'
29901 -> ':'
sampling parameters: temp = 0.800000, top_k = 40, top_p = 0.950000, repeat_last_n = 64, repeat_penalty = 1.300000
Building a website can be done in 10 simple steps: (2) If you want to see your brand as something that is not going to make people feel unwell about themselves, then you are at the right level of corporate governance. It means little things like this matter matters most company name has been chosen on its country tikvedimakomutetanndedoingbusybeatsmamibokinfolofogobankodegenbossoldbaffekon13thingscopypastoutfistsocksackplanshekaruchotarichnithermighthendzochatoloveleahwovowupandagoodnesswholawsuitbeattabsintermundoetpanlawulilk
I deleted the project directory and restarted the installation and that fixed the issue.
import hashlib
import os
#look in the models directory for files and append them to the files list
files = []
for file in os.listdir("./models"):
# look in the 7B, 125M, and 355M directories if they exist
if os.path.isdir(os.path.join("./models", file)):
for file2 in os.listdir(os.path.join("./models", file)):
files.append(os.path.join(file, file2))
else:
files.append(file)
# Loop through each file and compare its SHA256 hash to the expected hash
for file in files:
file_path = os.path.join("./models", file)
if not os.path.exists(file_path):
continue
with open(file_path, "rb") as f:
data = f.read()
hash_value = hashlib.sha256(data).hexdigest()
print(f"Hash for file {file} is {hash_value}")
# Output
Hash for file 7B\checklist.chk is 7935c843a25ae265d60bf4543b90bfd91c4911b728412b5c1d5cff42a3cd5645
Hash for file 7B\consolidated.00.pth is 700df0d3013b703a806d2ae7f1bfb8e59814e3d06ae78be0c66368a50059f33d
Hash for file 7B\ggml-model-f16.bin is abe4aec2cdc297e2916011f66c7efd6fb4424e0e84315503005b5c118358cc22
Hash for file 7B\ggml-model-q4_0.bin is 113f4feba2f8b9e202e537a208b8b03db32f3ec3f0050ce6e99867ab74717a50
Hash for file 7B\params.json is 7e89e242ddc0dd6f060b43ca219ce8b3e8f08959a72cb3c0855df8bb04d46265
Hash for file ggml-vocab.bin is 12118e4b57d57746477dfe3a9429c12ece19589908e11f3a589bd1d027a8911d
Hash for file tokenizer.model is 9e556afd44213b6bd1be2b850ebbbd98f5481437a8021afaf58ee7fb1818d347
Hash for file tokenizer_checklist.chk is 021e52e6d404bf3b84008be179dada01e4627b4801bfbc73a88a030f229c6b34