Wrong and irrelevant answers with very Basic Usage

[zvigrinberg]

I followed Basic Usage section in documentation, and it appears that when sending very basic prompts for completion, the Model returns irrelevant completions/answers in the good scenario, and in worst scenario it returns delusional completions that looks like they were randomly picked without any connection, or related matching pattern or context. Within the response itself, there is endless repetition of sentences, that limited only by the requested number of maximum tokens to predict ( -n flag)

Basic prompt for completion ( deterministic):

python run_inference.py -m models/Llama3-8B-1.58-100B-tokens/ggml-model-tl2.gguf -p "internet is" -temp 0

Response from the Model:

internet is a first step in the development of a new generation of robots that can be used in a wide variety of applications, medical, military, and industrial. The robot is designed to be able to perform a wide variety of tasks, including the ability to perform tasks that are not currently possible for humans. The robot is designed to be able to perform tasks that are not currently possible for humans. The robot is designed to be able to perform tasks that are not currently possible for humans. The robot is designed to be able to perform tasks that are not currently possible for humans. The robot is designed to be able to perform tasks that are not currently possible

Basic prompt for completion 2 ( with temperature =1 , to increase creativity):

python run_inference.py -m models/Llama3-8B-1.58-100B-tokens/ggml-model-tl2.gguf -p "internet is" -temp 1

Response:

internet is a second of the five senses in the body and is closely linked to the other four. The other four senses include sight, smell, taste and touch. The human eye has the ability to see a wide variety of colors, from red to green to blue to purple. In the same way that humans can see a variety of colors, the human eye can also see a variety of shapes and sizes, such as large, small, square or triangular. It is a fact that the human eye is very useful and can see a lot. The human eye is able to see objects that are far away and objects that are close up to the eye