candle
candle copied to clipboard
huggingface
Qwen 3-1.7b looks like something is wrong and doesn't stop properly.
Candle version: main Platform: Mac Studio Max M1 Mode: Qwen 3-1.7b, (download by huggingface-cli) Execute cmd:
git clone https://github.com/huggingface/candle.git
cd candle-examples
cargo run --release --example qwen --
--prompt "What is the speed of light?"
--model 3-1.7b
--tokenizer-file ../../models/qwen3-1.7b/tokenizer.json
--weight-files "../../models/qwen3-1.7b/model-00001-of-00002.safetensors,../../models/qwen3-1.7b/model-00002-of-00002.safetensors"
--temperature 0.3
--top-p 0.5
--repeat-penalty 1.5
--repeat-last-n 16
Got:
Qwen 3-1.7B
Running `target/release/examples/qwen --prompt 'What is the speed of light?' --model 3-1.7b --tokenizer-file ../../models/qwen3-1.7b/tokenizer.json --weight-files ../../models/qwen3-1.7b/model-00001-of-00002.safetensors,../../models/qwen3-1.7b/model-00002-of-00002.safetensors --temperature 0.3 --top-p 0.5 --repeat-penalty 1.5 --repeat-last-n 16`
avx: false, neon: true, simd128: false, f16c: false
temp: 0.30 repeat-penalty: 1.50 repeat-last-n: 16
retrieved the files in 300.917µs
Running on CPU, to run on GPU(metal), build this example with `--features metal`
loaded the model in 7.719477208s
What is the speed of light? What are its properties?
The Speed Of Light
What is the speed of light? What are its properties?
The Speed Of Light
What is the speed of light? What are its properties?
The Speed Of Light
What is the speed of light? What are its properties?
The Speed Of Light
What is the speed of light? What are its properties?
The Speed Of Light
What is the speed of light? What are its properties?
The Speed...
^C
Indeed. Here's what I get when I add --features=metal. It also starts repeating eventually but overall much better.
What is the speed of light? What are its properties?
The Speed Of Light
Light is one form of electromagnetic radiation, and it travels at a speed that's the fastest known in our universe. The exact value of this constant is 29,978 kilometers per second (or about a million miles) in vacuum. This is the speed of light, and it's often denoted by c.
The Speed Of Light Is The Fastest Known
Light is the fastest known form of electromagnetic radiation. It travels at a speed that's approximately 30 kilometers per second in vacuum. This is the same as a million miles, or about 186 thousand mi per second. The speed of light is the same in a vacuum, but it changes when passing through different media like water or glass.
The Speed Of Light Is Constant
In a vacuum, the speed of light is constant. It's not affected by any other factors like distance, time or mass of the object. This is a fundamental property that's been confirmed by many experiments and observations.
The Speed Of Light Is The Same In All Reference Frames
This is one of the key principles in Einstein's theory of relativity. According to this principle, no matter how fast you're moving relative to a light source or observer (as long as your speed is less than the velocity of light), you will always measure a constant value for c. This is the principle of relativity.
The Speed Of Light Is The Limit For All Massless Particles
In the theory of relativity, mass is related to energy by E = mc². This means that the more massive an object is, or if it's moving at a high speed (close to the velocity of light), then its energy becomes larger. However, for massless particles like photons (which are the carriers of light), their energy is given by E = pc, where p represents the momentum. This means that even though photons have no rest mass, they still carry energy and momentum. The speed of light is the limit for all massless particles, meaning that they can't travel faster than c.
Using --features=accelerate does not affect the output, so that eliminates (at least) matmul as the problem.
Give it a go after you change --repeat-last-n to 17. For me it prevents the repetition.
Could it be that it's simply the settings used that is the issue?
cargo run --release --example qwen --
--prompt "What is the speed of light?"
--model 3-1.7b
--tokenizer-file ../../../models/qwen3-1.7b/tokenizer.json
--weight-files "../../../models/qwen3-1.7b/model-00001-of-00002.safetensors,../../../models/qwen3-1.7b/model-00002-of-00002.safetensors"
--temperature 0.3
--top-p 0.5
--repeat-penalty 1.5
--repeat-last-n 17
Finished release profile [optimized] target(s) in 0.21s
Running candle-main/target/release/examples/qwen --prompt 'What is the speed of light?' --model 3-1.7b --tokenizer-file ../../../models/qwen3-1.7b/tokenizer.json --weight-files ../../../models/qwen3-1.7b/model-00001-of-00002.safetensors,../../../models/qwen3-1.7b/model-00002-of-00002.safetensors --temperature 0.3 --top-p 0.5 --repeat-penalty 1.5 --repeat-last-n 17
avx: false, neon: true, simd128: false, f16c: false
temp: 0.30 repeat-penalty: 1.50 repeat-last-n: 17
retrieved the files in 312.583µs
Running on CPU, to run on GPU(metal), build this example with --features metal
loaded the model in 11.540451208s
What is the speed of light? What are its properties?
The Speed Of Light
Light, as we know it from everyday experience and science textbooks alike is a form of electromagnetic radiation. It travels at an incredibly high speed, and in the vacuum it is approximately 301 million meters per second. This speed, which we will call c for now is the maximum possible velocity in our universe. It's also a constant, which means that it has the same value in all reference frames of observers. This is one reason why Einstein's theory of relativity has such a profound effect on our understanding and perception.
The speed is so fast that it's hard to imagine. For example, if you were in a spaceship moving at the speed of light and then turned on your headlights, you would see a flash in front. But if the spaceship was moving at 90% of light speed, then you would see a flash in front and behind. But if the spaceship was moving at 95% of light speed, then you would see a flash in front and behind. But if the spaceship was moving at 95% of light speed, then you would see a flash in front and behind. But if the spaceship was moving at 95% of light speed, then you would see a flash in front and behind. But if the spaceship was moving at 95% of light speed, then you would see a flash in front and behind. But if the spaceship was moving at 95% of light speed, then you would see a flash in front and behind. But if the spaceship was moving at 95% of light speed, then you would see a flash in front and behind. But if the spaceship was moving at 95% of light speed, then you would see a flash in front and behind. But if the spaceship was moving at 95% of light speed, then you would see a flash in front and behind. But if the spaceship was moving at 95% of light speed, then you would see a flash in front and behind. But if the spaceship was moving at 95% of light speed, then you would see a flash in front and behind. But if the spaceship was moving at 95% of light speed, then you would see a flash in front and behind. But if the spaceship was moving at 95% of light speed, then you would see a flash in front and behind. But if the spaceship was moving at 95% of light speed, then you would see a flash in front and behind. But if the spaceship was moving at 95% of light speed, then you would see a flash in front and behind. But if the spaceship was moving at 95% of light speed, then you would see a flash in front and behind. But if the spaceship was moving at 95% of light speed, then you would see a flash in front and behind. But if the spaceship was moving at...
With argument --features metal:
cargo run --release --features metal --example qwen --
--prompt "What is the speed of light?"
--model 3-1.7b
--tokenizer-file ../../../models/qwen3-1.7b/tokenizer.json
--weight-files "../../../models/qwen3-1.7b/model-00001-of-00002.safetensors,../../../models/qwen3-1.7b/model-00002-of-00002.safetensors"
--temperature 0.3
--top-p 0.5
--repeat-penalty 1.5
--repeat-last-n 17
Finished release profile [optimized] target(s) in 0.23s
Running candle-main/target/release/examples/qwen --prompt 'What is the speed of light?' --model 3-1.7b --tokenizer-file ../../../models/qwen3-1.7b/tokenizer.json --weight-files ../../../models/qwen3-1.7b/model-00001-of-00002.safetensors,../../../models/qwen3-1.7b/model-00002-of-00002.safetensors --temperature 0.3 --top-p 0.5 --repeat-penalty 1.5 --repeat-last-n 17
avx: false, neon: true, simd128: false, f16c: false
temp: 0.30 repeat-penalty: 1.50 repeat-last-n: 17
retrieved the files in 723.458µs
loaded the model in 656.242959ms
What is the speed of light? What are its properties?
The Speed Of Light
Light, as we know it from our everyday experience and the scientific understanding of physics is a form electromagnetic radiation. It travels at an extremely high speed in vacuum, and it is the fastest known form of matter. The exact value for this speed in a vacuum, is approximately 298 million meters per second (m/s), or about $3 \times 10^8$ m/sec. This is also known as the speed of light in vacuum, and it's denoted by c.
The exact value for the speed of light in a vacuum is defined as exactly 299,792,458 meters per second. This is a defined value in the International System of Units (SI), and it's used as an exact constant. This value is also known in the SI unit of meters per second, and it's used as a standard for measuring other quantities.
The speed of light is also the maximum possible velocity in a vacuum, and it's one of those fundamental constants that is used to define the SI unit for distance, which was originally defined as one meter. The speed of light is also used in the definition and measurement for other physical quantities, such as time.
The speed of light is a fundamental constant in physics and plays an important role throughout the field. It's used to describe how electromagnetic waves propagate, and it is also a key factor in the theory of relativity. The speed light can be measured using various methods, such as the use of interferometry or by measuring how long it takes for light to travel a certain distance.
The speed of the electromagnetic wave is equal in vacuum, and it's also a key factor for understanding how light interacts with matter. The speed of the wave is also affected by other factors, such as medium through which it travels. For example in water or glass the speed of light is slower than that measured vacuum.
The exact value for c in a真空 (vacuum) is defined as exactly 299,792,458 meters per second. This is a defined value in the International System of Units (SI), and it's used as an exact constant. This value is also known in the SI unit of meters per second, and it's used as a standard for measuring other quantities.
The speed of light is also the maximum possible velocity in a vacuum, and it's one of those fundamental constants that is used to define the SI unit for distance, which was originally defined as one meter. The speed of light is also used in the definition and measurement for other physical quantities, such as time.
The speed of light is a fundamental constant in physics and plays an important role throughout the field. It's used to describe how electromagnetic waves propagate, and it is also a key factor in the theory of relativity. The speed light can be measured using various methods, such as the use of interferometry or by measuring how long it takes for light to travel a certain distance.
The speed of the electromagnetic wave is equal in vacuum, and it's also a key factor for understanding how light interacts with matter. The speed of the wave is also affected by other factors, such as medium through which it travels. For example in water or glass the speed of light is slower than that measured vacuum.
The exact value for c in a真空 (vacuum) is defined as exactly 299,792,458 meters per second. This is a defined value in the International System of Units (SI), and it's used as an exact constant. This value is also known in the SI unit of meters per second, and it's used as a standard for measuring other quantities.
The speed of light is also the maximum possible velocity in a vacuum, and it's one of those fundamental constants that is used to define the SI unit for distance, which was originally defined as one meter. The speed of light is also used in the definition and measurement for other physical quantities, such as time.
The speed of light is a fundamental constant in physics and plays an important role throughout the field. It's used to describe how electromagnetic waves propagate, and it is also a key factor in the theory of relativity. The speed light can be measured using various methods, such as the use of interferometry or by measuring how long it takes for light to travel a certain distance.
The speed of the electromagnetic wave is equal in vacuum, and it's also a key factor for understanding how light interacts with matter. The speed of the wave is also affected by other factors, such as medium through which it travels. For example in water or glass the speed of light is slower than that measured vacuum.
The exact value for c in a真空 (vacuum) is defined as exactly 299,792,458 meters per second. This is a defined value in the International System of Units (SI), and it's used as an exact constant. This value is also known in the SI unit of meters per second, and it's used as a standard for measuring other quantities.
...