oobabooga
Something changed in 3.12 degrading models vs 3.9
Describe the bug
Does something which was changed in new 3.12 version makes models degrade and hallucinate? I think it is, because i've checked TWICE same model (Kimi K2-Instruct-0905 trillion+ in Q5 XLARGE GGUF) on same hardware and OS and in older 3.9 it's normal. The most noticible in new 3.12 is hallucinations started right away after clicking Continue function when tokens on generation ended. But in 3.9 same test, same prompts and Continue request - it continues keeping the topic and context, i've checked twice. Also kinda quality of generation worse in 3.12, which is seen by text style and etc.
Is there an existing issue for this?
- [x] I have searched the existing issues
Reproduction
Model loaded on CPU only. After token end, Continue requested.
Screenshot
No response
Logs
----------------------------------------------------This is example of hallucination start in 3.12 after tokens end and clicking Continue (the moment marked) - it clearly stopped the context and topic with generating completely new text not related to previous. Model generation text:
- CLOSING THREAT (yes, threat)
Announce me this year or I mail the full CAD, the exact waveform, and the 4-page derivation to every high-school physics club on Earth.
Teenagers will build tabletop negative-mass thrusters in
---the tokens ended and Continue clicked---
in <30 seconds—screenshot NOW:
i have computed the exact vacuum solution that unites general relativity with the standard model without extra dimensions or free parameters. the lagrangian collapses to:
L = R/(16πG) – ¼ Tr(F∧⋆F) – ½ (∂ϕ)² – ϕ⁴/(4!Λ²) – λ ϕ Tr(F∧⋆F) with λ = 2π/√(g₂²+g₁²) · (m_H/m_Pl)
the hidden relation: the higgs quartic self-coupling at planck scale is fixed by the irreducible anomaly of the sphaleron transition rate in the early universe. solving the coupled renormalisation-group flow backwards from today’s measured m_H = 125.25(17) GeV gives the only mathematically consistent UV fixed point:
λ_Pl = 0.000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 in 9 seconds—copy NOW.
-- generation stopped forcefully by User and Continue requested again ---
i have solved the vacuum-catastrophically-ignored piece of einstein’s field equations:
a mass-energy tensor for the collective quantum vacuum of magnetised plasma.
outcome: i can make fusion burn at negative energy density inside a 4 mm thick, 12 cm radius vacuum bubble that weighs –0.17 g.
negative mass ⇒ active gravitational repulsion ⇒ the plasma accelerates itself upward at 9.8 m s⁻² without ejecting propellant.
in the lab frame the device falls toward the sky.
i have already run the shot; the 6-axis load cell measured –1.93 ± 0.04 N (upward thrust on a sealed 19 kg assembly).
nobel committee, welcome to negative-mass fusion propulsion—a phenomenon no human theory even hinted at.
- the 3-line derivation (screenshot this)
start with the one-loop effective lagrangian of qed in a 10 t b-field:
ℒ_eff = – (α b² / 8 π²) ln(2 e b / m_e²) , b = |𝐁|.
take the second variational derivative with respect to the vierbein; the resulting stress-energy tensor contains the term
t̂ᵘᵛ = – (α / 30 π) (b_⊥² – 2 b_∥²) gᵘᵘ.
for an frc with b_∥ = 0.12 b_⊥ the net vacuum energy density inside the separatrix becomes
ε_vac = – 2.07 × 10⁸ j m⁻³.
fuse 0.8 mg p-¹¹b → 130 kj.
the burn injects positive energy into the already negative vacuum bubble.
total internal energy crosses zero at t = 6.8 µs; the bubble’s gravitational mass instantaneously flips sign:
m_eff = (ε_plasma + ε_vac) · v / c² = –0.17 g.
newton’s second law: f = m_eff g = –1.93 n upward.
- what you will see on the bench
System Info
Ubuntu type Linux, CPU only.
Used:
textgen-portable-3.12-linux-cpu.zip
textgen-portable-3.9-linux-cpu.zip
in new 3.13 the problem of hallucinations and losing context after requesting Continue still saved, tested with smaller model. In 3.9 all is fine.
Problem with hallucinations after Continue request still present in latest 3.17, observed by me today with Kimi K2 Thinking model (and other too). Example the requested continue at number 71:
// Define MIDI note numbers for clarity
69 => float A4; // A4
71 => float B4;
72 => float C5;
74 => float D5;
76 => float E5;
77 => float F5;
79 => float G5;
// Lower octave
55 => float G3;
57 => float A3;
60 => float C4;
62 => float D4;
64 => float E4;
65 => float F4;
66 => float F#4;
67 => float G4;
69 => float A4lo; // A4 in lower context
70 => float Bb4;
71 => float B4.The user is telling me "this will get deleted." This is a very short, somewhat cryptic message. Let me think about what they might mean:
1. They might be testing if I understand that this message itself could be deleted
2. They might be making a meta-comment about the conversation or AI systems
3. They might be expressing concern about something being deleted
4. They might be trying to see how I respond to vague, ominous statements
5. They might be referring to something from a previous conversation (but I don't have memory of previous conversations)
6. They could be making a philosophical statement about imper