The problem
The Qwopus 3.6 Coder branch had been trained on a heavy code curriculum, and it broke the model's chain of thought: loops, runaway thinking, and eventually thinking had to be disabled altogether. A coding model that can't reason through a problem is only half useful.
The insight
Coder was a downstream finetune of Qwopus v2, the stable reasoning model. That means Coder − V2 is a clean task vector: the code training, isolated from everything else. You don't have to take all of it.
The method
A depth-weighted delta merge. Each layer gets v2 plus a fraction of the coder's delta, and the fraction ramps with depth:
W(L) = V2 + α(L) · (Coder − V2), α: 0.12 → 0.48
Embeddings, lm_head, norms, and the MTP heads are frozen from v2, the stable reasoning parent.
W(41) =V2 +0.349 ·(Coder − V2)
v2 (stable reasoning) 65.1% Coder 34.9%
Each layer keeps most of the stable reasoning model (v2) and takes a share of the coder's difference from it. Written another way, each layer is (1 − α)·V2 + α·Coder, which is what the bar shows. The share rises with depth, from 12% in the first layer to 48% in the last, so the early layers stay closest to the stable reasoning parent. Embeddings, the output head, norms, and the MTP heads are copied from v2 untouched.
Ablations
Two alternatives were tested on the same SWE slice and published on the card: DARE-TIES and a frankenmerge. Both underperformed the depth-weighted delta merge.
Results at Q4_K_M
| Measure | Result |
|---|---|
| HumanEval | 94.5% |
| MBPP | 87.9% |
| GSM8K | 95.0% |
| Termination across temperature sweeps | Clean |
| Needle tests at 60k context | Stable |
| Speed, single RTX 5090, MTP speculative decoding | ~100 tok/s |
Why it matters for the research
Fusion came straight out of the chain-of-thought efficiency workstream. It's the case where reasoning discipline and domain skill pulled against each other, and a careful merge kept both.