State commitment learning: training language models to distinguish computation from memory
The paper trains language models to separate persistent state from temporary computation, using counterfactual erasure to reduce reliance on hidden thoughts.
Intelligence analysis by GPT-5.4 Mini
The authors argue that reasoning models often blur scratch work and durable state, so later answers can depend on dead ends. They introduce state commitment learning, which rewards answers that still work after hidden thoughts are erased.
It is like teaching a student to write only the important answer on the final page, not to depend on messy scratch paper. The paper says a model should still be right even after those rough notes are erased.
Analysis
What the paper is trying to fix
The paper treats a familiar problem in reasoning models as a training issue: once a model generates hidden thoughts, those tokens stay in context and can influence later predictions. That means a final answer may rely on failed attempts, dead ends, or private scratch work rather than on information that is actually safe to keep.
The proposed objective
The authors call their approach state commitment learning. The goal is to teach a model to distinguish between information that should become persistent state and information that is only temporary computation. To make that trainable, they define persistent-state sufficiency, a criterion for checking whether an answer is still usable after hidden thoughts are erased.
How CERL works
They then introduce Counterfactual Erasure RL (CERL). Under the same prefix, CERL evaluates two paths: one where hidden thoughts are kept, and one where they are erased. Reward is given only when the erase path still produces a correct answer. In other words, the model is pushed toward answers that do not need scratch work to remain valid.
What the paper reports
The paper says CERL was tested across mathematics, long-chain logic, scientific QA, and multi-turn tool-use evaluation. In those settings, it substantially reduced answer dependence on hidden thoughts without sacrificing accuracy. The abstract also says it consistently outperformed correctness-only RL and long-answer SFT baselines.
The core claim is narrow but important: a reasoning model should not just reach the right answer, it should reach an answer that remains right even when its private scratch work is removed.
Key points
- The paper argues reasoning models mix temporary computation with persistent state.
- It proposes state commitment learning to separate the two.
- Persistent-state sufficiency is the paper’s trainable criterion for checking whether an answer still works after hidden thoughts are erased.
- CERL rewards answers that remain correct even when hidden thoughts are removed.
- The abstract says the method reduces dependence on hidden thoughts without hurting accuracy.
If the method works broadly, language models could become more reliable because their answers would depend less on private scratch work. That could help in tasks like math, long reasoning, science questions, and tool use where hidden mistakes often leak into the final answer.
The reported gains are based on the paper’s own evaluation setup, so the approach may not cover every real-world failure mode. If the erasure protocol does not match how models are actually used, the improvement could be narrower than the abstract suggests.
