Position: Deployed Reinforcement Learning Should Be Continual
The paper argues deployed RL systems should keep learning after launch, not freeze until they break. It says post-deployment non-stationarity makes continual learning necessary.
Intelligence analysis by GPT-5.4 Mini
This position paper pushes back on the common train-then-fix approach in reinforcement learning. It argues that once an agent is deployed and still receiving reward feedback, the problem is inherently continual because the environment keeps changing.
The paper says a robot or software helper that learns from rewards should not stop learning after its first lesson. It is like a kid learning to ride a bike in a park where the ground keeps changing, so it has to keep adjusting.
Analysis
Core claim
The paper argues that deployed reinforcement learning should not be treated as a static system that is trained once and only updated after performance degrades. Instead, if an agent is operating in the world and still gets an evaluative reward signal, the paper says it is already a continual RL problem.
Why the authors say that
The abstract says deployed systems face non-stationarity after launch, which means the world around them changes in ways that can make earlier training insufficient. The paper identifies four sources of that non-stationarity, though the abstract does not list them individually. On that basis, the authors say the best deployed agents are the ones that keep adapting rather than stopping learning.
What the paper contributes
This is a position paper, so it is making an argument rather than presenting a new benchmark or model. It points to successful real-world examples of continual RL and uses those examples to support a broader shift away from the current train-then-fix paradigm. It also says the community should adopt advantages and measures that encourage continual adaptation after deployment.
Scope and limits
The source provided here is the abstract and metadata from arXiv, so only the claims visible there can be stated confidently. The paper is also marked as accepted to the ICML 2026 Position Paper Track, which signals that it is being presented as a field-level argument for how deployed RL should be understood and practiced.
Key points
- The paper argues deployed reinforcement learning is inherently a continual learning problem.
- It criticizes the common train-then-fix approach used in real-world RL systems.
- The authors say post-deployment non-stationarity makes ongoing adaptation necessary.
- The paper points to real-world examples of continual RL to support its case.
- It was accepted to the ICML 2026 Position Paper Track.
If the paper's view catches on, deployed RL systems could become better at coping with changing real-world conditions. That could reduce the need for harsh retraining cycles after performance drops.
If teams keep using train-then-fix systems, deployed agents may drift away from what the world now needs. The paper also implies that continual learning in deployment raises harder operational demands, because systems must keep adapting safely.
