How Much Memory Does Your Agent Actually Need?

The Right Dose of Memory Depends on the Model

When it comes to equipping an agent with agentic memory, it's not a simple matter of switching it on. Instead, the right dose of memory depends on the model's capability. In a study by IBM Research, eight models spanning the capability spectrum were evaluated to determine the optimal amount of memory required for each.

Strong Models Want the Full Guideline Set

Strong models with headroom want the full guideline set, including rare edge-case lessons. They have the capacity to absorb and apply all of it. DeepSeek-V3.2 (671B MoE) climbed +9.5 percentage points in task completion when given its full self-mined guideline set.

Smaller or Weaker Models Get Drowned by a Large Guideline Set

Smaller or weaker models get drowned by a large guideline set. For these, a tight, high-confidence core plus a handful of task-relevant guidelines retrieved per task works best. gpt-oss-120b (117B MoE) gained +16.1pp with this selective approach — while the full guideline set gained less and cost ~50% more tokens.

Already-Saturated Models Show No Measurable Gain

Already-saturated models show no measurable gain. We call this the saturated pattern — the label describes what we observed, not a proven cause. The model may already have been near its ceiling on these tasks, the guidelines may not have addressed its remaining failures, or it may not have applied the guidance effectively. GLM-5 (745B MoE) sat here in our runs.

The Practical Takeaway

The practical takeaway holds either way: the right dose of memory depends on the model, and we can calibrate it. Learning happens around the model, not inside it. 'Memory' here doesn't mean replaying a past transcript. It means a guideline set — strategies that worked, mistakes to avoid, and edge cases — distilled from the agent's own prior trajectories.

Results Across the Spectrum

We evaluated on AppWorld — 585 multi-step tasks (168 test_normal + 417 test_challenge ) across 9 simulated apps (calendars, messaging, payments, and so on). Tasks are scored two ways: whether the agent fully completes each task ( TGC — Task Goal Completion ) and whether every variant of a scenario passes ( SGC — Scenario Goal Completion , a stricter, all-or-nothing bar). Full definitions are in the appendix.

The Three Configurations

Because the confusing part of any memory study is what's actually in the context window , we define the configurations up front. Both memory configurations draw from the same guideline set , mined once (via the loop above) from AppWorld's training split only. What changes between them is only how that one set is delivered — the full guideline set injects all of it every step, while curated retrieval delivers a selected subset — never how the guidelines were produced, and no test-split data ever goes into building it.

Configuration

Configuration What's in the agent's context

Baseline No memory — the agent as shipped.

Full guideline set Every mined guideline, injected on every ReAct step.

Curated retrieval A fixed, high-confidence core of those same guidelines plus a few task-relevant ones retrieved for each task (a fixed portion + a variable portion).

The Three Patterns, in One View

Representative models from the eight-model sweep, measured by task completion (TGC) on test_normal : Figure 1. Represe