IBM Research Evaluates Agentic Memory Calibration

IBM Research has released findings on the use of ALTK-Evolve, a method that allows AI agents to learn from their own past trajectories by distilling behavioral guidelines without updating model weights or requiring human annotation. The research suggests that agentic memory is not a universal feature but a variable that must be calibrated based on a model's tier.

Testing across eight models on the AppWorld benchmark, researchers identified three distinct patterns. Strong models with performance headroom benefit most from receiving a full set of distilled guidelines. Conversely, smaller or weaker models perform better with a selective approach, utilizing a compact core of guidelines supplemented by task-specific retrieval. This selective method proved both more accurate and cost-effective for weaker models, increasing task completion by 16.1 percentage points with only a 5% increase in token usage. Meanwhile, already-saturated models showed no measurable gains from additional memory. These findings highlight that memory dosage should be tailored to a model's specific capabilities to optimize performance and efficiency.