📌 Highlights
OMCE turns experience management into an online routing problem. A contextual-bandit controller chooses whether to act directly, retrieve an episode, use guidance, or distill guidance from retrieved episodes. Task feedback updates the policy and the memory, so experience use can change as the stream progresses.
The authors evaluate four benchmarks across four backbones. They report the highest first-attempt success on 11 of 16 benchmark–backbone configurations. On GPT-5.2, the routing ablation reaches 67.4% OSR on MultiChallenge and 53.3% on BBEH; the winning routing variant differs on BigCodeBench. The paper also reports 5.7–62.5 times lower per-task token use than ACE with GPT-5.2.
The result is evidence for selective routing in the evaluated setup, not a guarantee that memory helps every task. The authors note dependence on timely feedback, stale policies under distribution shift, memory growth, and the absence of rewards for routes that were not selected.
- Four actions make invocation, memory format, and distillation explicit controller choices.
- Reported benchmarks span conversation, general reasoning, code generation, and repository-level software engineering.
- Controlled analyses of memory use and controller freezing focus on GPT-5.2.
🎯 Introduction
Experience-oriented LLM agents retain interaction feedback for later tasks, storing it as concrete trajectories, reusable skills, or more abstract guidance. The paper argues that availability alone does not ensure utility: semantic similarity can retrieve unhelpful experience, concrete cases may not transfer to different tasks, and generalized principles may omit details needed for execution. Expanding a guidance store also costs computation and may cause memory growth.
The paper identifies three unresolved control decisions: whether to invoke experience, which form to use, and when to consolidate episodes into guidance. It positions this as different from controllers that allocate fixed external resources such as tools or reasoning budgets, because an agent's own experience is heterogeneous and changes over time.
OMCE's objective is to learn these decisions from task feedback during test-time adaptation. It combines episodic and guidance memories with a lightweight meta-controller, then evaluates whether learned routing improves online task success and token efficiency against memory and adaptation baselines.
🔬 Methodology
For each query in a sequential stream, OMCE retrieves candidate episodic insights and guidance entries by cosine similarity between the query embedding and each entry's applicable-scenario description. Episodic records contain the task trajectory, immediate reward, and case-specific insights. Guidance entries encode principles intended to transfer across related cases. The controller summarizes retrieval relevance, quality, and episodic outcome diversity in a compact state rather than reading raw memory text.
The controller chooses among DIRECT, EPISODE, GUIDE, and DISTILL. DIRECT supplies the query without retrieved experience. EPISODE injects retrieved cases; GUIDE injects retrieved principles. DISTILL first uses an LLM to derive a guidance entry from retrieved episodes, stores it, and injects it. The resulting context conditions the task-solving LLM. The paper uses action-wise linear reward models with Thompson Sampling or LinUCB selection, including an exploration term. The controller does not require a separate LLM call for action choice.
After execution, OMCE records every outcome in episodic memory and extracts case-specific insights. Following GUIDE or DISTILL, it can refine injected guidance using the trajectory and reward. Bandit statistics update from the selected action's reward only, with a forgetting factor. Thus controller adaptation and memory evolution proceed together, although the paper's controlled experiments also examine them separately.
equation 1
Meaning: Defines the retrieval-derived controller state as eight features and a constant bias.
equation 2
Meaning: Specifies how each chosen action forms the context supplied to the task-solving agent.
equation 3
Meaning: Defines the per-action parameter estimate and exploration-adjusted action choice.
equation 4
Meaning: Shows how selected-action feedback updates bandit statistics and how older evidence is discounted.
📊 Experiments
The evaluation uses four online-adaptation benchmarks: MultiChallenge for multi-turn conversation, BBEH for advanced general reasoning, BigCodeBench for code generation, and SWE-bench Verified for repository-level software engineering. Four backbone models are evaluated: GPT-5.2, DeepSeek-V3.2, DeepSeek-V4-pro, and Kimi-K2.5. The comparison includes a base LLM, Reflexion, MemP, ReasoningBank, ACE, and OMCE variants. One-shot success rate (OSR) is the primary metric and measures success on the first attempt; cumulative success rate (CSR) measures success within P = 4 attempts. Reflexion is reported with CSR only because its first attempt matches the base model under the protocol.
The authors report that OMCE has the highest OSR on 11 of 16 benchmark–backbone configurations. On MultiChallenge it leads ReasoningBank, ACE, and MemP for all four backbones by 13.5 to 20.5 percentage points over the strongest of those baselines. On SWE-bench Verified it leads the same baselines across all four backbones by 1.8 to 10.0 points. It matches or leads the strongest baseline on three BBEH backbones, while BigCodeBench results are less consistent: OMCE leads with GPT-5.2 but trails MemP on the other three backbones.
The GPT-5.2 routing ablation shows that no fixed memory form wins throughout: fixed episodic use has 50.3% OSR on BBEH versus 43.1% for fixed guidance use, while the ordering reverses on MultiChallenge, 56.0% versus 63.7%. The controlled analysis reports lower OSR for Always-Both and Random than OMCE on each benchmark; freezing the controller also lowers OSR in all four cases. CSR changes are not uniformly aligned with OSR, including an increase under frozen control on BigCodeBench. The efficiency analysis uses average prompt and output tokens per task and reports OMCE's GPT-5.2 token use as 5.7–62.5 times lower than ACE. These are paper-reported results; the supplied evidence does not provide uncertainty intervals.
Table 1
Caption: Main results under the online adaptation setting. Each entry is reported as OSR / CSR (%), where CSR is measured within P = 4 attempts. For Reflexion, “-” denotes that no first-attempt evaluation applies, as explained in the metric definitions above.
| Benchmark | Fixed-aEPISODE OSR / CSR | Fixed-aGUIDE&DISTILL OSR / CSR | OMCE-HEU OSR / CSR | OMCE-TS OSR / CSR | OMCE-UCB OSR / CSR |
|---|---|---|---|---|---|
| MultiChallenge | 56.0% / 91.2% | 63.7% / 94.5% | 64.5% / 95.2% | 67.4% / 97.1% | 66.3% / 95.6% |
| BBEH | 50.3% / 76.8% | 43.1% / 73.8% | 45.6% / 71.7% | 53.3% / 78.7% | 46.8% / 71.0% |
| BigCodeBench | 49.0% / 59.0% | 49.6% / 58.0% | 47.4% / 57.4% | 49.6% / 58.9% | 51.3% / 59.2% |
| SWE-bench Verified | 64.2% / 64.8% | 64.4% / 64.8% | 64.0% / 64.8% | 64.8% / 64.8% | 64.8% / 64.8% |
Why It Matters: Shows the GPT-5.2 routing comparison, where the better fixed or learned action strategy depends on the benchmark.
Table 3
Caption: Controlled analysis on GPT-5.2, OSR / CSR (%). For each benchmark, uses the variant with the higher GPT-5.2 OSR in tab:main_results (TS on ties). Frozen Controller stops controller updates mid-stream while memory keeps evolving; Always-Both and Random read one shared memory snapshot without updating it.
| Benchmark | Variant | OMCE | Always-Both-and-Update | Frozen Controller | Always-Both | Random |
|---|---|---|---|---|---|---|
| MultiChallenge | OMCE-TS | 67.4% / 97.1% | 63.7% / 95.2% | 61.9% / 94.1% | 63.0% / 83.5% | 62.6% / 83.9% |
| BBEH | OMCE-TS | 53.3% / 78.7% | 51.1% / 77.6% | 49.8% / 75.6% | 49.7% / 71.8% | 48.2% / 67.4% |
| BigCodeBench | OMCE-UCB | 51.3% / 59.2% | 48.3% / 53.5% | 49.5% / 65.4% | 47.5% / 55.6% | 46.5% / 57.0% |
| SWE-bench Verified | OMCE-TS | 64.8% / 64.8% | 55.0% / 55.2% | 64.2% / 64.6% | 48.8% / 49.2% | 53.0% / 53.4% |
Why It Matters: Tests routing and continued updates against controlled memory-use conditions on GPT-5.2.
pic:ArchitectureOverview
Caption: Architecture of . The lightweight Meta-Controller selects a cognitive action specifying whether and which experience to inject into the task prompt. The agent solves the task with the augmented context containing the query and any injected experience. Memory evolution records outcomes in episodic memory, distills guidance before execution when DISTILL is selected, and refines injected guidance afterward using task feedback.
Why It Matters: Shows how task execution, retrieval, routing, and memory updates connect in the proposed online loop.
pic:performance_vs_efficiency
Caption: Performance--efficiency trade-off on four benchmarks using GPT-5.2. The x-axis represents the one-shot success rate, and the y-axis reports the token cost in log scale (average total prompt and output tokens consumed per task).
Why It Matters: Visualizes the reported success-versus-token-cost comparison across all four benchmarks.
fig:case_study
Caption: Self-correction detection in MultiChallenge. The meta-controller routes the query to GUIDE ($a_ GUIDE$) rather than to episodic cases that enforce the user's outdated constraint.
Why It Matters: Illustrates a concrete failure mode of indiscriminate episodic retrieval and the paper's routing response.
🔮 Conclusion
The paper presents OMCE as a way to move experience-augmented agents from fixed retrieval and consolidation routines toward learned control over memory use. Its controller jointly decides whether to use experience, which form to inject, and when to distill guidance, adapting from observed task feedback.
Across the reported benchmarks and backbones, the results favor OMCE on first-attempt success in most configurations and show a reported token-cost advantage over ACE with GPT-5.2. The evidence also shows benchmark-dependent performance and mixed CSR effects, so the result supports adaptive routing in the evaluated protocol rather than a universal guarantee that a learned memory controller will improve every task stream.
🛠️ Future Research Improvements
The authors identify several concrete next steps. Because the bandit receives feedback only for the selected route, reflective estimates of counterfactual rewards for unchosen routes could provide additional learning signals; the paper presents this as a possible approach, not a demonstrated result. Timely feedback is another dependency, so evaluation with sparse or delayed rewards would clarify how reliably the policy adapts when outcomes arrive late.
The authors also point to memory aging or eviction to address redundancy and semantic collisions as the stores grow, and suggest explicit memory auditing and pruning. These mechanisms should be evaluated for their effects on retrieval quality, routing decisions, and cumulative task success. Distribution-shift tests would help determine how quickly stale experience and controller parameters can recover.
Finally, the controlled comparisons of memory use and continued policy updates focus on GPT-5.2. Repeating those interventions with the other reported backbones would test whether the component-level findings generalize. The paper does not establish which proposed memory-maintenance or counterfactual methods will work best.
🏭 Potential Industry Use Scenarios
The paper's benchmarks suggest candidate settings where experience routing could be worth testing: conversational assistants that must reconcile a user's updated constraints, coding assistants that reuse prior task cases or repository-level lessons, and general reasoning agents that operate over repeated task streams. The MultiChallenge case study illustrates the conversation scenario: guidance about handling corrections helps avoid following an outdated constraint preserved in episodic examples.
For a product team, OMCE's main operational value would be choosing when to add stored context and when to turn repeated cases into reusable guidance. This could matter when prompts have cost or context limits and when tasks recur often enough for feedback to improve later decisions. The reported token comparison with ACE provides a reason to measure this trade-off in deployment-like workloads.
These are plausible evaluation targets, not evidence of production deployment. The paper reports benchmark experiments rather than live service outcomes, and its approach assumes feedback can be obtained after tasks. Builders should first verify feedback availability, task-distribution stability, memory growth controls, and the cost of LLM-based insight extraction, distillation, and refinement in their own environment.
💬 Critical Analysis
A strength of the design is that it makes memory policy explicit and comparatively inexpensive at the controller level: retrieval signals feed small action-wise linear models, and action selection does not require another LLM call. The evaluation is broad across four benchmark domains and four backbone models, while ablations compare fixed memory forms, a heuristic, learned variants, shared-memory policies, and a frozen controller. The authors also distinguish first-attempt success from success within a four-attempt budget, which matters because retry-based methods can have high CSR without strong OSR.
The findings are not uniform. OMCE's reported first-attempt lead depends on benchmark and backbone, and its two learned variants do not have a single consistent winner. On BigCodeBench, for example, OMCE trails MemP on three backbones. In the controlled GPT-5.2 analysis, freezing the controller lowers OSR but raises CSR on BigCodeBench, showing that conclusions depend on the metric. The paper reports no uncertainty intervals in the supplied main-body evidence, so the stability of close differences is difficult to judge from these figures alone.
Reproduction also depends on implementation details that should be checked in the full paper and code: retrieval candidate construction, reward mapping, exploration configuration, memory prompts, and token-accounting boundaries. The reported controlled interventions for routing and memory evolution are limited to GPT-5.2, leaving open whether those causal interpretations transfer across backbones. Finally, online learning assumes feedback is timely and informative, while only selected routes receive observed rewards. Memory growth, feedback delays, or changing task distributions could weaken both the stored knowledge and the learned policy.