Each section below summarizes one paper's reported method, experiments, and limits from its full review. The studies use different tasks and metrics; their results are not directly comparable.
arXiv:2611.10240
PACE: Keeping the Right Tokens in the KV Cache Is Not Enough for Long-Horizon LLM Agents
PACE argues that long-horizon KV-cache management must solve two separate problems: retaining the tokens a future agent step will need and keeping those tokens close enough in rotary-position space to remain usable. Its planner-aware retention converts dependency information into a foresight bonus, while bounded positions exactly re-rotate and renumber surviving keys so their rotary distance remains bounded. On a pre-registered synthetic coding benchmark, Qwen3-8B with a 2,048-token budget passed 97% of hidden checks at 100K tokens, compared with 42% for H2O and 3% for SnapKV. At 1M and 2M tokens, PACE passed 99% and 100% of checks on one 24 GB GPU while using under 0.7 GB of KV state.
How it works
PACE sits inside the inference engine's KV-cache manager and changes which tokens survive, which physical positions they carry, and how session state persists across turns and memory tiers. The planner sends a blueprint listing earlier message spans and the future steps that will read them. For a span with next reference s⋆, PACE adds a foresight bonus to the cumulative attention score. The reported scale is σ = 10^5, giving a floor of σ/2 = 50K, above the observed unreferenced cumulative-attention maximum of 25K in the 2M-token sessions.
What the paper evaluates
The paper evaluates a custom execution-scored long-horizon coding benchmark rather than a conventional static long-context dataset. Each session begins with three randomized definition steps specifying classes, fields, fee constants, an exception code, status values, and a fee rule. Approximately 2,000-token Python standard-library filler chunks then lengthen the session. Three late probes request fee_for, settle, and a complete module. Hidden unit tests execute the generated code, while a separate recall metric checks whether the answer contains the session's names, values, rounding mode, quantum, and status facts.
Limit to keep in view
The main benchmark is a synthetic coding-session task with randomized specifications and Python standard-library filler; other agent workloads may have different retention and distance behavior.
arXiv:2611.05139
Meta$^2$: Precomputing Reusable Knowledge for Online Harness Evolution in Long-Horizon Agents
Meta² learns reusable guidance for deciding when and how to modify an AI agent's harness during a long-horizon task. Offline, it contrasts trajectories from an agent without evolution, an evolver guided by the current Meta-Skill, and an evolver with privileged training-time verifier information; online, a frozen Meta-Skill guides KEEP or EVOLVE decisions at checkpoints while edits remain local to the active run. With GPT-5.6-Terra, Meta² reports 81.00% on Terminal-Bench 2.1 mean@5 and 44.44% on DeepSWE 1.1 mean@3, outperforming the evaluated baselines.
How it works
The online component starts from a minimal seed harness h^(0) and runs a task agent continuously in one environment. At checkpoint k, the evolver receives the instruction, newly accumulated trajectory, permitted workspace observations, and the status of the prior edit. It chooses KEEP or EVOLVE; an EVOLVE action applies a bounded notice, instruction update, hook, or tool edit to the remainder of the current run without resetting the environment or discarding progress. The offline component learns a textual Meta-Skill M.
What the paper evaluates
The evaluation uses Terminal-Bench 2.1 and DeepSWE 1.1, with each benchmark randomly split into training and test sets at a 1:2 ratio. Terminal-Bench reports mean@5 over five independent runs per held-out task, while DeepSWE reports mean@3 over three independent runs. The primary backbone is GPT-5.6-Terra with medium reasoning effort; an additional Terminal-Bench experiment uses DeepSeek-V4-Flash-0731 with high reasoning effort. Performance is higher-is-better, while aggregate dollar cost is reported separately for offline construction, online deployment, and the full evaluation.
Limit to keep in view
The contrast groups provide directional evidence but do not establish the causal effect of any individual checkpoint edit.
arXiv:2611.03523
DistDebug-Bench: Can LLM Agents Diagnose Bugs in Distributed Systems?
DistDebug-Bench evaluates whether LLM agents can locate the code-level causes of real failures in distributed systems before attempting a repair. Its 120 incidents span 11 systems, five programming languages, and 10 failure patterns, with 218 annotated buggy blocks. The main evaluation shows that agents often reach the right files but are less reliable at identifying precise causal blocks, particularly when reports omit direct code references or usable stack traces. GPT-6 Astra with Mini-SWE-Agent achieved 83.9% file coverage and 67.7% block coverage. Repeated runs also varied, and broader diagnoses could trade precision for coverage. The benchmark measures a distinct debugging step that patch-based evaluations can conflate with repair.
How it works
Benchmark instances are built from public issue reports and fixing changes in production-scale distributed systems. The authors retain closed issues with an associated fix and a concrete failure description. They favor realistic evidence, including logs, stack traces, reproduction steps, linked issues, or symptom descriptions, while removing or ignoring later thread comments that disclose the cause.
What the paper evaluates
The central benchmark comprises 120 common incidents across 11 distributed systems, five programming languages, and 10 failure patterns. The paper characterizes the evidence structure as well as aggregate performance: 96 reports contain code identifiers, 28 contain stack traces, and 20 contain logs. In 75 cases no usable project-internal stack trace is present. Of the 120 reports, 83 do not mention the exact gold method in the title or description. The labels contain 218 blocks; 45% of incidents need multiple blocks and nearly a quarter span multiple files. The main model sweep runs 13 configurations on Mini-SWE-Agent.
Limit to keep in view
The benchmark filters for issues whose evidence supports diagnosis and whose fixes permit precise labels, so its incidents do not represent every production debugging case.
arXiv:2611.05320
PyRUA-DataFarm: LLM Agent as a Cost-Efficient Teleoperator for Real-Ready Robot Data
PyRUA-DataFarm uses a coding agent to turn one real calibration episode and a task request into verified robot-training data. The agent reconstructs the scene in simulation, writes an expert controller and hidden success judge, generates exact actions and states, and uses a real-image-conditioned video model to render synchronized multi-view observations. On five real-robot tasks, policies trained only on farm data achieved 93 successes versus 84 for equally sized teleoperation datasets in the original scenes, and 87 versus 46 under unseen tablecloths and colored lighting. The average reported cost was $21 per task for adaptation, $5 for a render contract, $0.80 per additional episode, and $107 per hour of three-view data.
How it works
DataFarm separates a shared capability layer from task-specific artifacts. For each request, the agent produces grounding G, task contract T, sampler S, expert C, judge J, render contract R, and batch configuration B. Grounding identifies objects, geometry, articulation, cameras, and metric scale; quantities that cannot be determined from the calibration episode are marked as measured, fitted, assumed, or unknown and are handled through simulation randomization rather than unsupported certainty. The workflow behaves like a compiler.
What the paper evaluates
The evaluation is task-based rather than on a named public dataset. It uses five bimanual tasks on a dual-arm PiPER robot: stack cups, weigh apple, beat drum, pull drawer, and fold towel. Rigid and articulated tasks use SAPIEN through RoboTwin 2.0; fold towel uses Isaac Sim with particle cloth. Each task starts from one calibration episode with a head camera and two wrist cameras and requests 100 generated episodes. Policies are trained on three equal-size sources of N=100 episodes with a shared 50-step action horizon, batch size 128, and 50k training steps: real teleoperation, simulator rendering with domain randomization, and farm-rendered observations.
Limit to keep in view
The method depends on a simulator that can reproduce the required skills; tasks that fail simulator qualification need a new simulator variant before data can be produced.
Read the full reviews
- PACE: Keeping the Right Tokens in the KV Cache Is Not Enough for Long-Horizon LLM Agents · arXiv:2611.10240
- Meta$^2$: Precomputing Reusable Knowledge for Online Harness Evolution in Long-Horizon Agents · arXiv:2611.05139
- DistDebug-Bench: Can LLM Agents Diagnose Bugs in Distributed Systems? · arXiv:2611.03523
- PyRUA-DataFarm: LLM Agent as a Cost-Efficient Teleoperator for Real-Ready Robot Data · arXiv:2611.05320
Source note: findings and limitations above come from the linked full paper reviews. Follow each original arXiv paper for the authors' complete methods and results.