VRL-Bench

A UNIFIED SIMULATOR FOR AGENT MEMORY SYSTEMS

Memory can help.
It can also hurt.

Does experience make your agent better?
VRL-Bench lets you find out through fair trial-and-error simulation.

Code Get started Paper Explore the replay

TRIAL-AND-ERROR REPLAY

See what the agent saw.

Step through its actions alongside the memory it received and the trial and action budgets.

RECORDED RUN

WebArena · Reflexion · GPT-4.1

Replay your runs ↗
RECORDED PAGENative page snapshot at the selected replay positionEXECUTED ACTION

Actual saved pages, actions, and memory. Playback makes no model calls.

CONTROLLED MEMORY EVALUATION

Memory gains are not automatic.

Across three models on MiniWoB and WebShop, leading memory methods improve success in some settings and reduce it in others. Fair trial budgets make these differences visible.

All eighteen success-rate changes versus memory-free Retry for Reflexion, DC-Cu* and ACE*, across three models on MiniWoB and WebShop. Positive and negative changes appear, with paired 95 percent confidence intervals.
Success-rate change versus Retry within six trials, in percentage points. Bars: paired 95% confidence intervals. Filled points: intervals excluding zero.

Read the findings and experimental settings

EXPLORATION & EXPLOITATION

When should experience
be replayed?

Retained experience can guide an agent—and displace useful exploration. Reflection-exposure experiments revealed this tension in verbal reinforcement learning.

Control experience exposure with the Replay gate, or study an integrated scheduler such as VEX².

Explore the exposure experiments ↗
Fixed reflection exposure experiments on AlfWorld and HotPotQA show that more replay does not always yield greater success.

A FAIR PLACE TO STUDY AGENT MEMORY

Make experience testable.

In verbal reinforcement learning, an agent turns interaction feedback into language-based experience for later attempts. VRL-Bench provides a common simulation workflow for developing and comparing these memory systems.

CONTROL

Match the conditions.

Keep actors, task cases, reset rules, trial budgets, and interaction limits consistent while changing the memory method.

OBSERVE

Follow the process.

Inspect what reaches the next trial, what the agent does, and how it uses its remaining opportunities.

MEASURE

Evaluate the difference.

Measure success, recovery, and trials to solve. Use paired intervals, exact tests, and conditional power to interpret comparisons.

Real-world tasks and the same agent connect to a trial-and-error simulator with replaceable memory methods, models and environments, controlled replay exposure, and statistical analysis. The evaluated memory implementation returns to the original agent.
Start from real tasks. Develop and evaluate memory in simulation. Reuse the evaluated implementation in the same real agent. Full architecture ↗
ONLINE ENVIRONMENTSMiniWoBWebShopWebArenaOSWorldYour tasks ↗

BUILD ON A COMMON FOUNDATION

Your next memory system
starts here.

Use a reference method, connect your own implementation, and improve it from task-level feedback. Compare a new version with the previous one under the same conditions.

Reflexion NeurIPS 2023DC-Cu* EACL 2026ACE* ICLR 2026VEX² Exploration–exploitation schedulingYour method Build, evaluate, iterate ↗

* Adapted to failure-only trial and error. Memory-free Retry is included as a common exploration baseline.

01

Establish a baseline.

Run memory-free Retry on your tasks.

02

Add your memory.

Evaluate the complete implementation.

03

Improve from feedback.

Compare each candidate with its predecessor.

Run. Inspect. Compare.

Start with a real MiniWoB browser task. The example configurations save page snapshots for replay; use the same workflow with other prepared environments.

Get started
AFTER ENVIRONMENT AND MODEL SETUP
vrlbench simulate --config examples/configs/miniwob_retry.json
vrlbench simulate --config examples/configs/miniwob_reflexion.json
vrlbench replay runs/miniwob_retry runs/miniwob_reflexion
vrlbench compare runs/miniwob_retry runs/miniwob_reflexion

Simulations call your configured model. Saved-run replay makes no model calls.

Run a simulationEnvironment setup and a complete comparisonDevelop a memory systemIntegration, candidate evaluation, and feedbackAnalyze resultsMetrics, paired inference, and conditional power

Latest news