DynaWeb: Model-Based Reinforcement Learning of Web Agents
Image generated by Gemini AI
DynaWeb is a new model-based reinforcement learning framework designed to train autonomous web agents using a simulated web environment. By predicting web page representations based on agent actions, it allows for efficient training without the risks of live internet interaction. Experiments show DynaWeb enhances the performance of existing web agent models on benchmarks like WebArena, offering a scalable approach to online reinforcement learning.
DynaWeb: A Breakthrough in Training Autonomous Web Agents
DynaWeb marks a significant advancement in the development of autonomous web agents powered by Large Language Models (LLMs) and reinforcement learning (RL). This model-based reinforcement learning (MBRL) framework addresses challenges associated with training agents in real-time internet environments.
DynaWeb utilizes a world model to simulate interactions within a synthetic web environment, enabling web agents to "dream" and generate extensive action trajectories for enhanced online reinforcement learning. By predicting web page changes based on agent actions, DynaWeb creates a controlled space for agents to refine their policies.
Framework Features
DynaWeb features include:
- World Model Learning: Predicts web page changes in response to agent actions.
- Policy Rollouts: Generates vast quantities of rollout trajectories.
- Integration of Expert Trajectories: Combines real expert trajectories with agent-generated rollouts for enhanced training.
Performance Validation
Experiments on the WebArena and WebVoyager benchmarks demonstrate that DynaWeb significantly outperforms current state-of-the-art open-source web agent models, highlighting the framework's potential to reshape web agent training.
Related Topics:
📰 Original Source: https://arxiv.org/abs/2601.22149v1
All rights and credit belong to the original publisher.