Adaptive Training for Robot Learning

Overview

My research asks how a teacher can understand a robot student and adapt three parts of training: tasks, auxiliary rewards, and simulation conditions. GACL and Reward Training Wheels (IROS 2025) address the first two decisions; LUCID (preprint, 2026) extends the agenda to execution-informed domain randomization. These are complementary methods with distinct mechanisms and evaluations.

Explore the animated research story for teacher–student diagrams, classroom analogies, a scheduling demo, and linked results.

Grounded Adaptive Curriculum Learning (GACL)

GACL (Wang et al., 2025) (with Zifan Xu, Peter Stone, and Xuesu Xiao) introduces a teacher-student paradigm in which an informed teacher generates training tasks by monitoring student performance in real time, while grounding the curriculum in limited reference samples from the target task distribution so training remains relevant to deployment.

Diagram of the GACL teacher-student curriculum-learning framework

The GACL teacher adapts task difficulty from live student performance while grounding tasks in the target distribution.

Published results: 6.8% higher success rate than state-of-the-art curriculum methods on wheeled navigation in constrained environments, and 6.1% higher on quadruped locomotion in challenging 3D confined spaces.

Reward Training Wheels (RTW)

RTW (Wang et al., 2025), co-first-authored with Tong Xu, automates auxiliary reward shaping: a teacher adaptively weights auxiliary reward components as the robot’s proficiency grows, while the primary objective stays fixed. Weights can rise or fall; they are not constrained to fade monotonically.

Diagram of the Reward Training Wheels teacher-student framework

The RTW teacher observes the student's reward history and re-weights auxiliary rewards each iteration.

Published results: In simulation, RTW achieved a 2.35-point increase in navigation success and a 122.62% relative improvement in off-road mobility on vertically challenging terrain, reaching the respective performance thresholds about 35% and 3× faster. Sim-trained policies achieved 5/5 physical off-road trials versus 2/5 for expert-designed rewards, with up to 47.4% reduction in orientation angles (more stable poses).

LUCID: pacing domain randomization

LUCID is a 2026 preprint on humanoid motion tracking. A frozen temporal encoder compares issued-command and measured-execution histories. A bounded PI scheduler adjusts shared randomization intensity, with return-based backoff when training degrades.

The study reports 88.9% vs. 76.8% full-randomization simulation completion against filtered-error PI (Table III). On the physical Unitree G1 with 40 ms added delay, completion was 38/60 vs. 23/60 across four motions, three checkpoints, and five repetitions (Table VII). These are preprint study results. Only the policy and low-level controller are needed at deployment.

Platforms

Wheeled ground robots navigating highly constrained spaces
Quadruped locomotion in confined 3D environments
Off-road vehicles on vertically challenging terrain
Humanoid motion tracking in LUCID simulation and physical G1 experiments (preprint)

Papers

L. Wang, Z. Xu, P. Stone, X. Xiao. “GACL: Grounded Adaptive Curriculum Learning with Active Task and Performance Monitoring,” IROS 2025. arXiv
L. Wang, T. Xu, Y. Lu, X. Xiao. “Reward Training Wheels: Adaptive Auxiliary Rewards for Robotics Reinforcement Learning,” IROS 2025. arXiv

References

2025

IROS
GACL: Grounded Adaptive Curriculum Learning with Active Task and Performance Monitoring

Linji Wang, Zifan Xu, Peter Stone, and Xuesu Xiao

In 2025 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2025

Abs DOI arXiv Bib PDF

Curriculum learning has emerged as a promising approach for training complex robotics tasks, yet current applications predominantly rely on manually designed curricula, which demand significant engineering effort and can suffer from subjective and suboptimal human design choices. While automated curriculum learning has shown success in simple domains like grid worlds and games where task distributions can be easily specified, robotics tasks present unique challenges: they require handling complex task spaces while maintaining relevance to target domain distributions that are only partially known through limited samples. To this end, we propose Grounded Adaptive Curriculum Learning, a framework specifically designed for robotics curriculum learning with three key innovations: (1) a task representation that consistently handles complex robot task design, (2) an active performance tracking mechanism that allows adaptive curriculum generation appropriate for the robot’s current capabilities, and (3) a grounding approach that maintains target domain relevance through alternating sampling between reference and synthetic tasks. We validate GACL on wheeled navigation in constrained environments and quadruped locomotion in challenging 3D confined spaces, achieving 6.8% and 6.1% higher success rates, respectively, than state-of-the-art methods in each domain.
@inproceedings{wang2025gacl, title = {GACL: Grounded Adaptive Curriculum Learning with Active Task and Performance Monitoring}, author = {Wang, Linji and Xu, Zifan and Stone, Peter and Xiao, Xuesu}, booktitle = {2025 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)}, year = {2025}, pages = {591--596}, doi = {10.1109/IROS60139.2025.11246910}, }
IROS
Reward Training Wheels: Adaptive Auxiliary Rewards for Robotics Reinforcement Learning

Linji Wang, Tong Xu, Yuanjie Lu, and Xuesu Xiao

In 2025 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2025

Abs DOI arXiv Bib PDF

Robotics Reinforcement Learning (RL) often relies on carefully engineered auxiliary rewards to supplement sparse primary learning objectives to compensate for the lack of large-scale, real-world, trial-and-error data. While these auxiliary rewards accelerate learning, they require significant engineering effort, may introduce human biases, and cannot adapt to the robot’s evolving capabilities during training. In this paper, we introduce Reward Training Wheels (RTW), a teacher-student framework that automates auxiliary reward adaptation for robotics RL. To be specific, the RTW teacher dynamically adjusts auxiliary reward weights based on the student’s evolving capabilities to determine which auxiliary reward aspects require more or less emphasis to improve the primary objective. We demonstrate RTW on two challenging robot tasks: navigation in highly constrained spaces and off-road vehicle mobility on vertically challenging terrain. In simulation, RTW outperforms expert-designed rewards by 2.35% in navigation success rate and improves off-road mobility performance by 122.62%, while achieving 35% and 3X faster training efficiency, respectively. Physical robot experiments further validate RTW’s effectiveness, achieving a perfect success rate (5/5 trials vs. 2/5 for expert-designed rewards) and improving vehicle stability with up to 47.4% reduction in orientation angles.
@inproceedings{wang2025reward, title = {Reward Training Wheels: Adaptive Auxiliary Rewards for Robotics Reinforcement Learning}, author = {Wang, Linji and Xu, Tong and Lu, Yuanjie and Xiao, Xuesu}, booktitle = {2025 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)}, year = {2025}, pages = {15262--15267}, doi = {10.1109/IROS60139.2025.11247039}, }