A teacher does more than assign harder homework. They choose useful exercises, change the guidance, and introduce challenging conditions when the learner can benefit.
My research brings this principle to reinforcement learning: use evidence about the robot to adapt its training.
A FAMILIAR IDEA, A ROBOTICS QUESTION
01
Choose the next exercise.
Give a student practice that addresses what they have not yet mastered, while staying relevant to the subject.
TASKS
02
Adjust the guidance.
Change which hints and intermediate steps receive attention, while keeping the learning objective the same.
REWARDS
03
Prepare for less ideal conditions.
Move from quiet practice to distractions and time pressure at a pace the student can handle.
CONDITIONS
An analogy for the research agenda. Each method has its own signals, controller, and experiments.
THE METHODS / THREE PARTS OF TRAINING
What changes when the teacher understands more?
The shared principle is student-aware adaptation. The implementation depends on the decision being made.
01 / ADAPT THE TASKS
GACL
Choose useful practice. Keep it grounded.
Robot task spaces are complex, and generated challenges can drift away from the target domain. GACL represents tasks in a learned latent space and gives its teacher task and performance history.
The teacher proposes training tasks; alternating reference and generated samples keeps the curriculum connected to the environments that matter.
FROM FIGURE 2 · SIMPLIFIED INTERACTIVE SCHEMATICTask curriculum
▦Reference tasks → VAEA learned representation of task structure
↓
◎Teacher + learner historyTask context and student performance
↓
⇄Grounded practice → PPO studentAlternate reference and generated tasks
↶ Performance returns to the teacher
A pretrained VAE encodes the task space. The curriculum operates on structured task representations.
See the paper diagram and technical detail
The full method includes a PPO student and antagonist, a teacher over task representations, and alternating sampling. This teaching diagram omits those lower-level connections; the paper is the technical reference.
02 / ADAPT THE REWARDS
Reward Training Wheels
Keep the destination. Adapt the guidance.
A fixed set of auxiliary rewards can become poorly matched to a changing student. RTW keeps the primary objective and learns how much emphasis to give auxiliary reward components.
Its teacher observes previous weights and reward histories. The action is a new weight vector for the student's learning process.
FROM FIGURE 1 · SIMPLIFIED INTERACTIVE SCHEMATICReward curriculum
Primary objective+Weighted auxiliary rewards
Observe the learning historyPrevious weights, primary rewards, auxiliary rewards
↓
Teacher chooses auxiliary weights
Illustrative weights · not an experimental trace
↓
Student learns with adjusted guidanceTask performance feeds the next teacher decision
The teacher receives learning history, rather than assuming the same shaping weights help at every stage.
See the paper diagram and technical detail
Weights can increase or decrease. “Training wheels” is the motivation; it is not a guarantee that every auxiliary weight decreases monotonically or reaches zero. Reward penalties are not hard safety constraints.
03 / PACE DOMAIN RANDOMIZATION
LUCID
Read execution. Pace the challenge.
Delay, uncertain dynamics, and noisy observations prepare policies for transfer—but severe perturbations can disrupt learning too early.
LUCID compares histories of issued joint targets and measured positions in a frozen, learned motion space. The execution gap helps decide the next training block's randomization intensity.
Bounded PI update + return backoffSet the next block's shared DR intensity
AT DEPLOYMENT Trained policy → low-level controller → G1
Compare 25-sample command and execution histories from the same episode. The waveforms here are schematic.
What “understanding” means here
A denoising-pretrained temporal VAE supplies a frozen, 32-dimensional comparison space. “Understanding” refers to learned temporal motion structure—not an estimate of the student's intentions or a diagnosis of the physical cause.
A 90th-percentile block discrepancy drives a bounded proportional–integral scheduler. Low task return supplies an independent override. This teacher is an encoder plus feedback controller, not a learned RL teacher policy. Neither encoder nor scheduler runs onboard.
Move the execution gap above or below its nominal reference. See how proportional feedback, accumulated error, and a low-return override change the next intensity.
An interactive calculation of the preprint's scheduler equations. Input values are illustrative; this is not a trained robot or a replay of experimental data.
Below referenceAt referenceAbove reference
RANDOMIZATION INTENSITY0.500
Nominal conditionsFull configured ranges
Start at intensity 0.5 with warm-up already complete. Advance to apply one update.
Equation and assumptions
e = 1 − gap/reference I = clip(I + e, −0.8, 0.8) λ ← clip(λ + 0.04 × clip(0.8e + 0.15I, −1, 1), 0, 1)
After warm-up, two consecutive low-return blocks apply λ ← 0.70λ and reset the integral, overriding the PI update. This demo starts after warm-up with valid histories; Algorithm 1 also specifies warm-up and empty-window handling. A shared intensity couples channels; it does not set each channel's difficulty independently.
THE EVIDENCE / SCOPED COMPARISONS
From training decisions to robot behavior.
Wheeled navigation. Quadruped locomotion. Physical off-road driving. Humanoid motion tracking. Each result answers a specific experimental question.
Published · simulation
Wheeled navigation success
GACL and CLUTR evaluated in the reported BARN navigation simulation benchmark.
Table II reports mean ± standard deviation: 81.85 ± 2.51% versus 76.67 ± 2.74%.
This is a simulation result; it does not establish GACL hardware deployment.
These are paper-reported results, not a new reproduction. Different experiments use different tasks and protocols; values should not be compared across experiments. LUCID results are from a 2026 preprint.
RESEARCH MEETS ENGINEERING
Work across the robot-learning stack.
I connect adaptive-training research with simulation, policy interfaces, and evaluation. My mechanical-engineering background and AWS systems work complement that research.
C++ / ROS 2 / ONNX inference, observation construction, normalization, temporal history, and runtime diagnostics.
04
Physical evaluation and systems
Research with off-road and G1 physical evaluations; C database-engine work and Python performance tooling at AWS.
The studies span wheeled navigation, quadruped locomotion, off-road driving, and humanoid motion tracking. My separate G1 residual/base inference project currently focuses on simulation integration and policy-interface validation.
