The Traversal Critic Project

Do Video-Language Models Make Good Traversal Judges?

Privileged-to-visual reward distillation for humanoid navigation — a controlled study that runs the three controls VLM-reward papers usually skip: the privileged teacher, a frozen-feature probe, and a preregistered policy test.

⏳ Status (Aug 2026): 3-seed × 3-arm contact-physics PPO matrix running — the policy verdict is preregistered and pending. No policy-benefit claim is made yet.
📄 Current draft 📦 Stage-A release (pre-closure preregistrations) 🗄️ PDF (Aug 7, pre-v5) 📈 Findings so far 🧪 Progress
In 30 seconds: Everyone is starting to use video-language models as reward judges for robots — and almost everyone validates them the same way: correlation with labels on held-out clips. We ask the harder questions. If a privileged simulator rubric made your labels, how much does the judge lose relative to that teacher? Does the fine-tuned 2B judge even beat a ridge regression on its own frozen vision features? And does any of it move a real policy? We fine-tune Cosmos3-Edge (2B) into a 1–5 traversal critic for humanoid navigation clips, then run all three controls under MuJoCo contact physics with a frozen GEAR-SONIC whole-body controller on a Unitree G1. The answers so far are not what the headline-correlation methodology would report — and that is the point.

The three controls

Each one changes what the evidence means. Together they decide whether the judge — rather than its labels or its backbone — deserves the credit.

1 · The privileged teacher, as a reward arm

The deterministic rubric that generated the training labels also runs as its own reward-shaping arm ("oracle"). Whatever the visual judge achieves is measured against direct access to its teacher — the proxy-vs-gold comparison of reward-overoptimization studies, run in the loop.

2 · A probe of the judge's own frozen features

A media-grouped ridge probe on the critic's own frozen SigLIP2 vision tower asks: did autoregressive fine-tuning add anything beyond the pretrained representation? (Spoiler: in-domain, it didn't.)

3 · A preregistered policy matrix

3 seeds × 3 reward arms (hand-crafted / privileged / critic) × 300,032 PPO steps under contact physics, with frozen selection rules, ≥90% audited reward-coverage requirements, and decision gates written down before any endpoint exists.

The approach: judge, not simulator, not policy

Video foundation models are used as simulators (imagine futures) or policies (VLAs) — both fight the models' weakness and put billions of parameters near the control loop. We use the third option: judging a completed clip is one forward pass, asynchronous, and training-time only. Nothing large ever deploys: the only learned component on the robot side is a 0.43M-parameter navigation policy at ~3.3 Hz commanding the frozen 50 Hz whole-body controller.

System architecture: training loop with critic vs deployment stack
Three layers. A frozen GEAR-SONIC whole-body controller (50 Hz) handles walking, crouching, crawling. A small vision policy (~0.43M params, 3.3 Hz) picks heading, speed, and posture mode. The 2B critic exists only on the training side, scoring completed episode clips through an audited asynchronous queue.
1 — The problem is the reward.

Traversal quality in occupied clutter — squeeze the gap with margin, duck the table, don't crowd the person — is perceptual and hard to write down. Community evaluations of social navigation reach the same conclusion: quality metrics are unsolved.

2 — Supervision is free in sim.

Privileged simulator state (clearance, contacts, person distance, progress) converts to four-axis ordinal labels deterministically. 2,000 audited contact-physics rollouts, zero human annotations, scene-disjoint 1,568/432 split.

3 — The judge matters only where the state is absent.

Inside the simulator the privileged score is strictly better. A learned visual judge earns its keep only if the rubric survives transfer to pixels — RGB logs, real-robot video, other simulators. That transfer is exactly what the controls measure.

What the critic learns to judge

Procedurally generated cluttered indoor scenes — narrow doorway gaps (0.6–1.2 m), low tables, furniture, moving people — traversed by scripted drivers spanning five quality bands. The auto-labeler scores each episode on safety, clearance, social distance, and progress, with safety as a hard ceiling.

Clean traversal rollout
score 5 — clean clip

Collision-free, brisk approach

Routes through gap centers with margin. The score rates the clip's quality under the rubric.

Skimming traversal rollout
score 3 — sloppy

Grazing the gap edge

Gets through, but skims the wall at speed and hesitates — no crash, no craft.

Unsafe traversal rollout
score 1 — unsafe

Walks through a person

Safety is a hard ceiling: any person contact or fall caps the overall score at 1 regardless of other axes.

One episode per ground truth score
One episode per ground-truth score, frames left to right in time (development-generation camera shown; the clean v5 reproduction uses a third-person chase view).

Real contact dynamics, real consequences

The full released SONIC stack — planner → motion-token encoder → decoder → PD torques — runs under MuJoCo contact physics in our cluttered scenes. Clutter pushes back, falls are real, and the mode decision (walk vs. crawl) has physical consequences. These clips are scripted routes, not the learned policy: they demonstrate the environment and control substrate that the reward-arm comparison runs on.

Physics walk through narrow gap
physics — scripted route

Narrow-gap walk under contact dynamics

Held-out scene, 0.8 m doorway. The controller absorbs a doorframe brush instead of the pose being teleported past it.

Physics crawl under low table
physics — scripted route

Hand-crawl under a 1.19 m table

Walks two gaps, drops to hand-crawl, passes under the tabletop, reaches the goal.

Physics yield to crossing person then crawl
physics — scripted route

Yield to the person, then crawl

A naive stop-when-close rule deadlocks or stops in the person's lane in most held-out scenes — when and where to yield is perceptual. That decision is what the trained policy must learn.

Findings so far (clean v5 reproduction, evidence-gated)

Everything below comes from the audited v5 protocol: 2,000 labeler-v3 contact-physics rollouts, scene-disjoint validation, frozen selection rules, machine-checked provenance. Earlier development results (v1–v4, kinematic pilots, single-seed physics pilots) are preserved as history and support no claims.

Fine-tuned 2B critic (autoregressive digit)

r = 0.565
  • Only checkpoint with valid outputs on all 432 scene-disjoint validation clips (iteration 100 of 800).
  • Later checkpoints rank better — up to r = 0.705 — but each emits 1–2 invalid generations.
  • OOD: all 42 outputs stay in [1,5], but impaired gait (n=1) scores above the clean-walk mean — the ordering test fails.
  • Interface audit (Aug 11): training, validation, and policy scoring used different temporal interfaces (4 / 4–48 / 5–32 frames) — this number is historical, not a matched-interface estimate; a matched re-evaluation is preregistered.

Ridge probe on the same frozen vision tower

r = 0.705
  • Media-grouped weighted ridge on frozen SigLIP2 features — no autoregressive tuning at all.
  • OOD: restores the impaired-below-clean ordering the critic loses…
  • …but 8/42 unclipped predictions fall outside the 1–5 range. Each readout fails a different criterion.

The diagnosis (evolving): the traversal signal survives the frozen vision tower; the fine-tuned path loses quality to a combination of the generative digit interface and — as a release-blocking preprocessing audit found on Aug 11 — a mismatched temporal input path (the fine-tuned model effectively saw four frames per clip). A preregistered 17-row shortcut battery (executed Aug 12) has now bounded the label side: rubric labels concentrate at the endpoint (terminal frame alone r = 0.666; duration + terminal r = 0.683) while the first half of each clip recovers only r = 0.44, and masking the final 10% of frames drops the best masked readout to 0.574 (shuffled-label control 0.01). The probe's 0.705 therefore exceeds a low-resolution endpoint readout by only ~0.04 — so the remaining question, under the preregistered temporal-route × decoding factorial and the corrected-generation critic, is whether any model readout uses more than the endpoint. Two more preregistered cells executed Aug 12: zero-training SigLIP2 text-anchor similarity carries no ranking signal at all (r = −0.14), and a tensor-level audit found the critic's vision tower drifted from the public SigLIP2 release (437/437 tensors) — the traversal signal is reachable by a trained linear readout of a privately continued-pretrained tower, not by off-the-shelf similarity. The controls keep catching exactly the things headline correlations hide; that is the point of the study.

2,000 / 500
Audited contact-physics rollouts / procedural scenes, labeled by the deterministic four-axis rubric; scene-hash 1,568/432 train/validation split; zero human annotations.
42 OOD clips
15 real Unitree G1 clips + 27 cross-simulator clips. No collision or fall examples exist in this corpus, so it cannot certify adverse-event judgment — we say so, rather than claim transfer.
9 × 300,032
PPO runs × steps in the preregistered matrix (3 seeds × hand-crafted / privileged / critic arms), with ~13,700 audited critic scorings at ≥99.9% coverage so far. Verdict pending.

What we do not claim (yet)

This project's credibility rests on saying exactly what the evidence supports. As of August 2026:
  • No policy-benefit claim. Whether critic shaping beats the hand-crafted baseline — and how it compares to its privileged teacher — is decided by the preregistered matrix now running, not by correlation.
  • No OOD calibration or transfer claim. The critic stays in range but misorders impaired gait; the probe orders correctly but leaves the range; the corpus has no negatives.
  • No real-robot deployment claim. All policy training is simulation with a third-person camera; real-robot video is used only to stress-test the judges.
  • No scaling or causal claim from development history. v1–v4 and the single-seed pilots used different data, cameras, and selection — they motivated the protocol and prove nothing.

Progress

Paper

Do Video-Language Models Make Good Traversal Judges? Privileged-to-Visual Reward Distillation for Humanoid Navigation

Video-language models (VLMs) are increasingly used as robot reward judges, yet they are typically validated by label correlation alone — without the privileged signal that supplied their labels, without a simple readout of their own frozen visual features, and without a policy-level test. We run all three controls in humanoid traversal. A deterministic simulator rubric labels 2,000 contact-physics rollouts, and a compact 2B VLM is fine-tuned to emit one ordinal score per clip. The controls change the story a headline correlation would tell: the only fully parseable checkpoint reaches Pearson r = 0.565 on scene-disjoint validation, later checkpoints rank better but emit invalid outputs, and a media-grouped ridge probe on the same frozen vision tower reaches r = 0.705 with no autoregressive tuning at all. Under visual domain shift, each readout fails a different criterion — the VLM is bounded but misorders impaired gait; the probe restores the ordering but leaves the ordinal range — and the corpus supports no adverse-event claim. The decisive test, a preregistered three-seed comparison of hand-crafted, privileged, and VLM-derived reward shaping under contact physics, is in progress; we make no policy-benefit claim.

Read the current draft   Historical PDF (Aug 7)

Status: working draft under an evidence-gate discipline — claims C1–C5 are materialized only after their gates close. Target venue: ICRA 2027 (RA-L/ICLR fallback). Research code will be released with the paper.