apt-rl overnight report · 2026-08-09

Ten standing policies, one physics test, and real trot steps by 3 AM

The from-scratch APT-RL stack would not walk. Tonight we found out why -- three times, at three different depths -- and ended with the decoder producing verified four-legged trot stepping from real library latents. Two arms are training on that result now.

Native control0.91 tracking @ iter 205 — install & wrapper exonerated
Our action space0.14 same everything, only actions swapped — 7× behind
Bank open-loop4/4 feet stepping at trot cadence, duties 0.43–0.55
Audit passes3 page-render equations, implementation layer, HTML/MathML
The failure your eye named: warm-start from random-phase sinusoids, checkpoint 100, AptClock-Native-A1-v0, difficulty 0.0–0.2. Three feet planted, one paddling, 18 N·m of prior torque rocking the body — "dry humping." Periodicity without phase structure.

The night in five findings

  1. The suicide economies were real but shallow. Six shakedowns died of reward arithmetic (dt-neutered death penalty, joint-limit exit, kneeling exit) — each fixed, each revealing the next. Fixes hold: no run since SD6 has collapsed.
  2. The prior could not lift a foot. Open-loop test: library-amplitude latents through the decoder at the paper's PD regime produced ZERO contact transitions. The τdec/kp ratio at A1 scale is 5–10× worse than HOUND's — the paper's bootstrap mechanism (Fig 5A: prior-driven motion from iteration 0) physically cannot operate on this robot without rebalancing. Authority sweep fixed it policy-free: kp 20 / τ-gain 3 walks at 0.51 m/s open loop.
  3. Standing is a fixed point. Even with authority, the mean policy froze its latent in time (temporal z-std 0.148 vs 0.561 across dims): static net + static obs = no oscillation. Countermeasures: 2 Hz phase-clock observation + warm-starting from the gait itself.
  4. The install was never the problem. Native IsaacLab A1 task through our own wrapper: tracking 0.91 by iteration 205. Same env with only our action space: 0.14 at 300. The action space is the bottleneck — quantified, not suspected.
  5. Phase relationships are the gait. Random-phase sinusoid latents → body-rocking (the video above). Real encoder-μ trajectories over library trot cycles → all four feet stepping at trot cadence, duties 0.43–0.55, transitions 5.8–6.5/s. The chain library → encoder → decoder → sim stepping is verified end to end for the first time.

Running overnight

Bank-warm arm (desktop): actor cloned from real-trot-cycle rollouts, then PPO on the native walking recipe + phase clock + widened latent clamp (the 2σ clamp was amputating genuine library latents at |z| up to 3.2). Clock-only arm (server): same config, no warm start — the control that separates the clock's contribution from the clone's. Reference bars: isolation 0.14, native ceiling 0.91.

Status at publish time: arms launched FINAL UPDATE 2026-08-09 morning -- THE ARCHITECTURE VERDICT. With the PD anchor aligned (a 0.35 rad default-pose error was the entire 'authority problem' -- the prior works open-loop at library-true torque once fixed: 0.358 m/s, duties 0.53-0.69), matched arms measured the prior's marginal value for RL: WITHOUT prior, tracking 0.267@200 and eplen 1067 -- the best cold learner this stack has produced; WITH prior, 0.005-0.008 and constant falls at BOTH noise levels. Every coupling mode now tested; the open-loop prior is a disturbance, not a scaffold, on an A1 -- likely because the 8-dim sagittal decoder gives zero hip feedforward while perturbing the lateral plane. Endgames on the table: ship the no-prior APT-shaped line (extending now), write up the negative result, and/or build the 3D library (12-dim torque) as the one untested rescue. Full ledger: docs/RUNLOG.md.; this page updates with their curves and the final verdict videos in the morning.

Paper-fidelity status

Three independent equation checks now agree (400 dpi page renders; implementation-layer audit with tensor values; arXiv HTML/MathML per your suggestion — supplementary truncated there, but control law, kp 80/kd 2, a_scale 0.2, latent-KL 2.5e-6 all confirmed). Audit-2 found two real wiring defects, both fixed: raw ±33.5 N·m torque observations dominating the MLP, and phase-locked episode resets in every run to date. The full ledger: docs/PAPER_AUDIT.md, docs/RUNLOG.md.