apt-a1 · overnight run · 2026-08-07

Reproducing APT-RL's motion-library half on the Unitree A1

The paper's §1a claim is that a pre-generated library of optimised trajectories, compressed into a latent action space, beats learning locomotion from scratch. This is that pipeline built end to end for our 14.5 kg A1 — generator, library, TVAE, and an RL policy whose action space is the library — plus the ablation that tests the claim.

The pipeline, and what each stage cost

SRBD generatorBézier GRFs, Newton shooting on a periodic cycle, as Cheetah 2 does it~100 µs / trajectory

A1 feasibility filtertorque, friction, reach, joint velocity, against measured limits~50% kept

TVAEshared encoder, gait-specific torque decoders R² 0.9943

RL policygait at 2 Hz + 16-dim latent at 100 Hz + aux 36k env-steps/s

The result: does a motion-library prior help?

Two PPO runs at an identical 78.6M-step budget on identical rewards and environment. One emits a 16-dim latent into the frozen library decoder; the other has that channel removed and must drive the joints through the auxiliary output alone — plain RL, no prior.

With library priormean speed-tracking error0.202 m/s

No prior (ablation)same budget, same reward0.309 m/s

The prior wins by 35% on average — but the average hides the whole story.

commanded (m/s)	library achieved	error	no-prior achieved	error
0.00	-0.00	-0.00	-0.06	-0.06
0.25	-0.24	-0.49	+0.01	-0.24
0.50	-0.16	-0.66	+0.41	-0.09
0.75	+0.63	-0.12	+0.89	+0.14
1.00	+0.78	-0.22	+1.23	+0.23
1.25	+1.26	+0.01	+1.17	-0.08
1.50	+1.35	-0.15	+1.38	-0.12
1.75	+1.61	-0.14	+0.78	-0.97
2.00	+2.03	+0.03	+1.16	-0.84

The advantage is entirely at speed. At 1.75 and 2.00 m/s the no-prior policy misses by 0.97 and 0.84 m/s; the library policy misses by 0.14 and 0.03. Below 0.75 m/s the ranking inverts — the prior is actively worse, and at 0.5 m/s it drives the robot backwards and falls.

Two explanations for that, both wrong

Wrong #1: "the library has no standing gait." It does — 3,008 trot and 2,883 bound entries sit below 0.1 m/s, about 9% of each — and the policy stands correctly on command, achieving −0.00 m/s at cmd 0.00 and surviving. The failure is confined to the 0.25–0.75 m/s band, not to standing.

Wrong #2: "40% of the library is reverse speed we never command." That part is true — the library samples v uniformly over [−1, +2] — but it is not the cause. Regenerating forward-only, retraining the TVAE (R² 0.9944, unchanged) and the policy at equal budget made things 3.7× worse, in every speed band including the one it was meant to fix.

library speed range	mean error	low ≤0.75	high ≥1.5
v ∈ [−1, 2] (full)	0.202	0.319	0.106
v ∈ [0, 2] (forward only)	0.743	0.438	1.097
no prior	0.309	0.134	0.643

The reverse trajectories were helping — most plausibly by giving the latent manifold broader coverage, so the decoder is better conditioned in the forward region too. Narrowing a library to what you think you need is the wrong instinct. The low-speed inversion is still unexplained. Next suspect is the reward rather than the library: the velocity kernel exp(−1.5 e²) makes a 0.25 m/s error cost 0.09 out of 2.5, so there may simply be no incentive to be precise down there — which would also explain why the no-prior policy is only mediocre there rather than good.

Watch it

library prior — 2.0 m/scommanded 2.00, achieved +1.98 m/s, survived the full episode

no prior — 2.0 m/scommanded 2.00, achieved +0.78 m/s — less than half

library prior — 0.5 m/scommanded 0.50, achieved −0.31 m/s and fell at 4.3 s

no prior — 0.5 m/scommanded 0.50, achieved +0.44 m/s, stable

Four ways a healthy training curve lied

Each produced a rising return and a rising episode length while the robot did something useless. None was visible without evaluating per commanded speed.

1. The crouch trap. Alive + height + upright paid ~1.0/step for standing still; a 1 m/s tracking error paid 0.03. After 3M steps the policy stood at z = 0.16 and achieved 0.00 m/s at every commanded speed, while return climbed 35 → 479.

2. Unbounded exploration. The entropy gradient on log_std is constant at ent_coef × act_dim, while actions clip at ±3 — past the clip a wider policy is nearly free. std grew 0.61 → 3.5, training went bang-bang, and the deterministic mean action sat far off the optimised distribution. That run reported return 991 / episode 568 of 600 and walked backwards when commanded forward.

3. Command range mismatch. Trained on U(0, 1.5), evaluated to 2.0 — a test-design failure reading as a policy failure.

4. The forward-only library. Best return of all three runs (1348) and the longest episodes (592.9/600) — while tracking 3.7× worse than the run scoring 1283.

Return conflates tracking with survival, and here survival is nearly saturated. The first two production runs finish at 1283 and 1273 — a 0.8% difference across a 35% difference in tracking.

What this does not show

Flat ground, sim only. No terrain, no perception, no sim2real. Contact parameters were never calibrated against the real robot.
One seed each. The 35% gap is a single comparison, not a distribution.
The low-speed inversion is unexplained, with two candidate causes now ruled out.
The A1's legs are 53.4% of its mass, and SRBD treats them as massless — a far worse approximation than for the 31 kg MIT Cheetah 2 this method comes from.