Shixin Luo et al. PIE: Parkour with Implicit-Explicit Learning Framework for Legged Robots. arXiv 2408.13740, 2024. Written from scratch.
PIE trains a parkour policy in one stage, with no teacher and no student. A single estimator reads depth and proprioception and predicts both explicit quantities and an implicit latent. I wrote it for the A1 in Isaac Lab on Sep 15, checked its equations with tests, and launched a run at the paper's full budget. Within the first 1,098 iterations the VAE latent had all but collapsed and the terrain curriculum never left the first row. The record ends there.
Most perceptive parkour policies train in two stages. A teacher with privileged terrain knowledge learns the skills first, and a student that sees only the camera learns to copy it. PIE skips the copy. One estimator network takes ten frames of proprioception and the two most recent depth captures and predicts four things. Three are explicit and supervised against simulator truth: body velocity, the clearance under each of the four feet, and a compressed code that decodes to a height map. The fourth is implicit, a VAE latent trained to predict the next proprioceptive frame. The actor acts on all four plus the current observation. A critic sees true velocity and a clean height map. Eq. 4 adds the five losses with unit weights.
There's no official code, so everything is mine: the environment, the estimator, and a PPO loop written directly in PyTorch rather than taken from rsl_rl. It comes to about a thousand lines of Python with its tests. The only things reused were the Isaac Lab install and the camera mount from my earlier A1 work: 0.33 m forward, 0.08 m up, pitched 20° down.
The paper gives the structure but few sizes, so the README lists every choice. That covers network widths, PPO settings, image resolution, the height-map grid and the terrain layout. Three points in the paper were ambiguous enough to change the math:
I also recorded two Isaac Lab conventions the paper can't have specified. Every reward term is multiplied by the 20 ms control step, and joint acceleration is differenced over the last 5 ms physics substep.
tests/test_equations.py pins these down numerically. It feeds uniquely numbered inputs through the observation to fix absolute joint angles and term order. It checks that power sums absolute per-joint power and that the action second difference is right. It also compares the KL against PyTorch's own and checks its gradients. Other tests cover recurrent resets, sequence replay and keeping actor and estimator gradients separate.
Two integration smokes, then 300 iterations on 1,024 robots, which took 673 seconds. A review against the installed Isaac source caught one convention: joint angles had been relative to the default pose, and Eq. 1 uses absolute angles. That was fixed before the 300-iteration run. Evaluated on the easiest terrain row, the checkpoint finished 0 tracks, and every one of its 64 robots fell.
4,096 robots for 10,000 iterations, from scratch, on the server, at about 6 seconds an iteration. That's roughly 17 hours.
A second pass over Eqs. 1–4 and every Table I term found no further algebra bug. This is when the equation tests went in.
A read-only look at the running job. That snapshot is the last record in the repo.
PPO itself was stable. The first update had one large KL spike; after that, recent KL stayed between 0.008 and 0.015, and nothing diverged. Robots stayed up longer, and by iteration 1,098 episodes averaged about 10 seconds. Two things had gone wrong, though.
From around iteration 200, the KL between the VAE latent and its prior sat near 1×10-4, summed over all 16 dimensions. That means the posterior matched the standard-normal prior almost exactly, so the implicit half of "implicit-explicit" was carrying close to nothing. That isn't evidence that the explicit heads failed, but the curves can't tell whether they helped either.
The mean terrain level peaked at one robot in 4,096 and was zero for the last 100 updates. No robot finished a track. The likely reason is a mismatch I built in myself. Promotion requires covering more than 8.5 m along the track, but each command pairs a forward speed with a random yaw rate of up to ±1.2 rad/s for 10 seconds. A robot can track those commands perfectly and still walk in a circle.
As a sanity check, I integrated 100,000 ideal planar trajectories under that command distribution, with no terrain and no falls. About 7.0% got more than 9 m down the track. With yaw set to zero, 80.5% did. That's geometry, not a measured success rate, but it means most robots could never have earned a promotion however well they walked.
Some regression showed up near the end of the window too. Comparing iterations 999–1,048 with 1,049–1,098, mean episode length fell from 13.15 to 11.35 seconds, and base-contact terminations rose from 0.415 to 0.519. One more unresolved question: the estimator takes 128 Adam steps per rollout against PPO's 20, and that cadence could shift the policy after PPO measures its KL. Nothing measured links it to the regression.
PIE's reward table did live on: MOVE names it as its reward, and the MOVE work tried it in the end. That story is on the MOVE page.