Learning to land.
A policy-gradient agent, a small spacecraft, and the same controls in your hands.
Keep the lander upright, shed speed, and bring it onto the pad. Try a flight before watching the agent. Getting close is easy; arriving slowly is the work.
Interactive flight test
Human versus PPO
Loading PPO checkpoints…
- Altitude
- —
- Vertical
- —
- Horizontal
- —
- Engine on
- —
- Touchdown
- —
- Score
- —
- Flight
- Ready
01 / THE TASK
Four actions. Several things to get right.
The controller can coast, fire the main engine, or turn with either side thruster. Main thrust acts along the lander's body: correcting sideways motion requires managing its angle as well as its speed. A late correction can fix one problem and create another.
The agent receives eight numbers: horizontal and vertical position relative to the pad, two velocities, angle, angular velocity, and two foot-contact indicators. It sees those measurements directly, not the rendered image. The human pilot has the picture and instrument readouts. This is a playable comparison, not an experiment claiming equivalent human and machine information processing.
This is a browser-native adaptation of the classic Lunar Lander task, using Planck rigid-body physics. It is not Gymnasium's benchmark, and its scores should not be compared with Gymnasium's published thresholds. The training worker and this game import the same environment implementation.
02 / LEARNING THE POLICY
From a state to an engine command.
A small neural network maps the state to four action probabilities. During training, the controller samples an action, observes what happens, and collects the reward. A separate value network estimates how much future reward to expect from the current state.
The update uses proximal policy optimization (PPO). An advantage estimate says whether an action's outcome was better or worse than the value estimate expected. PPO changes the action probabilities while clipping the surrogate objective to limit the incentive for large probability-ratio changes on the sampled data.
Here is a probability ratio, not the environment reward. Clipping this objective is not a guarantee that the policy is optimal, or that every update improves it.
Training explores by sampling actions. In the game, the agent chooses its most probable action. No learning takes place in your browser: it runs the exported network against the same simulator used for training.
03 / WHAT THE SCORE MEASURES
A landing is more than a score.
The reward combines progress toward the pad, slower motion, and a more upright attitude with costs for engine use and elapsed time. Landing and crashing produce separate terminal outcomes. Reward shaping gives the learner useful feedback before it manages a complete landing.
The flight panel reports score, touchdown speed, and engine use separately. A policy can exploit a poorly chosen reward or become good at only a narrow range of starts. A single attractive replay does not establish reliability, and a high score does not by itself establish a safe touchdown.
For a head-to-head attempt, both pilots get the same seed, terrain, starting state, action limits, physics clock, and score function. The simulation pauses when the tab loses focus. Your previous flight stays in memory for comparison; it is not uploaded, and there is no public leaderboard.
04 / THE RUN
Check the controller, not just the highlight.
The final policy landed on all 64 held-out starts. The early checkpoint landed on none. By the middle checkpoint, landing was already reliable on this test panel; the remaining training made little difference to its mean score.
| Checkpoint | Training decisions | Landings | Mean reward |
|---|---|---|---|
| Early | 315,392 | 0/64 | -134.5 |
| Middle | 1,572,864 | 64/64 | 129.8 |
| Final | 3,145,728 | 64/64 | 130.2 |
64 successful flights do not imply a zero failure rate. These starts come from the same bounded distribution used in training, not unfamiliar gravity, damaged engines, or an adversarial test.
Training details
One training seed (20261009), 3,145,728 simulator decisions, 32 parallel environments. Separate policy and value networks each have two 64-unit tanh hidden layers. Training uses PyTorch on CPU; the simulator runs in Node. This run took 127 seconds, excluding evaluation.
PPO collects 128 decisions per environment before an update, uses minibatches of 1,024 for up to four epochs, and linearly anneals the learning rate from 0.0003 toward zero. Discount 0.99; GAE parameter 0.95; clipping 0.2; entropy coefficient 0.01. The downloadable record includes the remaining settings, software versions, and source hashes.
Episodes stop after at most 400 decisions (40 simulated seconds). Training bootstraps the value at that time limit, but not at a crash or landing: it treats the limit as an interruption of a discounted continuing objective. Evaluation reports the undiscounted reward actually collected before the episode ends. No demonstration data or heuristic pretraining is used.
Checkpoint comparisons describe one training run. They are not a multi-seed comparison of RL algorithms. Evaluation starts are held separate from training, and the final checkpoint is the end of the declared training budget rather than the winner of a search over the evaluation set.
What this project is.
An implementation of an established method, with a game you can inspect by playing. The contribution is the working connection between the learning code, the policy's behavior, and the task—not a new RL algorithm.
It also gives us a concrete starting point for offline evaluation later: record the flights, hide the simulator from an evaluator, and ask which claims about another pilot those records can support. That is a separate question from the online training shown here.