Independent RL Lab
A chronological record of research notes and calibration studies, including their questions, methods, limits, and reproducibility materials.
A trained policy, and the same controls in your hands
Fly a lunar lander, watch a policy-gradient agent, and compare their flights from the same starting conditions. An implementation of PPO with shared browser and training physics.
PPO · Browser-native physics · Interactive flight comparison
One promising checkpoint did not reproduce consistently across six training roots
One calibration checkpoint suggested persistent harm from sixteen missing rewards. Six predeclared DQN checkpoint forks did not reproduce a stable directional effect, so the study stopped.
Paired checkpoint forks · Tagged uniform replay · Predeclared stopping rule
When one adverse outcome becomes hard to unlearn
A controlled impulse-response study of how event timing, step-size rules, and TD backup depth determine whether one adverse outcome changes the final policy.
Exact calculation · Paired TD(0) simulation · Common random numbers
The policy boundary in a noisy maze
An exposed route is shorter. A protected route is slower. Exact references predict how three TD-control methods should differ; the full runs show where that ordering holds and where finite training does not recover it.
Q-learning · SARSA · Expected SARSA