Gradient Gators.

A research question needs a repeatable experiment

Gradient Gators investigates how reward design influences reinforcement-learning behavior and robustness when an environment changes. Before a comparison can say anything useful about rewards, the surrounding software has to make the experiments comparable: scenarios must be identifiable, the environment must behave consistently, and each run must leave enough evidence to understand what happened.

My work centers on that infrastructure. I lead training and integration within our team, with implementation contributions spanning survivability checks, frozen scenarios, deterministic collections, environment integration, a PPO smoke test and the pilot runner. Teammates contribute the environment, generation, reward functions, observations, evaluation and visualization. The project is a shared research system; the work described here is my part of it. Witness implementation, scenario snapshots, pilot runner.

Checking whether a path can actually be survived

A connected map is not necessarily a survivable map. An agent may have a geometric route to the goal but accumulate too much damage along the way. That distinction matters when generated scenarios become inputs to a learning experiment: an impossible scenario can distort the interpretation of an unsuccessful policy.

I implemented a witness approach that constructs candidate action sequences and replays them using the environment's mechanics. It considers shortest-path and damage-aware candidates, with bounded waiting variants to account for timing. A successful replay provides a concrete trajectory that reaches the goal alive. Survivability PR and implementation.

The boundary is deliberate: a successful witness demonstrates feasibility; an unsuccessful bounded search does not prove impossibility. The search also does not establish an optimal policy. That narrower claim makes the artifact useful without confusing a practical scenario check with a complete planner.

Freezing inputs before training

Generating a world and describing its settings is not enough to guarantee that another run will use the same input. I added frozen scenario snapshots, stable identities and checksums, then deterministic collections and schedules that the training path can consume. These make scenario identity explicit across generation, replay and training. Snapshot PR, collection and schedule PR, environment integration PR.

This separates two concerns: generating candidate environments and selecting a known sequence of inputs for an experiment. It also provides a useful debugging boundary. When a run behaves unexpectedly, the scenario artifact can be inspected alongside the configuration rather than inferred from an informal description of the generator.

Deterministic inputs do not remove every source of variation. Model randomness and episode lengths still matter, and equal training-step budgets do not automatically mean equal episode exposure. Reproducibility requires recording those conditions as well as freezing the maps.

Turning a training call into an inspectable run

The pilot runner checks input collections, records configuration and revision information, tracks interactions, saves checkpoints and logs, and performs a held-out integration episode. The purpose is to leave a traceable run instead of only a final model file. Runner implementation.

Before longer experiments, I added a short PPO smoke path covering training, saving, reloading and held-out use. Its 576 training steps exercise the integration; they are not evidence that a policy learned to solve the task. That distinction lets a quick check answer a useful engineering question without being mistaken for a research result. PPO smoke-test PR.

Evidence and the next research boundary

The November 4, 2026 CI run at revision 6b8e1ce recorded 457 passed, 2 skipped and 17 warnings. This is dated execution evidence for the software checks, not a coverage percentage or a measure of agent performance. Exact CI run.

As of November 5, all five reward calculators are implemented, while the runner at that revision still accepts sparse-reward pilots only. Its held-out check uses one scenario; it does not implement the complete comparative evaluation. The existing dashboard is also described as using mock data. Reward implementation PR, runner restrictions, dashboard status.

The next milestone is to connect the remaining experimental modes and retain complete evaluation artifacts. Until then, the demonstrated contribution is the infrastructure that makes a controlled study possible. Dated research updates will distinguish implementation milestones, verified runs and eventual comparative findings.