The most useful correction in this project was to stop using execution as a substitute for learning. A trainer can make real updates while optimizing behavior that never solves the environment.

Wordle on 360M

The original model, a 50-update SFT checkpoint and SFT followed by two RL updates were evaluated on the same 20 words with four seeded attempts per word. Here, pass@1 is the mean single-attempt win estimate over the four samples. Pass@4 is the fraction of evaluation words solved at least once in those four attempts. Formatting a legal-looking guess is not winning the game. One evaluation answer appeared in the initial RL rollouts. The report also checks the matched 19-word unseen-answer subset, which remains zero. The overlap must stay visible rather than disappear in a summary. The longer run stopped at update 131. Its 1,763 recorded episodes had no wins. After update 100, all 561 recorded episodes ended in invalid moves, and 275 of 280 completed pairs tied in reward. Earlier partial credit could survive a later invalid move under the original Hub rewards. Those rewards were preserved while the numerical admission bug was fixed.

Qwen3-0.6B

A separate 100-update LoRA SFT run lowered recorded loss from 2.9620 to 0.4319 and increased accepted opening moves from 60/80 to 80/80. All 100 updates had nonzero gradients. Exact pass@1 and pass@4 still remained zero. Reasoning-budget pilots went as high as 4,096 tokens per turn. These were small diagnostic samples, not a large benchmark. More room to think was not enough to establish successful multi-turn gameplay.

Reverse Text

The next environment had a similarity reward that could distinguish partially correct answers. The pilot selected the shortest 256 upstream prompts, trained on 224 and held out the shortest 32. Training and held-out prompt sets were disjoint, but held-out prompts were shorter. Both evaluations sampled the same four seeds per prompt. The similarity delta is +0.01091. A paired bootstrap over the 32 tasks, retaining their four samples together, gave a 95% interval of [-0.00234, +0.02477]. It includes zero. There was only one training run. All ten updates had finite losses and nonzero gradient norms, and the final adapter was served for evaluation. This verifies integration. It does not establish reliable learning or exact task success.

What remains to test

A shorter-string curriculum or an upstream SFT warmup may provide a better starting point. GSM8K is exposed through the reviewed taskset, but this record contains no successful GSM8K learning result. Harbor integration was discussed as another direction; the implemented menu is not a general Harbor benchmark launcher. Wordle report · Qwen SFT results · Reverse Text result Continue with the installed command.