Deployment and trainer each hold model weights. Requests and responses produce trajectories. Trajectories receive rewards and are assembled into a dataset. The dataset moves to the trainer, where three minibatch updates change its weights. The reward curve updates when the trainer receives a dataset, using the deployed checkpoint’s completed optimizer steps. Each observed reward is 0.1 plus 0.9 times two times arctangent of optimizer steps divided by five, divided by pi. Dots mark the received datasets; straight segments connect adjacent dots. Its vertical axis stays between zero and one; its horizontal axis starts at zero to ten and expands as training proceeds. The updated checkpoint is quantized, shown in grayscale, and synchronized back to the deployment. The cycle repeats automatically.

Deployment and trainer exchange datasets and updated model weights Data flows from the deployment to the trainer. Updated weights flow back through quantization. Both systems retain their own weights throughout the loop. Users Requests Responses Deployment Model weights Quantized Checkpoint 1 Serving requests Trainer Model weights Trainable Checkpoint 1 0 0.5 1 reward optimizer steps Trace 1 Trace 2 Trace 3 WEIGHTS Quantize weights