Staleness log-IS ratio |log πθ − log πθk| 0.000 0.005 0.010 0.015 0.020 0.025 0.030 0 48 96 144 192 240 Optimizer steps Nats