Key findings
Same configs, two engines — the real bridge is the ground truth
Real fleet ceiling (warehouse_xl)
Bridge missions at 12–20 robots
central_aisle vs offline estimate
Best RL gain over FIFO
Fleet scaling
warehouse_xl · 4 chargers · FIFO — does adding robots add throughput?
The offline engine peaks at 12 robots (24 missions). The real bridge peaks at 4 and completes zero missions from 12 robots up — every 12–20-robot run ended in a detected fleet freeze. Safe real-world ceiling on this floor: 8 robots.
Layout comparison
6 robots · 2 chargers · FIFO — the floor plan is the only variable
Both engines rank standard first. They disagree hardest on central_aisle: usable offline (13 missions), near-total freeze on the real bridge (1 mission, 15 of 30 observation windows frozen) — the single centre aisle becomes a one-lane choke point once charging traffic joins mission traffic.
Charger scaling
warehouse_xl · 20 robots · FIFO
| Chargers | Offline missions | Offline block (s) | Bridge missions | Bridge block (s) |
|---|---|---|---|---|
| 1 | 14 | 5170 | 0 | 15966 |
| 2 | 14 | 5306 | 0 | 15674 |
| 3 | 17 | 4979 | 0 | 16003 |
| 4 | 14 | 6011 | 0 | 16028 |
1 → 4 chargers changes nothing in either engine: block time stays flat and the bridge fleet stays frozen. At this fleet size the constraint is floor space and junctions, not charging capacity.
FIFO vs RL
Missions per 15 min, matched configs
| Config | FIFO | RL | RL vs FIFO |
|---|---|---|---|
| warehouse_xl · 6r · 4c (bridge) | 10 | 11 | +10% |
| warehouse_xl · 20r · 4c (bridge) | 0 | 0 | both freeze |
| central_aisle · 6r · 2c (offline) | 13 | 16 | +23% |
| standard · 6r · 2c (offline) | 18 | 18 | no change |
| zoned_io · 6r · 2c (offline) | 8 | 8 | no change |
| warehouse_xl · 6r · 2c (offline) | 18 | 18 | no change |
RL earns +10–23% only where contention exists (central_aisle, dense fleets short of collapse). In sparse configs it matches FIFO exactly, and at 20 robots both policies freeze — priority ordering cannot break structural deadlocks.
Method & caveats
How these numbers were produced, and where they can mislead
Two engines, same configs
- Offline batch engine (scripts/scenario_suite.py): deterministic, no battery model. Its rows feed the Recent benchmarks table and /compare.
- Real ROS bridge, headless (warehouse_rviz_bridge.py under a fake clock): full traffic manager, reservations, safety stops, station yields, dock gates and battery — the code production runs. 900 s of simulation per scenario, no RViz.
Caveats
- Offline numbers are optimistic upper bounds: that engine has no battery model and less physical friction (no safety stops, station yields, or dock gates).
- charger_wait_time reads 0 even during charger crises — it only counts waiting AT a station, not being blocked on the way to one. The live production fleet shows exactly this blind spot.
- Production DB still enforces the old limits (≤10 robots, ≤2 chargers), so the 12–20-robot rows live only in data/benchmark_runs.jsonl until migration 014 is applied.