Plug WareMax into a Gymnasium RL loop

WaremaxAllocEnv exposes the task-allocation decision as a semi-Markov decision process (SMDP): the agent is only asked to act when a robot becomes free and a task is waiting.

The observation and action spaces

The observation is a Dict:

{
  "robots": (64, 8),       # per-candidate robot features
  "task": (6,),            # the task awaiting assignment
  "action_mask": (64,),    # which candidates are legal this step
}

The action is an index into the masked candidate robots. Because the candidate set is variable-sized, WareMax pairs the env with a permutation-equivariant candidate-scoring policy — the right inductive bias for this action space.

A minimal loop

from waremax_gym import WaremaxAllocEnv

env = WaremaxAllocEnv(scenario="scenario.yaml", reward_mode="routed", seed=12345)
obs, info = env.reset(seed=12345)
for _ in range(steps):
    action = policy(obs)   # index into masked candidates
    obs, reward, term, trunc, info = env.step(action)

The four reward modes

• sparse — reward at terminal events only (baseline)
• dense — per-step shaping with all delay buckets (baseline)
• attribution — per-task causal delay decomposition as reward
• routed — per-decision controllable cost: assignment wait plus travel to pickup

An honest finding

On the built-in presets, learned dispatching matches nearest-robot and round-robin but does not surpass them — the system is capacity- and contention-bound, so state-blind round-robin is near-optimal. WareMax exists in part to make that finding reproducible and to help you find the regimes where dispatching choice actually has leverage.

Built by Skelf Research.