Harishankar Somasundaram · Writing · 22 August 2026

Optimising Hospital Flow with Discrete-Event Simulation and Multi-Agent RL

A discrete-event simulation advances a clock from one state change to the next rather than in fixed ticks. This one models an Irish acute hospital — 14 departments and 278 beds — as a binary-heap queue of future events, and a MADDPG multi-agent reinforcement-learning policy sets staffing inside it every simulated hour, floored by the HSE Safe Staffing Framework ratios.

Key takeaways

What the article covers

Frequently asked questions

What is discrete-event simulation and why use it for a hospital?

A discrete-event simulation advances a clock from one state change to the next rather than in fixed ticks. Nothing happens between events, so simulating a week of hospital activity costs only as much as the events in it. The hospital model keeps a binary min-heap of future events — arrivals, service completions, transfers, discharges, staffing changes — pops the earliest, applies it, and schedules whatever follows. That maps naturally onto a hospital, where the interesting quantities are queues and waiting: a patient occupies a bed for a sampled service time, then either transfers to another department or leaves, and if no bed is free they queue. Fixed-timestep simulation would either miss short events or waste enormous effort on empty intervals.

Where does the simulation get its patients and parameters?

Service times are log-normal per department, with parameters fitted in log space — ED is (1.2, 0.7), giving a median stay near 3.3 hours, while ICU is (3.8, 1.0), a median near 45 hours. Movement between departments follows a transition matrix derived from MIMIC-IV transfer sequences: 20% of ED patients are admitted directly to Medicine, 8% to ICU, 20% leave through the discharge lounge. Bed capacities are the Irish HSE department layout, 278 beds across 14 departments. In production the engine's own Poisson arrival stream is disabled and admissions arrive from the data-ingestion service, so the DES mirrors the platform's real patient flow rather than inventing one.

How is the simulation connected to the rest of the hospital platform?

Through three inbound channels. Bed management posts the ward census every few seconds, which advances the engine's clock and updates each department's occupancy. The digital twin forwards MIMIC-IV admissions as they are replayed, injecting real patients into the DES. The deterioration service pushes NEWS2 clinical escalations, and the discharge-lounge and ED-flow services contribute their own signals. Every one of those crossings is written to an action log that the dashboard renders, so the integration is visible rather than assumed.

How does multi-agent reinforcement learning set staffing?

Each of the 14 departments is an agent. Every agent observes 12 local features — patient count, capacity ratio, average wait, average length of stay, one-hour and four-hour admission rates, staffing ratio, transfers pending in and out, mean acuity, and time of day as a sine and cosine pair — and outputs 4 continuous values: a change in doctors bounded to plus or minus 3, a change in nurses bounded to plus or minus 5, a transfer priority and a discharge threshold. Training uses MADDPG: each agent has its own actor, but the critics see every agent's observations and actions, so an agent learns that staffing its own ward affects its neighbours. At execution time only the actors run, on local observations, so inference is a small forward pass per department.

What does the reward function actually reward?

Five bounded terms per department, summed and clipped. A wait penalty, always active and capped so one 20-hour outlier cannot dominate the gradient. A queue-depth penalty, the most direct early-warning signal. An occupancy penalty that begins biting at 80% rather than 100%, so the agent adds capacity before a ward tips over. A throughput bonus, capped so it cannot be used to rationalise staff cuts. And an optional per-head staffing cost applied only above the establishment baseline. The bounds matter more than the terms: an unbounded component will saturate or dominate, and the policy will optimise for it.

How much does the learned policy actually improve on static staffing?

Over 8 independent runs of the same 168-hour congested workload at 12 arrivals per hour, the adaptive policy averaged 21.9% lower mean queue wait — 87 to 120 minutes against a static baseline of 133.1 — and 11.6% higher throughput. The static arm returned exactly 133.1 minutes in all 8 runs, so the control is reproducible and the comparison is like for like. An earlier figure of 11.3% was measured on a wait metric that blended queue wait with in-department dwell; separating the two roughly doubled the relative effect without changing the model. The honest reading is about a fifth off queue wait on a congested week, quoted with its spread rather than as a single headline number.

Would this reduce trolley waits in an Irish hospital?

Not on this evidence. The Irish problem is real and large — the INMO counted 114,029 patients admitted without a bed in 2025 — and a low-double-digit reduction in mean wait would matter if it transferred. But it has been demonstrated on a simulation calibrated to MIMIC-IV, an American ICU dataset, with an Irish department layout imposed on top. The simulation is a model of a hospital, not a hospital, and a policy that improves the model improves the model. Establishing real-world effect would need local calibration against a specific hospital's own flow data, a prospective comparison, and clinical governance that this project does not have.

What stops the agent proposing unsafe staffing?

A hard floor derived from the Irish Department of Health's 2018 Framework for Safe Nurse Staffing, enforced after the policy acts rather than trusted to it: 1:1 in ICU, 1:2 in HDU, and ward-specific ratios elsewhere. Any nurse count below the floor for the current census is raised to it and the override is recorded. Separately, staffing recommendations can be routed to an EU AI Act Article 14 human-oversight queue where a clinician confirms or rejects each one, and production deployment mode locks that oversight on.

Is this running on real patient data or in a real hospital?

No to both. The platform runs on MIMIC-IV derived records and a discrete-event simulation, on a homelab machine. It is a demonstration of architecture and method rather than a clinical system, nothing it produces is clinical advice, and no part of it has been used in the care of a patient.