The problem
Sepsis is life-threatening organ dysfunction caused by the body’s response to an infection. It contributes to up to half of all hospital deaths in the United States, and timing matters: prior work shows patients may have a 4% higher chance of survival if diagnosed one hour earlier.
To identify sepsis, clinicians work through clinical calculators, such as the Sequential Organ Failure Assessment (SOFA), which scores six organ systems from 12 clinical variables. Most AI sepsis models, by contrast, take in the patient’s record and return a single risk number. That gap between the two workflows makes AI predictions hard to trust and hard to act on.
Simply feeding calculators into a model is not straightforward either. A calculator needs all of its component variables, and in real hospital data many are missing: the missing rate in our three datasets ranged from 65% to 75%. The usual fix, filling in the gaps with an imputation model, becomes inaccurate when so much is missing, and its errors flow straight into the prediction.
The idea
What if the model followed the clinician’s workflow? Instead of a black box that jumps from raw data to a risk score, SepsisCalc first estimates the same calculators clinicians use, organ by organ, and then predicts sepsis from both the raw observations and those calculator scores.
To deal with missing data, we do not fill in every gap. We represent each patient’s record as a temporal graph, a network that contains only what was actually observed, and let the model add a calculator to that graph only when it is confident the estimate is accurate.
How SepsisCalc works
Each patient’s record becomes a graph whose nodes are time points, observed variables (vital signs, lab tests, procedures, medications), the six organ systems, and clinical calculators. Edges link repeated measurements over time, things observed at the same moment, and interacting clinical events, such as a vasopressor given because blood pressure dropped.
A graph neural network, a model that learns by passing information along the edges of a graph, summarizes the patient’s state at each time point. From that summary, SepsisCalc does three things:
- Estimates the calculators. When all component variables happen to be observed, the true score is available, and the model learns to reproduce it. This teaches it how each calculator is computed.
- Decides which estimates to keep. A confidence score filters out calculators that would likely be wrong, for example when all of their components are unobserved. Only confident estimates are added to the graph, along with edges to the variables they summarize.
- Predicts organ dysfunction and sepsis. The model reads the updated graph again and outputs both a sepsis risk and a risk for each of the six organ systems.
We evaluated SepsisCalc on two public ICU datasets, MIMIC-III (21,686 patients) and AmsterdamUMCdb (6,560 patients), and on data from 85,181 patients at the Ohio State University Wexner Medical Center (OSUWMC). Starting when a patient arrives, the model predicts every hour whether sepsis will develop within the next four hours. We compared it against four clinical early-warning scores and seven deep learning models.
What we found
Performance is measured with AUC, which captures how well a model ranks patients who will develop sepsis above those who will not (1.0 is perfect, 0.5 is chance).
- Calculators help. Adding calculators improved sepsis prediction whether or not their component variables were fully observed, and models using them alerted roughly an hour earlier with higher recall.
- The estimates are accurate. When the true scores were available, estimation error was close to zero. When we hid 70% of component variables to mimic real-world gaps, SepsisCalc’s estimates had much smaller errors than computing the calculators from the latest available values. Error rose sharply when confidence was low, which is why low-confidence calculators are left out.
- SepsisCalc beat every baseline. Every deep learning model outperformed the traditional clinical scores, and SepsisCalc did best overall, for example 0.839 AUC on MIMIC-III and 0.848 on AmsterdamUMCdb. Removing any piece (event interactions, dynamic calculators, or organ-level prediction), or replacing the dynamic graph with imputation, lowered performance.
From model to bedside
We built SepsisCalc into a system deployed in the Epic EHR at OSUWMC that predicts sepsis risk hourly from the moment a patient arrives.
In the example above, the observed total SOFA score is unavailable because of missing variables, so SepsisCalc estimates it as 12. Clicking an organ on the radar chart, such as respiration, shows how its score evolved and the measurements behind it. Usability evaluation with clinicians is ongoing.
Takeaway
Accuracy alone does not make a clinical AI model usable. By building clinicians’ own tools into the model and being honest about what it cannot estimate, SepsisCalc gives a prediction that comes with evidence: which organs may be failing, and why the model thinks so. That makes it easier for clinicians to check the output and prepare targeted interventions before sepsis sets in.
