With the goal of reducing deaths related to postoperative complications, a UPMC team created an artificial intelligence-based tool to establish a risk score for patients going into surgery. That software predicts patient mortality 30 days post-surgery better than standard risk metrics, the scientists reported in a paper published last month in the journal JAMA Network Open.

The technology has been used at 20 UPMC hospital sites, in tandem with hospitalists and physicians, for the past two years.

“While new technology has really helped advance surgery, outcomes have really not changed much, and complications still occur,” said the paper’s first author, Aman Mahajan, chair of the Anesthesiology and Perioperative Medicine at Pitt’s School of Medicine and executive director of UPMC Perioperative and Surgical Services.

Aman Mahajan, chair of the Anesthesiology and Perioperative Medicine at Pitt’s School of Medicine, is first author on the new paper. (Aman Mahajan)

More than 4.2 million people die every year across the globe due to surgical complications, according to one 2019 estimate done by scientists at the National Institute for Health Research Global Health Research Unit on Global Surgery. Another 2019 estimate, by the University of Michigan Institute for Healthcare Policy and Innovation, found that 100,000 people die in U.S. hospitals each year after surgery.

Advertise with us

The team, composed of anesthesiologists, surgeons and data scientists, trained the AI on 1.4 million patients who had received surgery from UPMC between 2012 and 2019. Data was then disidentified from individual medical records so a person’s name and other identifiable information could not be traced to them.

In use, the model reads the medical records of patients scheduled for surgery and alerts relevant physicians of each patient’s mortality risk. The model is meant to provide a real-time risk assessment so that, if needed, patients can be “medically optimized” for surgery — that is, the surgery may need to be postponed to get a patient to a healthier state before operation. This could include treating diabetes, quitting smoking or increasing physical activity. The team is also working to bolster UPMC’s “prehabilitation” program to receive these patients.

Humans won’t be taken out of the equation. The technology is intended to be used in tandem with physician discernment and decision-making, said Mahajan. “The information on risk is provided to clinicians, and they can generate their own risk score,” he said. “There’s a human making the last call, and that’s really how it should be.”

More than 3,000 variables were used in building the machine learning model, such as patient history with diabetes and blood pressure, as well as social determinants of health.

Mahajan said that traditional machine learning models used in this context have, in the past, not taken social determinants of health into account, but that his team felt strongly about including variables to encompass that metric. This might include a patient’s proximity to healthy food options, education level, access to housing, chronic stress level and more.

Advertise with us

Due in part to the large dataset the model was trained on, Mahajan said it is representative of the population it serves. “We know our model is very accurate for our community. It may not apply to other communities,” he said.

While recalibrating the AI to work in a new region is not hard in theory, said Perry Wilson, an associate professor of medicine and public health at Yale University and director of Yale's Clinical and Translational Research Accelerator, the 3,000 variables standardized for UPMC might not be the same as those, say, Yale uses to build its own AI models. That complicates how such a model could be built for use in a different part of the country.

The study is a great first step, though, Wilson said.

Perry Wilson, an associate professor of medicine and public health at Yale University, has created similar medical technology using artificial intelligence. (Perry Wilson)

“I certainly want to see more studies of this type,” he said. Wilson has pursued similar tools for use in the medical sphere, like using an automated alert to encourage physicians to prescribe a life-saving medication to treat heart failure. A strength of Mahajan’s study, noted Wilson, was that the technology was actually implemented into the population it serves and validated in another cohort of more than 200,000 UPMC patients.

While a diverse and representative sample might protect against encoded bias in AI technology, however, it also has the potential to worsen it.

Advertise with us

“[J]ust because your model has a large number of people from minority groups does not mean these groups will be treated fairly,” Wilson said.

Hypothetically, if the model communicates a higher risk of mortality from surgery for Black patients, and surgery is frequently postponed or canceled for Black patients, that could worsen health disparities in that population. Wilson said guidelines ensuring humans remain in the loop and a systematic assessment of models over time can help prevent AI from steering outcomes in the wrong direction.

“Patients need to know when a model has been applied to them,” he added. “There has to be transparency for patients, too.”

Another risk Wilson highlighted is that, while medical data is disidentified from patients, reidentifying a patient is a definite possibility.

“A major risk is patient privacy,” he said, “but that’s true whether or not this research is done. The data is there no matter what, and health care systems are hacked. We accept that risk as part of how the modern world works.” Wilson added that it’s normal for AIs to be built from patient medical data.

Advertise with us

The main limitation he saw in the study is a lack of what he called “actionability.”

“Will the use of this [tool] improve outcomes for patients?” he asked. “What we need to show is that using this is better than not using it. Does the use of this tool get patients into prehabilitation?” He proposed a next step of a randomized controlled trial, in which half of the patients receive the alerts indicating risk and half do not.

“There’s a lot that has to be done, and it’s going to happen very quickly,” he said.

Hanna Webster: hwebster@post-gazette.com