Interview with New England Donor Services
Brandon McKown; Data Lead; Sam Endicott, Senior Director of Organ Optimization
(interview by Gina Dunne Smith, Executive Director, IIAM; Mark Hobeika, MD, Associate Professor of Surgery, Houston Methodist Hospital)

Sumary: Donation after circulatory death (DCD) is the fastest-growing category of deceased donation in the U.S., representing 32% of donors in 2022.  DCD unfortunately does not always result in recovery of organs, specifically in donors who do not progress to circulatory death in the timeframe required for the donation to be viable, straining limited resources and potentially disappointing donor families. Previous tools developed to predict DCD donor progression have had limited success and clinical utility. In this interview, leaders of New England Donor Services describe a novel computer algorithm which can more effectively predict outcomes of DCD donor’ progression, supporting decisions for transplant centers, organ procurement organizations (OPOs), and donor families.

Challenge: To support New England Donor Services (NEDS) staff with improved tools to help decide whether or not to proceed with a potential DCD case.

Objectives: To pursue all donation opportunities consistent with the donor or donor family’s wishes while appropriately allocating resources.

Mark: It’s so exciting to learn about your work using machine learning for DCD donor prediction.  How do you teach the machine to predict the outcomes of potential DCD cases?

Brandon: We reviewed 5 years of data, adding new data from each passing quarter so you are constantly feeding the machine and letting it learn.  These are donor and non-donor cases, authorized and unauthorized, that progressed both inside and outside the timeframe for donation which, for NEDS, has historically been within 90 minutes of extubation.  A total of 3079 referrals were used initially to teach the machine.  Our clinical directors identified 53 data points when assessing whether to go forward with a DCD, much of which is demographic data as well as neuro evaluation data regarding reflexes and clinical indicators like heart rate, urine output, creatinine, etc.  We found which data points have a statistically significant correlation with whether a case progressed inside or outside of the 90-minute timeframe.  By running the years of data with the 53 data points through our machine learning process, the result was that 14 of the 53 data points were statistically significant and are factors in the tool’s prediction and our staff’s predictions.

Gina: How does this differ from other tools used by OPOs around the country?

Brandon: We are always evaluating the tool and updating it with the most recent completed quarter’s data, so it is continuously being refined and informed by real data. Roughly 80% of the data is used to teach the model to find patterns and inferences in relation to the outcome.  The other 20% of the data is used to test the model against what it predicted. These two slices of the dataset create a learning cohort and validation cohort.

Mark: When do you apply the tool? Before or after family authorization?

Sam: Ninety-five percent of the time, we’re applying the tool before we talk to the family.  We aren’t revisiting the tool later in the donation process unless there’s a significant change in the clinical facts of the case.  The tool is only viewed by our clinical directors on call because the decision to pursue a DCD or not lies with them.

Gina: How are you using the tool to help your OPO make decisions?

Sam: We are trying to be more deliberate about where and how we place our resources. The tool is helping refine our practice toward increasing the accuracy rate, either way, of whether a DCD donor will progress within the timeframe that will allow for donation.  We will sometimes trust our own judgment over the tool in specific clinical scenarios because the dataset does not capture all of the patient characteristics.  We’d rather get it wrong in the way that we expend resources to support a possible donation that ultimately did not happen than get it wrong in the way that two people didn’t get a kidney that could have.  We’re never going to be 100%, so we want to err in the pro-donation direction.

Brandon: We have quarterly meetings to review any discrepancy in decisions to figure out what did we know that the machine didn’t or what did the machine know that we didn’t.

Sam: When we compared these predictions against the actual outcomes of donors for those cases in our test set to determine the validity of the prediction model, it has generally been between 70% – 75% in its prediction accuracy, and our staff has been correct nearly 79% of the time.  We know there’s always room to improve.

Mark: A take home for me is how your data people don’t live in isolation from the clinical people, this is how you innovate successfully.

Brandon: Absolutely.

Sam: What I think is great about the tool is that it considers data that is local. What is significant in New England may differ from what’s significant in other areas. If we ran this tool in Iowa, maybe the results would be very different. So it’s not only collaboration internally, but collaboration externally with our hospital partners to learn about the local factors we can incorporate into the tool.