Last week, we reached an historic milestone with the European Medicines Agency’s release of a draft qualification opinion on our 3-step PROCOVA™ procedure. The draft qualification opinion provides the regulatory framework for planning and conducting a Phase 2 or Phase 3 TwinRCT™, using patient-specific prognostic scores derived from participants’ digital twins to improve clinical trial efficiency without introducing bias. To our knowledge, it also represents the first draft regulatory qualification for a machine-learning-based approach to clinical trial design and analysis.

As we reflect on this achievement—one that was over two and a half years in the making—Steve Jobs' words come to mind: “You can't connect the dots looking forward; you can only connect them looking backward. So you have to trust that the dots will somehow connect in your future.” Likewise, Unlearn’s technology evolved through investigation, experimentation, and our unwavering determination to keep moving forward.

We founded Unlearn in 2017 inspired by the possibility of applying machine learning to improve healthcare. We imagined building machine learning models capable of simulating an individual’s health trajectory and asking what-if questions such as, “What if this person changed their diet?” and “What if this person stopped taking their medication?” While we dreamed of a distant future where every physician could choose the best treatment for their patient by first running tests on the patient’s virtual counterpart, we saw more immediate applications tackling many of the problems plaguing clinical trials.

Clinical trials are incredibly slow and expensive. With the end goal of decreasing clinical trial costs and timelines in mind, we developed the concept of creating “digital twins” of trial participants to answer questions such as, “What if this patient was part of the control arm? How would their disease progress?” We realized that this information could allow us to run more efficient, ethical trials using smaller control groups. Although the concept of a digital twin wasn’t entirely new, with prior applications in engineering, their ability to inform medical decisions and use in reducing the size of control arms still belonged within the realm of science fiction. Discovering how far we could push ML to solve these kinds of problems became our north star.

Today, we’re one step closer to making science fiction a reality with our TwinRCT technology, and the path we took to get here was as full of twists and turns as the famous Lombard Street in my home of San Francisco.

We didn’t start out pursuing an approach that could earn a regulatory qualification; quite the opposite, in fact. You see, synthetic control arms were all the rage back in 2018 when we first began thinking of ways that digital twins could be used within clinical trials. In a trial with an SCA, all trial patients receive the experimental treatment. Patient outcomes are compared to a control arm populated with patients selected from an external data source so that (if lots of assumptions are true) it appears as if they were randomized into the study along with the other patients. We figured we could improve on this approach using generative models trained on historical control data to create digital twins of trial participants. In a single-arm trial, each patients’ digital twin would act like an individualized control group, enabling more precise and individualized estimates of treatment effects. Unfortunately, we soon realized that all non-randomized trials with external control arms—even sophisticated ones—are prone to bias, because there are too many unknown confounding variables to trust the results.

But we didn’t give up. Rather, we realized that each patient’s digital twin could be used to compute a prognostic score from their baseline data collected at the beginning of a trial. The prognostic score could be adjusted for in the analysis of a randomized controlled trial to provide more precise estimates of treatment effects without introducing bias, even if the model used to create the digital twins doesn’t include all potentially relevant variables.  We wrote up a patent application in mid-2019 describing our 3-step methodology: train and validate a machine learning model to predict the control outcomes in a target trial, account for the use of the prognostic score derived from that model in the design of a prospective RCT, then adjust for the prognostic score in the trial analysis.

To ensure we were on the right track, we met with the FDA through a Critical Path Innovation Meeting to get feedback on potential uses of digital twins in clinical trials. The FDA confirmed that key aspects of trial designs for pivotal studies were the ability to control the bias and type-I error rate, even in the presence of “unknown unknowns” (i.e., unknown confounding variables).  Intuitively, our PROCOVA methodology did this, but we needed to prove it. We built out a biostatistics team and published a peer-reviewed paper in late 2020 mathematically proving that trials designed and analyzed with PROCOVA provide unbiased estimates of treatment effects and control the type-I error rate. But, we went a step further. We mathematically proved that PROCOVA is essentially optimal, adjusting for a prognostic score derived from a model trained to predict the control outcome leads to the most efficient trial possible at a desired power.

Trials that use PROCOVA always produce unbiased estimates of treatment effects, even if the models used to create digital twins of trial participants are biased. However, that doesn’t mean the models used to create the digital twins don’t matter. The better the model, the smaller the control group. Things really got exciting when we began working on TwinRCTs for Alzheimer’s studies and we discovered we could reduce control group sizes by up to 40%. As soon as we had evidence from case studies, we applied for our EMA qualification in April 2021. In the meantime, we continued to develop and test TwinRCTs within new therapeutic areas, entered a multi-year collaboration with Merck KGaA, and doubled the size of our company.

Unlearn has an ambitious mission to drive clinical trial timelines toward zero. It’s so ambitious, in fact, that we will never fully achieve it. Nevertheless, our TwinRCT solution powered by PROCOVA is a big step in the right direction— enabling smaller, faster trials that ultimately reduce the time it takes to develop new medicines while also limiting the number of patient volunteers randomized to control arms. That translates to new therapies getting to patients who need them faster and at lower prices. We believe that every trial should use our approach to become more efficient and reliable.

With the release of the draft qualification opinion by the EMA, we can now confidently say our TwinRCT solution is appropriate for the primary analysis of Phase 2 and Phase 3 trials with continuous responses. It adds zero bias, increases power, reduces required sample sizes, and makes optimal use of historical patient data. This moment is nothing less than game-changing for illustrating the impact that machine learning can have in the most highly regulated segments of the pharmaceutical industry to benefit both sponsors and patients.

You can read a summary of the draft qualification opinion here and learn how to apply the PROCOVA™ procedure step-by-step in our handbook here.