The role of outcomes research in drug development, clinical practice, & healthcare policy

How have you applied integrated evidence planning in your current position at Sanofi and previous roles in biopharma and data analytics?

Reverse translation is one method to achieve this, where RWD is applied at strategic points throughout a drug’s lifecycle. This approach is about translation and using patient data to enhance drug development and patient care at various stages. It’s widely recognized that the populations in clinical trials often differ from patients in real-world settings, and this discrepancy is significant. To bridge this gap, it’s essential to integrate RWD into development planning, including clinical trial design, to ensure proper representation.

When considering representativeness, there are a variety of sub-efforts to examine. These include revising inclusion and exclusion criteria and addressing enrollment challenges. Solving these issues isn’t a matter of a single solution; it requires addressing multiple facets individually. Recent publications have shed light on these topics, and there is active work in this area. I believe leveraging RWD is crucial in addressing these challenges and is a key area where it can provide substantial support and insights.

Dr. Stewart, could you share how your current projects are advancing the field and whether they incorporate elements of the strategies we’ve discussed?

Moving forward, the ability to leverage data robustly and meaningfully will be crucial for developing more inclusive, data-driven, and patient-focused evidence. This requires collaboration among regulators, industry professionals, researchers, advocates, and patients to ensure that the outcomes are not only scientifically sound but also meaningful.

Dr. Koshkin, how does outcomes research play a role in clinical practice, beyond the collaborative work undertaken with biopharma companies?

So, recognizing that clinical trials are not designed to answer certain questions has led to a shift towards alternative methods of data generation to assess drug effectiveness in various patient populations. Typically, clinical trials are not representative of the overall patient population, mostly because of stringent criteria they may have. And even though that is often the case, researchers still complete accrual, gather a result readout, and hopefully receive drug approval. If it is approved, the drug is then used in a real-world, patient population, which is similarly nuanced.

Another point to consider is the rapid pace of advancement in oncology and drug approvals. Many trials become somewhat outdated by the time they conclude because they were designed years prior, under different treatment landscapes. As new standards of care emerge, questions arise about the effectiveness of approved drugs in patients who have been exposed to newer treatments. RWE is critical for answering these questions, especially as the rate of oncology drug approvals accelerates and reliance on clinical trials alone is insufficient.

Could you elaborate on the initiatives Friends of Cancer Research has undertaken regarding response endpoints, and how they contrast with conventional clinical trial endpoints?

One challenge in real-world settings is the variability in data capture. Unlike clinical trials, where response evaluation criteria in solid tumors (RECIST) are defined and applied, RWD may not always be recorded with the same level of precision. Clinicians may rely on qualitative assessments rather than strictly quantitative imaging measures, which can influence the interpretation of RWD. Despite these challenges, our work demonstrates that real-world endpoints provide unique value and insights into the performance of therapies. It’s essential to acknowledge the nuances in data capture and ensure transparency in defining, collecting, and measuring these endpoints to arrive at meaningful conclusions and advance the field responsibly.

What are the key differences between physician-assessed responses and imaging-based responses, as compared to the RECIST criteria typically used in clinical trials?

Therefore, when comparing investigator assessments in RWD or clinical trials to centralized assessments, it’s important to be mindful that the numbers may not be directly comparable. Investigators might tend to overcall responses, likely reporting a higher percentage of responses than what a more strictly objective measure like RECIST would indicate. Still, capturing this clinical nuance is important, as it reflects the treatment’s impact on the patient’s well-being.

How do you balance the use of qualitative insights, which may not be captured by scans, with imaging-based assessments? Are these qualitative data points being incorporated into your studies?

Another important aspect is using the FDA “Estimands and Sensitivity Analysis” framework published in 2017,  which involves incorporating endpoints to account for various factors such as treatment and intercurrent events. Given that real-world studies differ from clinical trials, transparency is vital. For instance, scans are not typically done during scheduled visits, potentially affecting PFS assessments and hazard ratios. Understanding these impacts is crucial for drug development. It’s also worth noting that many biases may cancel out when comparing treatments within a real-world framework, particularly in comparative effectiveness studies.

What are your thoughts on emerging technologies that hold promise in outcomes research, particularly with AI’s role in propelling the field forward? What strategies should we adopt to preserve scientific integrity and ensure patient privacy?

One emerging area of interest is causal inference, which builds upon predictive modeling. It addresses the ‘What if?’ questions physicians often consider, such as the potential outcomes of treating a patient one way versus another. Causal inference aims to establish causality, not just make predictions, which has traditionally been the focus in real-world settings. Even traditional techniques, such as inverse probability weighting (IPW), can be integrated into this framework, which offers much to look forward to in the context of machine learning (ML) and deep learning.

Vadim Koshkin, MD: It’s essential to remember that all these technologies are tools, and the quality of data is paramount. Currently, many retrospective real-world studies, particularly those using claims databases, may suffer from lower data quality due to a lack of granularity and detail. Much of the work involves traditional chart reviews, where relevant data is manually extracted, which is incredibly labor-intensive. A new tool capable of performing this task more efficiently, such as a language learning model, could significantly reduce the workload here. However, the success of these projects still depends on the information recorded in patients’ medical records and what is accessible.

Mark Stewart, PhD: At a higher level, there is widespread recognition that integrating these tools into drug development can lead to significant gains in efficiency and provide new insights into data. When shifting towards using this data as evidence for regulatory bodies like the FDA, a different level of validation is required, which is currently a sticking point. The FDA’s recent meeting with their new Digital Health Advisory Committee touched on whether current regulatory frameworks are optimal for the latest generation of AI tools. The responses vary, but a common takeaway is that adaptations could improve the integration of these tools.

How are you ensuring the patient’s voice is central to the design and execution of outcomes research, and what innovative methods have you employed to achieve this?

However, balancing these against the cost of the drug, the trade-off between efficacy and toxicity, and other nuanced questions requires a patient’s perspective, which often differs from that of physicians and regulatory agencies.

Increasingly, patient advocacy groups and disease-specific organizations, such as the Prostate Cancer Foundation or the Bladder Cancer Advocacy Network, are incorporating patients into their boards and discussions. This inclusion is crucial and should continue to ensure that clinical trials are aligned with patient priorities and perspectives.

Mark Stewart, PhD: The inclusion of the patient’s voice in discussions is crucial, especially as we look to the future. The challenge lies not only in keeping pace with technology but ensuring that it is developed to serve patients, rather than the reverse. It’s easy to envision rooms filled with brilliant engineers discovering all the possibilities data can offer, yet it’s critical to have someone there considering the end goal. The key questions are whether these advancements will provide clinically meaningful and important information to physicians and patients, if they are actionable, or if they merely add to the confusion in an already complex environment.