HealthXL are proud to share that we are a recent collaborator with the American Medical Association (AMA) Physician Innovation Network (PIN). HealthXL members can now connect with peers, nationwide physicians and industry experts to validate their solutions, share best practices, get feedback to improve products, test solutions in practice, and also participate with the PIN in virtual panel discussions on key topics in health technology.
Both HealthXL and the AMA are bullish about evidence. HealthXL recently released our Digital Health Evidence Review 2019. Evidence is important and irreplaceable; however, evidence always requires interpretation and context. Working groups of physicians like the AMA can undoubtedly add this context, as well as drive innovation. Digital health currently lacks universally accepted frameworks for evidence generation, and this blog attempts to offer some ideas on how we, as an industry, can support innovation.
Evidence frameworks to support responsible health care innovation: An AMA perspective
Authors:
- Mira Irons, MD, American Medical Association Group Vice President, Chief Health & Science Officer
- Sean McConnell, Ph.D., American Medical Association Senior Policy Analyst
Innovation is a priority, as it is essential for achieving the quadruple aim of:
1) Better patient experience
2) Improved population health
3) Lower overall costs
4) Enhanced professional satisfaction of the healthcare team.
To realize this vision, physicians must trust the technology-enabled tools that they use in patient care, and patient safety must be protected throughout, via a path of responsible innovation. Currently, several challenges may stand in the way of routine adoption of appropriate, evidence-based innovations. In order to evaluate these innovations, specific evidence frameworks will be applied by researchers, regulators, payers, clinicians, as well as patients to assess these innovations. However, there may be a lack of consensus and shared understanding around what evidence frameworks are needed and what levels of evidence are appropriate for new technology-enabled innovations. To accelerate, or at least not impede innovation, there may be value in identifying and ensuring a common understanding of the different evidence frameworks in this space among all stakeholders.
Today we stand at a crossroads for responsible clinical innovation. On the one hand, the pace of innovation is ever-accelerating, with advances in augmented intelligence (AI), digital health and genomics poised to dramatically shift models for how care is provided to patients, towards prediction, prevention, and precision medicine. On the other hand, we still lack consistently applied evidence frameworks, a common taxonomy, and clearly defined or universally agreed upon levels of evidence or thresholds for these new rapidly emerging areas of innovation. For example, one outstanding question in this area is whether new types of evidence may be required to support these emerging technologies compared with those more traditionally used in medicine, e.g., randomized controlled trials (RCTs).
Challenges in building and applying evidence frameworks
Challenge #1 – Consistency for evidence frameworks being applied to disruptive clinical innovations enabled by new technologies.
RCTs have represented the ‘gold standard’ for generating clinical evidence in many domains. However, despite their clear benefits in terms of leveraging quality study designs, RCTs also have limitations. RCTs require significant investments in time and money, and may require prohibitively large numbers of participants, particularly in the case of rare diseases, which may invoke ethical considerations. This may call into question the role for RCTs as a de facto gatekeeper which may restrict patient access in many evolving areas, such as genomic testing and precision medicine.
Furthermore, RCTs may not be able to keep up with rapid iterative updates in these new technologies, including adaptive machine learning systems, or even updates for the list of genes in a panel, as any such variation could render previous RCT results obsolete. This inflexibility may limit RCTs from being the best assessment tool for many forms of innovation such as digital health applications and AI solutions. Yet another challenge for RCTs is a lack of sufficient overlap in characteristics between trial participants and patients in the real world. At times this situation may be due to restrictions such as specific exclusions, but however it may arise, underrepresentation of certain groups in trials may ultimately contribute to inequities in care, including when robust evidence remains lacking for use in certain groups of patients that might benefit.
Given the unique challenges of RCTs and traditional evidence generation, new avenues for evidence generation are being explored by many stakeholders. Real world evidence (RWE) has emerged as one solution that may help address some of the common challenges for RCTs. However, the shift to collection of real world data to produce RWE also poses challenges, including identification and mitigation of inherent sources of bias, and shift to more routine surveillance that may invoke privacy concerns for patients. Additional discussion is needed to better define how such models might work in tandem to promote efficient generation of the most appropriate forms of evidence for different applications.
Challenge #2 – Improved communication to address lack of transparency around evidence requirements.
Consistent interpretation of, and consensus around evidence depends on the quantity and quality of evidence, as well as application of specific frameworks to interpret the evidence. Quantity encompasses sample size, as well as number of independent studies from different groups. Quality includes effective removal of confounding factors and sources of bias via well-designed studies, and sufficient overlap between composition and diversity of study participants and relevant populations. Specific evidence frameworks also mean that evidence could be deemed sufficient in some contexts but not in others, depending upon how different criteria are applied.
Payers and other stakeholders have their own predispositions and schema for what they seek from innovation. Priorities undoubtedly may vary. However, when transparency is lacking, this can add confusion to the situation, and even frustration, when it is difficult for innovators to know in advance where they might stand on meeting any requirements, and how specific criteria are applied. This can contribute to costly wasted effort and give the impression that additional hurdles are being raised that cause undue implementation delays.
Transparency from the start is essential to ensure efficient introduction of innovations with demonstrated value. Ethical mandates also apply, such as preserving patient access to innovation in an equitable manner, which may be challenged when only the wealthy can afford to pay out of pocket for innovations. Stakeholders should openly communicate about evidence requirements and help promote transparent frameworks that facilitate access to innovation in a manner that provides clarity and equity.
Challenge #3 – Education to alleviate innovator confusion about different types and levels of evidence.
Evidence is foundational for scientific research and clinical progress. It is the cornerstone of our modern era of evidence-based medicine, and key to ensuring that clinical applications are safe, effective and equitable. Evidence should be collected routinely, not only during development stages, to help establish value and promote trust across different stakeholders, and to ensure that applications remain centered around patients. In a learning health system, there are opportunities for continual improvement throughout the process of applying new solutions, from initial innovation to improving implementation. Evidence is the engine that drives these advancements.
Less-seasoned innovators may be surprised to learn that evidence can mean different things to different stakeholders. For example, while evidence of safety and efficacy are both key for regulatory decisions, these types of evidence alone may be considered insufficient for payment and coverage. Additional factors such as evidence of clinical usefulness, cost-effectiveness, and generalizability for the population can also become key considerations for payment and coverage decisions. Some stakeholders may also look for additional signals such as practice recommendations and alignment of clinical guidelines, including from national medical specialty societies.
As not all forms of evidence are applied equally by all stakeholders, including different frameworks or varied levels of evidence, there may be a need for additional education in this area. Innovators may benefit from educational tools that provide additional clarity around specific expectations and requirements from different stakeholders. These educational tools may help alleviate confusion about why particular innovations may not meet thresholds under certain frameworks, and perhaps even speed the path to adoption for many new applications.
Challenge #4 – Collaboration to address lack of consensus around evidence among different stakeholders.
Inconsistent terminology and standards can contribute to challenges for interpretation of evidence. Clear definitions are foundational to ensure consistency, and standards bodies can play an important role to help ensure consistent terminology among different stakeholders. While consistent terminology is needed, clearly defined standards can also be a challenge. Occasionally, standards are lacking, and can be developed by relevant stakeholders such as national medical specialty societies via practice recommendations and clinical guidelines.
Alignment around key aspects such as appropriate endpoints or most important outcomes can also promote clarity. Besides these standards, thresholds for acceptable evidence should also be clearly defined. Collaboration between developers and appropriate clinical experts can help establish these thresholds for acceptable evidence. Incorporating comment and review processes can help provide additional transparency for the routine application of different evidence frameworks, such as during value or technology assessments.
It is important for innovators to understand evidence frameworks that may be required to help develop trustworthy tools. It is also imperative for physicians to understand these frameworks, to better empower them to develop trust in these innovative tools. Questions might include: “Is the data used in training the model also reflective of my patient population?” or “If a continuous learning algorithm is employed, how do we ensure that its output remains consistent with safety, efficacy, equity, trust, and value?”
Ethical principles to support responsible innovation
Whatever the evidence framework, core ethical tenets of health care should be advanced, including:
In conclusion, to help support responsible clinical innovation, we must identify and promote the most appropriate evidence frameworks that support robust evaluation of disruptive technologies including genomics, digital health and AI applications. It is important for stakeholders to develop, form consensus around, and provide additional specificity as well as transparency around different evidence frameworks. Improved communication, education, and collaboration around how standards are applied can help promote greater consistency for how applications are evaluated, to facilitate innovation.
Responsible evidence frameworks and standards are those that are consistent with established ethical principles such as patient safety and physician trust, ensuring that innovation moves towards demonstrated impact and value for all. As stakeholders evaluate the evidence for clinical applications, expectations should be clearly communicated, and decisions under a given framework should remain open to outside review. Otherwise, innovators may be surprised by the appearance of additional rungs on the ladder, and adoption of emerging technologies may be delayed, with unfortunate implications for patient health.
Check out the many ways the American Medical Association (AMA), together with partners, is driving innovation for digital health, augmented intelligence and precision medicine:
Augmented Intelligence
The AMA is committed to helping physicians harness AI in ways that safely and effectively improve patient care. Explore the AMA’s current policy and recommendations on the application of AI in health care, as well as education, insights and resources.
Founded by the American Heart Association, the AMA, the DHX Group and HIMSS, Xcertia is an industry collaborative formed to improve the quality, safety and effectiveness of mobile health applications. Xcertia invites other organizations and individuals to join the collaboration and add their expertise to the development of guidelines for mobile health apps.
Digital Medicine Payment Advisory Group
Digital medicine presents an opportunity to improve access and to offer cost-effective medical care to a large number of patients with diverse needs. Led by the AMA, this group of 15 advisors is responsible for identifying barriers to digital medicine adoption and proposing comprehensive solutions revolving around coding, payment, coverage and more.
Digital Health Implementation Playbook
New technologies are providing innovative ways to improve health care, enable lifestyle change and create efficiencies, and yet, clinical integration of these tools is still lacking. This playbook seeks to change that by providing the most efficient path to implementing digital health solutions, including key steps, best practices and resources to help organizations achieve adoption and scale.
Recognizing the need for physician input in new health care solutions, the AMA launched the Physician Innovation Network—a digital matchmaking platform where physicians and entrepreneurs can connect and collaborate to improve digital health care solutions.
Integrated Health Model Initiative
Living at the intersection of health care, technology and data science is where you will find the AMA Integrated Health Model Initiative™ (IHMI). By creating, enabling and applying common standards for industry-wide adoption, our mission is to make the world’s growing health data profoundly more portable and actionable.
The Current Procedural Terminology (CPT®) code set offers doctors and health care professionals a uniform language for coding medical services and procedures to streamline reporting, increase accuracy and efficiency.
Health2047 Inc. is a Silicon Valley-based innovation company that focuses on catalyzing and commercializing disruptive ideas to drive fundamental transformation in U.S. health care.
