Why Care About Evidence?

Over the last few years we have noticed the emergence of a number of digital health journals - Lancet Digital Health, NPJ Digital Medicine (Nature), NEJM Catalyst, BMJ Digital, JMIR - which indicates that evidence will be an integral part of medical and life sciences (digital health) research in the future. 

The HealthXL Digital Health Evidence Review 2019 report aims to identify, categorize, and summarize all existing, relevant studies in digital health. Through this report, we want to establish a clear understanding of the existing body of clinical evidence for digital health, its maturity, current gaps and needs.  

This is our first attempt (and anyone’s from what we have seen) in bringing together all digital health evidence data points to make sense of the space. While we are pleasantly surprised by the volume and progress of the evidence identified, this effort is far from over.

While evidence is important and irreplaceable, evidence always requires interpretation and cannot stand alone. Evidence can never dictate the optimal course of action, it should always be considered in the context of values and preferences.

‍Our Hypotheses ‍

Through this report, we wanted to evaluate the market with the intent to address the following  hypotheses: 

What We Found

Once we had parsed the data, the results we found offered a mixed-bag in terms of corroborating or contradicting our initial hypotheses. 

There was no positive correlation between the global NCD burden and number of digital health related studies published. Within this result we observed that protracted illnesses and conditions with increased chronicity appeared to have a higher density of digital health publications, which makes sense given the clear opportunity for technology to play a role in the management of disease at home. 

Innovation Hotspots 

Going into the investigation, we had hypothesised that there would be an observed concentration of digital health studies published in the US and in Europe. Supporting this presumption, the data showed that the US emerged as the most active publishing country with 42.42% of published digital health studies. The EU emerged as the second most active publishing region with 24.21% of published digital health studies. These results highlight the need for more diversity in representation beyond the US and EU alongside the limitation of our investigation to capture non-English language studies. 

We have also analysed and ranked academic centers and hospitals based on how active they are in publishing evidence in digital health. 

Top funded Digital Health Companies and their Evidence Generation Activity

A third result worth mentioning is our investigation of the correlation between the top funded digital health companies and their publication activity. While we would have hoped that the companies attracting the greatest amounts of investment would also be those championing evidence of their clinical efficacy, feasibility, etc, our hypothesis was challenged by the results that emerged in our dataset. 

While a number of companies amongst the top funded did have publications in some form, we did not see the hoped for correlation between the two variables. This finding in particular raised some relevant thoughts on the need for evidence as a prerequisite for digital health companies raising capital. Having said that, we are all in agreement that this is not the only important criterion - a large body of clinical evidence still does not imply commercial traction or industry adoption. 

Key Takeaways

1. In general, we did not find a proportional alignment between global NCD burden and volume of publication of digital health studies. 
2. Imaging, Genetics, and Mental Health emerged as the top fields of digital health studies across the top ten publishers by volume. The top publishers were those that occupy the space at the intersection of computer science, engineering and medical fields.
3. We identified more RCTs than we had anticipated. Using only RCTs as a benchmark for quality evidence in digital health will limit knowledge of which interventions actually work best in practice.
4. In terms of geographic spread, unsurprisingly, US and Europe emerged as the most active in evidence generation.
5. We could not establish a clear correlation between the top 5 funded digital health startups and the generation of (published) evidence. This leaves us with many unanswered questions in terms of whether we are investing in the right channels while developing digital health solutions. 
   

We also found lots of anomalies and aberrations. 

1. As a general rule of thumb, we believe riskier products in riskier disease areas should have higher evidence thresholds. This was not necessarily true from our dataset. 
2. A number of solutions that play in the D2C space do not necessarily warrant clinical validation, and this was evidence when we assessed D2C digital health companies and their products. 
3. You would assume that rare diseases will not be investigated as much. Conditions/ diseases that are rare, and have expensive or ineffective pharmacological interventions seem to present a market opportunity for digital interventions, as evidenced by a high return of studies in those areas (Neurology, for eg)
   

What Next? 

While we are pleasantly surprised by the volume and progress of the evidence identified, this effort is far from over.

This is our first attempt (and anyone’s from what we have seen) in bringing together all digital health evidence data points to make sense of the space. While we are pleasantly surprised by the volume and progress of the evidence identified, this effort is far from over.

HealthXL have developed a research management system to build on top of the foundation laid by Microsoft Academic, further augmenting their automated tagging systems with our own, as well as triggering human validation when required. Through the development of our own research management system, we have been able to develop a combination of automated and human validation processes to identify and match startups to the research they author, or for which their technology is the focus. If you’re interested in learning more or think you would get value out of such a system, it will be great to hear from you.

This blog is a sneak-peek into the data and results that we found. If you are a HealthXL member, for a deep dive into all of our results and detailed graphs of results, contact chandana@healthxl.com. If, as a non-member, you wish to purchase the report, contact julie@healthxl.com. 

Methodology

We sourced publications from Pubmed for the years 2000- 2018, and cross-referenced them with Microsoft Academic. We then assigned a digital health status which yielded 150,982 publications, did multiple rounds of data validation, and grouped for disease burdens using keywords and terms that could be bundled into each disease area (for eg, ‘heart’, ‘artery’, ‘aortic’, ‘chronic heart failure’, etc were all grouped under ‘Cardiovascular Disease’)