In terms of difficulty, many would say diagnosing a rare or undiscovered disease is more difficult than finding a needle in a haystack – after all, at least you know what the needle looks like.

In general, diagnosis is so challenging because signs and symptoms are frequently nonspecific. Symptoms like soreness, joint pain, or a rash can coincide with countless ailments and diseases of varying levels of severity, and they tell health professionals close to nothing about what is actually wrong with the patient. For this reason, health professionals traditionally employ differential diagnosis, which calls for the comparing and contrasting of various explanations for their observations. This strategy of differentiation requires acute analysis of patterns and correlations between symptoms that can sometimes seem disparate.

Medical algorithms are helpful tools that allow doctors to standardize their procedures for differential diagnosis. When in doubt, turning to proven flowcharts, decision trees, and look-up tables can help organize the process and leverage proven medical research. From a broad standpoint, medical algorithms and informatics allow health professionals to identify and treat a wide range of diseases. And ideally, such strategies would pinpoint groups of related, pathognomonic symptoms that align with a specific disease. However, when symptoms don’t add up, and all known medical explanations are exhausted, doctors and, most importantly, rare disease patients can be left at a standstill.

The Orphan Drug Act Has Incentivized Rare Disease Research

Established in 1983, the Orphan Drug Act has succeeded in encouraging pharmaceutical and biotech companies to address the unmet needs of rare disease patients. In 2016, interest in the rare disease space skyrocketed, as the FDA received a record-high 582 requests for orphan drug designation from pharma and biotech companies. Moreover, as new rare disease medicines continue to splash onto the market, worldwide sales have accelerated. In line with these observations, research from Evaluate Pharma suggests that orphan drug sales totaled to $114 billion in 2016 and are projected to double by 2022.

Rare Disease R&D is Still a Challenge

Despite growth in the rare disease space, there are approximately 7,000 classified rare conditions, according to the National Institutes of Health, and still only about 566 rare disease treatments currently in development. This discrepancy can be attributed to the expenses of pushing a treatment through clinical trials, as well as the complexity inherent in the rare disease space, which often lacks sufficient patients for R&D.

The problem is, scientists still understand relatively little about rare diseases and what they look like. Fibromyalgia, for example, has only recently gained acknowledgment as a major, debilitating condition, even though it affects nearly 450 million people worldwide. While the condition is characterized by intense rheumatic pain, sleep deprivation, and concentration issues, fibromyalgia symptoms are invisible in that they are not physically recognizable to health professionals, and as a consequence, many instances of the disease go undiagnosed. On the other hand, a noticeable condition like Phelan-McDermid syndrome is so rare that it does not even have enough patients around the world to support a clinical study.

The Key to Solving Rare Diseases: Big Data Techniques

For elusive conditions like fibromyalgia and Phelan-McDermid Syndrome, which either have invisible symptoms or sparse patient populations, big data techniques will be able to boost identification and treatment efforts. “Big data” is a hot topic across many industries, with many leaders lauding its ability to drive actionable insights and improve decision-making, and the same is true for the healthcare industry. For diagnostics and condition identification, in particular, big data techniques will allow for the amelioration of previously discussed medical algorithms. Instead of referencing a singular decision-tree or static look-up table, health professionals can use big data solutions to compare dynamic relationships on massive scales to track patterns and predict, with significant accuracy, rare diseases.

How Big Data Techniques Can Streamline Diagnostics

Disease Modeling – Using big data techniques, one can aggregate anything from anonymous health insurance claims to heart rate measurements to genetic code snippets in order to convert findings into diagnostic models. With massive sample sizes of data, it is even possible to correlate certain conditions with widespread variables, initiating the first steps of defining criteria for rare diseases. The more data that is used to justify models, the more complete the models will be in predicting diseases in patients. With enough archived data points, even the most evasive diseases will be identifiable.

Iterative Learning – How to determine if a certain variable has a significant effect on diagnosis? Hypothetically, even if a high percentage of patients diagnosed with a rare disease, say hereditary angioedema (HAE), have contracted chicken pox, the correlation isn’t significant because about 95% of all adults have contracted chicken pox anyway. Instead, using iterative learning and statistics can quantify which variables (potentially related to genetics) actually separate patients with rare diseases from the general population. Over many iterations (machine learning), big data techniques can be applied to numerous variables, from which the results are compounded together to classify the likelihood of a patient having a disease. Even with this much diligence, the answer can never be perfectly surefire, yet the increase in available data can sharpen the confidence level of the diagnosis.

The Future of Big Data and Medicine

Big data techniques have already allowed for impressive strides in research for many rare diseases. Only two generations ago, autism was a seldom diagnosed, misunderstood condition. By amassing large amounts of genomic data, scientists have been able to “hack” various autism-relevant DNA codes through the comparison of genomes of people with and without the condition. Furthermore, by identifying patches of DNA coding related to autism, health professionals will likely be able to treat patients by employing cutting-edge technologies like CRISPR to flip and swap specific nucleotide mutations.

Despite the promise of big data techniques in aiding rare disease diagnosis and treatment, full implementation and the maximization of benefits require further technological development. Artificial intelligence platforms, like IBM’s Watson, must be improved to streamline iterative learning and disease modeling. Moreover, cognitive and quantum computers are essential to supporting the massive calculations and processing power that coincide with big data techniques.

Pharmaceutical and biotech companies must capitalize on the development in this space by taking steps to incorporate big data techniques into their strategies. The end result could be a profound improvement in rare disease diagnosis, better treatments, and the alleviation of human suffering.

Learn More: R&D Data Strategy Consulting

How is your company addressing rare disease R&D? Is incorporating big data techniques part of your strategy? What challenges have you faced integrating new technologies to enhance treatments? Feel free to reach out to me if you’d like to discuss anything related to the rare disease space or big data trends! My contact information is located at the top of the page.

If you found this information interesting, subscribe to our blog to receive the latest insights:

Co-author and contributions by Adam Kershner.