Harnessing big data to capture patient voice
Client | Organon
Organon needed an innovate research strategy to analyze TONS of opinion data
Organon is a global women’s health pharmaceutical company. In 2023, their Outcomes Research Division approached us to develop a research methodology that could capture patient voice quicker and less costly than traditional interviews and focus groups. Their idea was to analyze thousands of product reviews left on publicly available medical review forums. We developed a solution that provided robust quantitative and qualitative findings that they could use to develop strategy speaking to three key customer segments: patients, providers, and payers.
How It Began
Discovering a new source of real-world patient voices
Our client’s prior market research found that their target demographic frequently used the internet and social media to find information and resources about their condition and effective treatments. Their idea was to leverage reviews from online medical forums as a source of real-word data on patient experiences.
Even Food and Drug Administration (FDA) guidance recognizes the potential of online tools as useful data sources “to complement literature review findings, inform the development of research tools (e.g., qualitative study discussion guides), or as a supplement to traditional research approaches (e.g., literature, one-on-one interviews, focus groups, or expert opinion).”
The analysis of unsolicited, unfiltered product reviews is a novel data source to support real world evidence generation of the patient voice. As we discovered, online reviews of products used to treat medical conditions are a powerful tool potential consumers use to learn patient perspectives about a treatment’s effectiveness, side effects, and value.
The Solution
Big + little data approaches to finding value
Mining online product reviews yields enormous amounts and types (structured vs. unstructured) of passively collected data (i.e., volume and variety). Moreover, the real time nature and quality (i.e., velocity and veracity) of these data required specialized analytic techniques to manage and extract value from this information. We developed a custom webscraper that automated the real-time extraction of these data and fed them into a SQL relational database.
Our next hurdle was making sense of it all. We designed a study protocol that used natural language processing, a type of machine learning that could determine whether the reviewer’s attitude towards the product was positive, negative, neutral, or another emotional attribute (aka “opinion mining”) just by the words they wrote. The benefit of this approach is that it quickly categorizes reviewer sentiment about a product and allows comparisons to be generated across multiple products, a key request of our client.
A common limitation of this approach is that it does not “go deeper” than characterizing the sentiment. Furthermore, as social scientists, we were highly concerned about the validity of these artificial intelligence algorithms. Our solution was to incorporate a traditional qualitative approach, using thematic analysis as well as human checks of the data. What we found exceeded all expectations.
Quick Stats
New Insights
Ratings, figures, and quotes, Oh My!
Our client was amazed at the breadth and depth of information we generated. Us too! Findings were able to bolster value statements, inform payer recommendations, and generate information to inform provider education. Insights complimented literature reviews, informed future studies, and suggested areas for future product development. Here’s a glimpse of the findings our project delivered:
Quantitative Metrics
Establish how representative reviews are of the broader market share and hone in on a target patient population.
Patient-provided ratings and sentiment analysis scores revealed patient likes and dislikes for guideline-recommended products.
Statistical hypotheses examined patient-preferences in different formularies and routes of administration.
Scores can be stratified by reviewer demographic characteristics.
Quantify adherence and reported use of other products (“cross products”).
Use word clouds to characterize common side effects, product characteristics, and value statements, which can be compared across products.
Qualitative Themes
Describe the disease state and its impact on relationships.
Identify which side effects are most related to medication discontinuation.
Discover reviewer-provided recommendations on how to use the products, such as to alleviate side effects.
Ascertain outstanding questions patients have about the product or myths about the condition.
See a preview of the results at our poster presentation in Atlanta, Georgia at ISPOR 2024 on Tuesday, May 7th 3:30pm.
Or contact us now to discuss how we can apply this methodology to outcomes research evidence generation for your new and emerging therapeutic areas!
