Scraping the news for insights

by Jon Scaccia December 21, 2021

Health equity is a major topic and research focus over the past few years. And with good reason! For those of us who want wellbeing to improve in our communities, we can’t have that happen unless we working for improved outcomes for all.

For that reason, health equity is more than an academic topic. Working toward health equity requires on the ground activity and engagement. Oftentimes, we don’t hear about health equity projects because people lack the time and resources to write them up for peer-reviewed articles. So where can we find these stories?

Looking in the news

The mainstream media can be a place where we can find stories on health equity. PubTrawlr is developing methods to incorporate news stories into our syntheses, but for today, let’s look at what has been discussed recently in Health Equity in a specific location, Missouri.

Our sister organization, The Dawn Chorus Group is involved in a community-based evaluation looking to reduce vaccine hesitancy and build the infrastructure to address other social determinants of health. One of these communities is in Missouri. As an experiment, we used a news crawler to pull all recent articles that talked about health equity AND Missouri. This yielded 84 full-text articles going back to November 9, 2017.

Publishers and high-level trends

This first figure shows who has been published articles. The outlet with the highest number was Health Affairs, which publishes a blog in addition to peer-reviewed articles. What is extra interesting is that most of these outlets are specialty publications. Politico is national, while KCUR and the City of St. Louis are local. So, we’re not seeing the term “health equity” filter down much into “general outlets,” at any regular frequency.

Using Words as Indicators

This word cloud and network plot show the most frequently occurring words and words strings across these articles. We can see several COVID clusters, such as the ones dealing with vaccines and delta. We also see a policy cluster, specifically around medical expansion.

Emotions in articles?

Something we don’t normally do at PubTrawlr is sentiment analysis, which looks for the emotions behind words. However, since we’re looking at mainstream coverage here, it may be interesting to see the types of emotions that are being conveyed.

This first figure uses the bing lexicon, which rates words as positive or negative. I plotted this trend over time by month to see how the discourse was rated. Interesting, we’ve had an upward swing over the past year.

To be a bit more nuanced, I also used the NRC lexicon, which assigns words to one of seven different emotions. The first figure shows that trust was represented over all other emotions.

I then plotted an NRC trend graph, looking at the proportion of emotions over time. Over the past month, we see fear and digust trending upward, so potentially that is due to omicron (or at least the discussion around omicron).

Topics in the Articles.

Finally, I ran a basic LDA topic model. We can see that the highest frequency topics dealt with policies and community services, followed by the hospital system and covid.

Summing it all up

Health equity is a major topic and research focus over the past few years. For those of us who want wellbeing to improve in our communities, we can’t have that happen unless we working for improved outcomes for all. For that reason, health equity is more than an academic topic. Working toward health equity requires on-the-ground activity and engagement (or at least it should).

So where can we find these stories? Looking in the news. The mainstream media can be one place where we can find stories.

PubTrawlr will be implementing a similar pipeline for you to explore themes and trends. Stay tuned!