As the length of the NHIS grows, the attention spans of respondents shrinks which means declining response rates. Maggie Koerth-Baker at FiveThirtyEight has put together an excellent article on why the impact that decreasing the length of the NHIS could have in the US:
“Every year since 1957, tens of thousands of Americans have opened their homes to government survey takers who poke and prod their way through a list of intimate and occasionally uncomfortable questions. That process is part of the National Health Interview Survey, the gold standard of health data in the United States.
Experts say it is unique in its sample size, the scope of its questions, and how long it has existed. The NHIS is crucial to our ability to track the prevalence of diseases and health-related behaviors, and to answer complex questions involving health, income and family demographics. And now it’s about to change. The National Center for Health Statistics, the branch of the Centers for Disease Control and Prevention that conducts the survey, is planning a big shift in how the NHIS works, one that some scientists fear will impair their ability to learn about things like how stepmothers can subtly disadvantage children — investigations that end up shaping everything from the way your money gets spent to the policies your legislators vote on.
At issue is the question of how long anybody can reasonably expect Americans to put up with sharing intimate details of their lives with taxpayer-funded bureaucrats.
The 35,000 households — more than 87,000 individuals — who will be interviewed for the NHIS this year will have to devote an average of 90 minutes of their time to the survey. They aren’t paid. There’s no prize at the end. “That’s a lot to ask of people,” said Stephen Blumberg, associate director for science at the National Center for Health Statistics.
The thing about a gold standard is that every researcher wants to be a part of it. Twenty years ago, the NHIS lasted about an hour. But over time, more government agencies and scientists wanted to add more and more questions to make it more and more useful. Meanwhile, participation has been dropping. In 1997, the survey had a 91.8 percent response rate. “The response rate today is about 70-73 percent,” Blumberg said.
So scientists and the government are caught in a tug of war over the survey. Pull too hard one way, and they risk losing access to valuable information. Yank it back the other, and Blumberg worries that the participation rate could fall below 60 percent.
But nobody knows whether making it shorter will actually increase the response rate. The NHIS is not the only survey that’s losing respondents over time. Across the board, fewer and fewer Americans are choosing to participate in surveys of all kinds. This trend has been documented by the National Research Council and the Pew Research Center. Scientists I spoke to repeatedly referred to this problem as “the elephant in the room.”
It’s not clear why this is, but the length of the survey is probably not the primary driver, said James Lepkowski, director of the University of Michigan’s program in survey methodology. “If length is an issue, what you should see is that people break off in the middle. That’s actually very rare.”
This article is a perfect illustration of a primary problem facing researchers today. Is shorter and simpler better? Not always.