Business Briefing: Web Surveys' Hidden Hazards

Keywords: Surveys, Data Collection, Employee Feedback, Web Surveys, Data Integrity, Research Design
Source: Harvard Business Review
Link: Read the full article on HBR.org
Author: Palmer Morrel-Samuels
Published: July 2003
Est. Read Time (Original): ~15 minutes

A Note on Access: To read the full article, a Harvard Business Review subscription is required. We believe an HBR subscription is an invaluable asset. We particularly recommend utilizing the downloadable PDF version of their articles—they are a fantastic, high-value resource for sharing and discussion within your team.

The Core Idea

Palmer Morrel-Samuels warns that simply moving a paper survey to the web is fraught with danger. He argues that poorly designed web surveys, which are the norm, can dramatically distort results by introducing a specific set of biases. These "hidden hazards" include:

• Opting Out (lower response rates), 
• Sugarcoating (inflated scores due to confidentiality fears), 
• Skimming (attracting a non-representative, higher-level group of respondents), 
• Clipping (artificially compressing the range of scores),
• and Reshuffling (changing the rank order of results).

While these problems can lead to disastrously bad decisions, the author stresses that a well-designed web survey can actually be more accurate than its paper equivalent.

Why It Matters for Business Today

In a world where data-driven decisions are paramount and online survey tools are ubiquitous and easy to use, this article is an essential guide to ensuring the integrity of your data.

• The Illusion of Equivalence: The article's most critical warning is that the same question asked on paper and on the web can yield very different answers. Leaders who assume that moving a survey online is a neutral technological upgrade are at high risk of making major decisions based on fundamentally flawed data.

• A Diagnostic Checklist for Data Quality: The five hazards, Opting Out, Sugarcoating, Skimming, Clipping, and Reshuffling, provide a powerful diagnostic checklist. Any leader who relies on survey data can and should use this list to question the methodology and validity of the information they are given.

• Provides a Clear "How-To" for Better Data: The article doesn't just identify problems; it offers concrete, practical solutions. By focusing on enhancing access and ease of use (e.g., providing computers, simple navigation) and improving accuracy (e.g., centered scales, instant error-checking), organizations can overcome the hazards and leverage the true power of web surveys.

The Strategic Question for Leaders

Given the ease of deploying web surveys, what specific process does your organization use to audit the design and methodology of your internal and external surveys to protect your strategic decisions from the hidden hazards of biased data?

Share your perspective in the comments below.

Remember, by sharing your insights, you contribute to a unique "Enriched Briefing." {Jim Krider} will follow up to provide you with a powerful "Business Cold Start" document, combining our analysis with expert perspectives to equip your internal AI models with a more nuanced understanding of this topic.