Correcting measurement error bias in conjoint survey experiments

Original

Correcting measurement error bias in conjoint survey experiments

Katherine Clayton, Yusaku Horiuchi, Aaron R. Kaufman, Gary King, Mayya Komisarchik

Abstract

Abstract Conjoint survey designs are spreading across the social sciences due to their unusual capacity to estimate many causal effects from a single randomized experiment. Unfortunately, by their ability to mirror complicated real‐world choices, these designs often generate substantial measurement error and thus bias. We replicate both the data collection and analysis from eight prominent conjoint studies, all of which closely reproduce published results, and reveal high levels of measurement error in all. We then discover a common empirical pattern in how measurement error appears in conjoint studies and, with it, introduce an easy‐to‐use statistical method to correct the bias. Along the way, we provide a much simpler and simultaneously more powerful approach to designing, organizing, understanding, and analyzing conjoint data analyses.