New Paper Finds Evidence of Fabrication in International Surveys

Data fabrication (i.e. creating fake survey data) has been a concern since the beginning of organized survey research. The risk has always been present that someone in the chain of survey data collection and processing would simply invent the data rather than gathering it the right way. A new paper from MPG President Steve Koczela and several co-authors shows the threat remains real today, and is in need of further attention. Another working paper paints an even starker picture, particularly in surveys conducted in non-OECD countries.

Koczela's new paper is published in the journal of the Statistical Journal of the International Association of Official Statistics, and was given this week at the International Total Survey Error Conference in Baltimore, MD.

Curbstoning and beyond: Confronting data fabrication in survey research Steve Koczela, Cathy Furlong, Jaki McCarthy and Ali Mushtaq

Abstract. The literature on survey data fabrication is fairly thin, given the serious threat it poses to data quality. Recent contributions have focused on detecting interviewer fabrication, with an emphasis on statistical detection methods as a way to efficiently target reinterviews. We believe this focus to be too narrow. The paper looks at the problem of fabrication in a different way, exploring new data that shows the problem goes beyond interviewer curbstoning. A surprising amount of apparent fabrication is easily detected through comparatively rudimentary methods such as analysis of duplicate data. We then examine the motivations behind survey data fabrication and explore the utility of fraud investigation frameworks in detecting survey data fabrication. We finish with a brief discussion of the importance of additional research in this area and suggest questions worth exploring further.