Assessing analytical robustness in cross‐cultural comparisons

A framework that enables the researcher to assess the robustness of empirical findings from culture‐specific response styles (CSRS) is proposed, and can be used to inform researchers and data analysts about the extent to which the validity of their conclusions is threatened by data contamination.

Abstract

Purpose The purpose of this paper is to critically review past recommendations to correct for cultural biases in empirical survey data sets, and propose a framework that enables the researcher to assess the robustness of empirical findings from culture‐specific response styles (CSRS). Design/methodology/approach The paper proposes to analyze a set of derived data sets, including the original data as well as data corrected for response styles using theoretically plausible correction methods for the empirical data at hand. The level of agreement of results across correction methods indicates the robustness of findings to possible contamination of data by cross‐cultural response styles. Findings The proposed method can be used to inform researchers and data analysts about the extent to which the validity of their conclusions is threatened by data contamination and provides guidance regarding the results that can safely be reported. Practical implications Response styles can distort survey findings. CSRS are particularly problematic for researchers using multicultural samples because the resulting data contamination can lead to inaccurate conclusions about the research question under study. Originality/value The proposed approach avoids the disadvantages of ignoring the problem and interpreting spurious results or choosing one single correction technique that potentially introduces new kinds of data contamination.