Asking the right questions, getting the wrong （提出正确的问题,得到错误的）.pdf

Asking the right questions, getting the wrong answers On the risks of using factor analysis on survey data Cees van der Eijk and Jonathan Rose Workshop on Issues in latent structure analysis using survey data A Nottingham-Warwick-Birmingham Advanced Quantitative Methods (AQM) Workshop – organised by the Nottingham ESRC DTC in collaboration with the Methods and Data Institute Nottingham, 8 March 2013 The Right Question “which items (if any) from a pool can be regarded as indicators of the same underlying phenomenon?” This is the right question when analysing survey data, because – Questionnaire design often includes multiple operationalisations – Even when not explicitly intended in the design of a questionnaire, many seemingly different behavioural, opinion or attitudinal items relate to a smaller number of underlying orientations – If a set of items can validly be regarded as indicators of a latent variable, respondents’ positions on that variable can be estimated better on the basis of a composite measure than by any single item (better: more reliably, and with more discrimination) – The conceptual framework of a study simplifies, becomes more general (and generalizable) if a set of items can validly be regarded as indicators of a latent variable. 2 Multiple indicators in surveys Most popular forms of multiple indicators for probing attitudes, orientations, predispositions etc. in surveys are • Dichotomies – yes/no – agree/disagree – Does / does not apply, etc. • Likert items: stimuli for which respondent has to indicate the extent of agreement or support. E.g., response categories ranging from ‘disagree strongly’ to ‘agree strongly’. Number of