Design and use of questionnaires: a review of best practice applicable to surveys of health service staff and patients
Health Technology AssessmentPublished 1 January 2001Open access
Elaine McColl, Ann Jacoby, L Thomas, Jennifer Soutter, Claire Bamford, N Steen
Citations795
SJR quartileQ1
SJR score1.36
SNIP1.69
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
The aim is to gather valid, reliable, unbiased and discriminatory data from a representative sample of respondents, but the information yielded is subject to error and bias from a range of sources.
Abstract
T he NHS R&D Health Technology Assessment (HTA) Programme was set up in 1993 to ensure that high-quality research information on the costs, effectiveness and broader impact of health technologies is produced
Keywords
Economics, Econometrics and FinanceHealth Professions
PsychometrikaCoefficient Alpha and the Internal Structure of Tests
43,211 Citations1951Lee J. Cronbach
Psychological BulletinConvergent and discriminant validation by the multitrait-multimethod matrix.
17,061 Citations1959Donald T. Campbell, Donald W. Fiske
This transmutability of the validation matrix argues for the comparisons within the heteromethod block as the most generally relevant validation data, and illustrates the potential interchangeability of trait and method components.
Mail and internet surveys : the tailored design method
9,659 Citations2007Don A. Dillman
Health Measurement Scales: A Practical Guide to Their Development and Use
9,651 Citations1989David L. Streiner, Geoffrey R. Norman +1 more
This chapter discusses the design of items, their properties, and methods of administration, and some of the considerations that went into designing the items.
Contemporary Sociology A Journal of ReviewsMail and Telephone Surveys: The Total Design Method.
6,895 Citations1982Earl Babbie, Don A. Dillman
International Journal of ForecastingJournal of the American Statistical Association
5,362 Citations1995Anne B. Koehler
Journal of Marketing ResearchQuestionnaire Design, Interviewing and Attitude Measurement
5,007 Citations1993Gerald Albuam, A. N. Oppenheim
Journal of the American Statistical AssociationSurvey Research Methods.
4,799 Citations1985Susan Losh-Hesselbart, Floyd J. Fowler
Measuring Health: A Guide to Rating Scales and Questionnaires
4,234 Citations1987Ian McDowell, Claire Newell
The theory and the leading methods of measurement, all of which rely on subjective judgments in questionnaires and rating scales are described, showing readers how to select the most suitable one, apply it, and score the results.
Journal of Medical GeneticsInternational Journal of Epidemiology
2,900 Citations1973Cédric Mahé Pontiano Kaleebu
It is suggested that repeated somatic mutations lead to cellular loss and inefficiency and that this may be an important part of the ageing process and an important function of the T cell system is immunological surveillance aimed at detecting and destroying potentially dangerous mutants.
Journal of Clinical EpidemiologyResponse rates to mail surveys published in medical journals
2,098 Citations1997David A. Asch, M.Kathryn Jedrziewski +1 more
Although several mail survey techniques are associated with higher response rates, response rates to published mail surveys tend to be moderate, and investigators, journal editors, and readers should devote more attention to assessments of bias, and less to specific response rate thresholds.
Journal of Marketing ResearchHow to Conduct Your Own Survey
1,941 Citations1996Seymour Sudman, Priscilla Salant +1 more
Health Technology AssessmentConsensus development methods, and their use in clinical guideline development.
1,795 Citations1998Murphy Murphy, Jed Black +5 more
This assessment aims to identify the factors that affect the decisions that emerge from consensus development methods and to assess the implications of the findings for the development of clinical guidelines.
Research Methods in Health: Investigating Health and Health Services
1,650 Citations1997April Bowling
Thinking About Answers: The Application of Cognitive Processes to Survey Methodology
1,589 Citations1995Seymour Sudman, Norman M. Bradburn +1 more
Health Technology AssessmentEvaluating patient-based outcome measures for use in clinical trials.
1,545 Citations1998Fitzpatrick, Davey +2 more
This research highlights the need to understand more fully the rationale behind the continued use of these devices, as well as the barriers to their adoption.
The Incorporated StatisticianSurvey Methods in Social Investigation.
1,440 Citations1958C. G. T., C. A. Moser
Journal of the Royal Statistical Society Series A (Statistics in Society)Statistics with Confidence: Confidence Intervals and Statistical Guidelines.
1,331 Citations1990Flavia Jolliffe, Michael Gardner +1 more
This comprehensive collection of methods for using confidence intervals, illustrative worked examples and helpful checklists this is a truly practical guide for clinical readers to a fundamental aspect of medical statistics.
Public Opinion QuarterlyAsking Sensitive Questions: The Impact of Data Collection Mode, Question Format, and Question Context
1,317 Citations1996Roger Tourangeau, Tom Smith
International Journal of EpidemiologyJournal of Epidemiology and Community Health
1,286 Citations1982
Journal of Clinical EpidemiologyPublication bias in meta-analysis its causes and consequences
1,254 Citations2000Alison J. Thornton
The design of the meta-analysis itself, and the studies included in it, are shown to be important among a number of sources of publication bias.
Improving Survey Questions: Design and Evaluation
1,140 Citations1995Floyd J. Fowler
Measuring health: a review of quality of life measurement scales
1,032 Citations1991Ann Bowling
The new edition of Measuring Health includes a new chapter on measuring global quality of life; updated analysis of measures of subjective well-being; and a revised and up-to-date selection of useful addresses.
American Sociological ReviewFactors Affecting Response Rates to Mailed Questionnaires: A Quantitative Analysis of the Published Literature
1,005 Citations1978Thomas A. Heberlein, Robert Baumgartner
Health Technology AssessmentQualitative research methods in health technology assessment: a review of the literature.
901 Citations1998Murphy Murphy, Robert Dingwall +3 more
Case studies are not necessarily restricted in scope and general concepts can be formulated, which may, upon further investigation, be found to be germane to a wider variety of settings.
Journal of Marketing ResearchAnswering Questions: Methodology for Determining Cognitive and Communicative Processes in Survey Research
697 Citations1996David W. Stewart, Norbert Schwarz +1 more
Medical CareComparisons of the Costs and Quality of Norms for the SF-36 Health Survey Collected by Mail Versus Telephone Interview: Results From a National Survey
618 Citations1994Colleen A. McHorney, Mark Kosinski +1 more
Norms for mail and telephone versions of the SF-36 survey are provided for use in interpreting individual and group scores, and reports of chronic conditions were more frequent, for mail than telephone respondents.
Recent research in psychologyResponse Effects in Surveys
523 Citations1987Hans‐J. Hippler, Norbert Schwarz
How to Conduct Self-Administered and Mail Surveys
457 Citations2003Linda B. Bourque, Eve P. Fielder
This book provides readers with an answer to this question while giving them all the basic tools needed for conducting a self-administered or mail survey.
Public Opinion QuarterlyInterview Mode Effects in Surveys of Drug and Alcohol Use: A Field Experiment
455 Citations1994William S. Aquilino
Health Technology AssessmentPublication and related biases.
409 Citations2000Song, Eastwood +3 more
This review aims to identify systematically and appraise studies that have examined methodological issues and provide empirical evidence about publication bias and other dissemination-related biases, including biases due to the time, type and language of publication, multiple publication, selective citation of references, database index bias, and biased media attention.
Public Opinion QuarterlyStimulating Responses to Mailed Questionnaires: A Review
401 Citations1975Arnold S. Linsky
Journal of Consumer ResearchCognitive Processes Used by Survey Respondents to Answer Behavioral Frequency Questions
339 Citations1987Edward Blair, Scot Burton
Medical Entomology and ZoologyMail Surveys: Improving the Quality
322 Citations1995Thomas W. Mangione
The Basics of Question Design Question Design The Advanced Course Sampling Pitfalls in Sampling The Basics of Avoiding Nonresponse Errors Additional Ways to reduce nonresponse errors.
Educational and Psychological MeasurementControlling Acquiescence Response Bias by Item Reversals: The Effect on Questionnaire Validity
321 Citations1981Chester A. Schriesheim, Kenneth D. Hill
BMJQuality of life measures in health care. II: Design, analysis, and interpretation.
319 Citations1992A. E. Fletcher, S M Gore +4 more
In trials in hypertensive patients dimensions need to be chosen that reflect potential adverse effects of treatment such as poorer work performance, problems with sexual function, and deleterious effects on mood.
Public Opinion QuarterlyEffects of Questionnaire Length, Respondent-Friendly Design, and a Difficult Question on Response Rates for Occupant-Addressed Census Mail Surveys
304 Citations1993Don A. Dillman, Michael Sinclair +1 more
GutStatistics with confidence - confidence intervals and statistical guidelines
304 Citations1989D. L. Wingate
A study of the antisccrctory profile of SKF 94482, a new H2 rcccptor antagonist which has a profound cffcct on daytimc acidity, and the role of sampling ratc in the characterization and analysis of 24-hour gastric acidity.
Measures of Need and Outcome for Primary Health Care
291 Citations1992David Wilkin, L Hallam +1 more
This work defines need and outcome, methods of measurement, and measures of functioning for mental illness and mental health, and multi-dimensional measures for disease - specific measures and social support are presented.
BMJGeneral practice postal surveys: a questionnaire too far?
273 Citations1996Brian McAvoy, Eileen Kaner
Approaches that might reverse this trend include monitoring all research activities going on in an area to ensure that practices are not overused, giving general practitioners incentives to participate, and improving the relevance of research and the quality of questionnaires.
Sage eBooksHow to conduct interviews by telephone and in person
261 Citations1995James H. Frey, Sabine Oishi
In a collapsible junction support bracket, a series of frameworks, vertically disposed, are pivotally mounted to each other through the agency of aseries of side stringers, and braces are removable to provide for collapsing of the bracket to facilitate its shipment.
Public Opinion QuarterlyOpinions on Fictitious Issues: The Pressure to Answer Survey Questions
239 Citations1986George F. Bishop, Alfred J. Tuchfarber +1 more
Public Opinion QuarterlyTask Conditions, Response Formulation Processes, and Response Accuracy for Behavioral Frequency Questions in Surveys
232 Citations1991Scot Burton, Edward Blair
Public Opinion QuarterlyWhite Respondents and Race-of-Interviewer Effects
229 Citations1975Shirley Hatchett, Howard Schuman
American Journal of Public HealthTelephone vs face-to-face interviewing in a community psychiatric survey.
202 Citations1993Shmuel Fenig, I. Levav +2 more
Telephone interviewing seems to be a reliable and efficient method in areas with a well-developed network of subscribers and face-to-face interviewing in a community psychiatric survey.
Educational and Psychological MeasurementThe Effect of Negation and Polar Opposite Item Reversals on Questionnaire Reliability and Validity: An Experimental Investigation
197 Citations1991Chester A. Schriesheim, Regina J. Eisenbach +1 more
Wiley series in probability and statisticsTowards a Theory of Self‐Administered Questionnaire Design
190 Citations1997Cleo R. Jenkins, Don A. Dillman
Public Opinion QuarterlyExperiments With the Middle Response Alternative in Survey Questions
173 Citations1987George F. Bishop
Public Opinion QuarterlyCarryover Effects in Attitude Surveys
169 Citations1989Roger Tourangeau, Kenneth A. Rasinski +2 more
American Journal of EpidemiologyVALIDATION OF A RETROSPECTIVE QUESTIONNAIRE ASSESSING DIET 10–15 YEARS AGO
167 Citations1989Janet L. Sobell, Gladys Block +3 more
In 1985, 216 men, participants in the Baltimore Longitudinal Study of Aging, were asked to report on their usual diet during the period 1971-1975 and correlations with past diet were approximately as good as those obtainable with present diet in other studies.
Public Opinion QuarterlyResponse Rates and Response Content in Mail Versus Face-To-Face Surveys
145 Citations1994Maria Krysan, Howard Schuman +2 more
BMJProfessionals as responders: variations in and effects of response rates to questionnaires, 1961-77.
144 Citations1978Anna Cartwright
General conclusions about the representativeness of the respondents when compared with basic data on those who did not respond are that younger better qualified doctors were more likely to respond and among general practioners those who worked in groups were morelikely to respond.
American Journal of Public HealthIncreasing response rates in physicians' mail surveys: an experimental study.
142 Citations1989Brigitte Maheux, Claudine Legault +1 more
It is becoming increasingly difficult to obtain high response rates in physicians' mail surveys, and a handwritten thank you note at the bottom of the letter accompanying the questionnaire and a more personalized mailout package increased response rates.
Public Opinion QuarterlyResponse Styles in Telephone and Household Interviewing: A Field Experiment
140 Citations1980Lawrence A. Jordan, Alfred C. Marcus +1 more
Public Opinion QuarterlyContext Effects on Survey Responses to Questions About Abortion
140 Citations1981Howard Schuman, Stanley Presser +1 more
Public Opinion QuarterlyImplications of Nonresponse for the Interpretation of Mail Questionnaire Data
136 Citations1960Marjorie N. Donald
Sociological Methods & ResearchEvaluating Race-of-Interviewer Effects In a National Survey
134 Citations1980Nora Cate Schaeffer
ePrints@IISc (Indian Institute of Science)PENGARUH MODEL DISCOVERY LEARNING BERBANTUANAPLIKASI WORDWALL TERHADAP HASIL BELAJAR IPASKELAS IV SEKOLAH DASAR
133 Citations2025J. Scott Armstrong
American Sociological ReviewFurther Evidence on Factors Affecting Response Rates to Mailed Questionnaires
130 Citations1982John Goyder
Public Opinion QuarterlyResponse Effects in Mail Surveys
129 Citations1990Stephen A. Ayidiya, McKee J. McClendon
Journal of Experimental Social PsychologyBelief accessibility and context effects in attitude measurement
128 Citations1989Roger Tourangeau, Kenneth A. Rasinski +2 more
Journal of Marketing ResearchCarryover and Backfire Effects in Marketing Research
123 Citations1993Barbara Bickart
Evaluation ReviewMail Surveys of Reluctant Professionals
111 Citations1985Seymour Sudman
The procedures deal with improving the cost-benefit ratio of cooperation for the busy professional, being sensitive to confidentiality concerns, and allowing the respondent a full range of answers, including written comments, instead of an answer.
American Journal of Public HealthTelephone versus in-person surveys of community health status.
104 Citations1982Carol S. Aneshensel, R R Frerichs +2 more
No statistically significant differences were found between the two interview methods for overall assessment of health status, illnesses reported for the previous four months, or reports of hospitalization in Los Angeles County adults.
Journal of Business and PsychologyBelieve it or not! longer questionnaires have lower response rates
104 Citations1990Michael J. Roszkowski, Andrew G. D. Bean
Public Opinion QuarterlySubsequent Questions May Influence Answers to Preceding Questions in Mail Surveys
101 Citations1995Norbert Schwarz, Hans‐J. Hippler
Public Opinion QuarterlyDo Respondents Really "Mark All That Apply" On Self-Administered Questions?
101 Citations1994Kenneth A. Rasinski, David J. Mingay +1 more
Public Opinion QuarterlyEthnicity-of-Interviewer Effects on Ethnic Respondents
97 Citations1981Michael F. Weeks, Richard P. Moore
Social Science Open Access Repository (GESIS – Leibniz Institute for the Social Sciences)A comparison of response effects in self-administered and telephone surveys
91 Citations1987George E. Bishop, Hans‐Jürgen Hippler +2 more
American Journal of Public HealthSelf-reports of HIV risk factors by patients at a sexually transmitted disease clinic: audio vs written questionnaires.
90 Citations1994Bradley O. Boekeloo, Luigi Schiavo +4 more
audio questionnaires may obtain more complete data and identify more HIV risk than written questionnaires and face-to-face interviews, and research is warranted about whether audio questionnaires overcome barriers to the completion and accuracy of HIV risk surveys.
The Journal of Experimental EducationResponse Rates in Survey Research: A Meta-Analysis of the Effects of Monetary Gratuities
90 Citations1992Kenneth D. Hopkins, Arlen R. Gullickson
Journal of Marketing ResearchA Reassessment of the Effects of Appeals on Response to Mail Surveys
90 Citations1980Terry L. Childers, William M. Pride +1 more
Journal of Marketing ResearchMonetary Incentives versus Promised Contribution to Charity: New Evidence on Mail Survey Response
89 Citations1982David H. Furse, David W. Stewart
Public Opinion QuarterlyEffects of Filter Questions in Public Opinion Surveys
82 Citations1983George F. Bishop, Robert W. Oldendick +1 more
Journal of Marketing ResearchAn Evaluation of the Effectiveness of Drop-Off Questionnaire Delivery
81 Citations1976Christopher Lovelock, Ronald Stiff +2 more
For lengthy questionnaires, personal delivery by lightly trained survey takers appears to yield higher response rates than mail surveys at competitive costs and provides for more precisely controlled samples and clearer identification of the nature of nonresponse bias.
Public Opinion QuarterlyAsking Comparative Questions: The Impact of the Direction of Comparison
79 Citations1995Michaela Wänke, Norbert Schwarz +1 more
Public Opinion QuarterlyEffects of Benefits Appeals, Mandatory Appeals, and Variations in Statements of Confidentiality on Completion Rates for Census Questionnaires
78 Citations1996Don A. Dillman, Eleanor Singer +2 more
Public Opinion QuarterlyComputer Assisted Telephone Interviewing: Effects on Interviewers and Respondents
74 Citations1984Robert M. Groves, Nancy A. Mathiowetz
There were few differences in response distributions obtained in the two modes, some evidence of reduced interviewer variance among CATI cases, but little difference in response rates between the methods.
PubMedNonresponse bias to mail survey questionnaires within a professional population.
74 Citations1980Hovland Ej, Elaine Romberg +1 more
Public Opinion QuarterlyPersonalizing Mail Questionnaire Correspondence
73 Citations1970Alan R. Andreasen
Journal of Marketing ResearchResponse Rates and Perceived Questionnaire Length in Mail Surveys
71 Citations1979Terry L. Childers, O. C. Ferrell
Computers in Human BehaviorResponses on computer surveys: Impression management, social desirability, and the big brother syndrome
71 Citations1996Paul Rosenfeld, Stephanie Booth‐Kewley +2 more
It was concluded that perceiving that one's responses are linked to a larger database may lead to more impression management on computer surveys.
American Journal of Public HealthEnhancing the self-report of alcohol consumption in the community: two questionnaire formats.
70 Citations1994Abby C. King
The results suggest that food frequency questionnaires may provide a better means for enhancing self-report of alcohol use than questionnaires that target solely alcohol intake.
Public Opinion QuarterlyPromised Contributions to Charity and Mail Survey Responses: Replication With Extension
64 Citations1988Raymond Hubbard, Eldon L. Little
Journal of Marketing ResearchTesting the Effectiveness of Alternative Foot-in-the-Door Manipulations
64 Citations1980Robert A. Hansen, Larry M. Robinson
The study compares the effectiveness of high and low involvement foot manipulations in generating compliance with both short and long forms of a mail questionnaire and finds that the high and the low involvementFoot manipulation generate significantly higher response rates and faster response times than a noncontact control situation.
Research in Higher EducationSolving the quandary between questionnaire length and response rate in educational research
63 Citations1982L. La Mar Adams, Darwin Gale
Public Opinion Quarterly"Secret Ballots" and Self-Reports in an Exit-Poll Experiment
61 Citations1995George F. Bishop, Bonnie S. Fisher
Psychosomatic MedicineAssessment of the Reliability of and the Effect of Neutral Instructions on the Symptom Ratings on the Moos Menstrual Distress Questionnaire
59 Citations1976Rosemary A. Markum
The Moos Menstrual Distress Questionnaire, Form T, was analyzed for split‐half and test‐retest reliability and it was suggested that Form T be used instead as originally intended to compare a single S′ ratings across the different phases of her menstrual cycle rather than to compare the ratings of different Ss in different cycle phases.
Public Opinion QuarterlyThe Perceived Threat of Nuclear War, Salience, and Open Questions
59 Citations1986Howard Schuman, Jacob R. Ludwig +1 more
Psychological ReportsPreconditions for Sensitivity in Measuring Change: Visual Analogue Scales Compared to Rating Scales in a Likert Format
59 Citations1995Lilian Pfennings, Leo Cohen +1 more
Findings provide some support for the preferred use of analogue scales over discrete response categories in the measurement of changes over time in first-year medical students.
American Journal of Public HealthTelephone health surveys: potential bias from noncompletion.
58 Citations1993Shiraz I. Mishra, Deborah A. Dooley +2 more
Public Opinion Quarterly"Don't Know" Boxes in Factual Questions in a Mail Questionnaire: Effects on Level and Quality of Response
58 Citations1988Gail S. Poe, Isadore Seeman +3 more
EpidemiologyEffects of Mailing Strategies on Response Rate, Response Time, and Cost in a Questionnaire Study Among Nurses
57 Citations1990Bernard C. K. Choi, Anita W. P. Pak +1 more
It is found that paper stamps, especially large commemorative stamps, on return envelopes increased the response rate and reduced the response time, as compared with other mailing strategies.
Psychology and MarketingThe interactive effects of monetary incentive justification and questionnaire length on mail survey response rates
57 Citations1994Paul M. Biner, Heath J. Kidd
Journal of Dental EducationNonresponse bias to mail survey questionnaires within a professional population
56 Citations1980Hovland Ej, E Romberg +1 more
It was concluded that subject matter does affect dentist response rates to mailed survey questionnaires, there are no differences in demographic data between dentist responders and nonresponders, and nonresponse bias does not affect the results of dental surveys on a typical dentist response rate.
…