<br><h3> Chapter One </h3> <b>Comparative Survey Methodology <p> <i>Janet A. Harkness, Mickael Braun, Brad Edwards, Timothy P. Johnson, Lars Lyberg, Peter Ph. Muhler, Beth-Ellen Pennell, and Tom W. Smith</i> <p> <p> 1.1 INTRODUCTION</b> <p> This volume discusses methodological considerations for surveys that are deliberately designed for comparative research such as multinational surveys. As explained below, such surveys set out to develop instruments and possibly a number of the other components of the study specifically in order to collect data and compare findings from two or more populations. <p> As a number of chapters in this volume demonstrate, multinational survey research is typically (though not always) more complex and more complicated to undertake successfully than are within-country cross-cultural surveys. Many chapters focus on this more complicated case, discussing multinational projects such as the annual International Social Survey Programme (ISSP), the epidemiologic World Mental Health Initiative survey (WMH), the 41-country World Fertility Survey (WFS), or the triennial and worldwide scholastic assessment Programme for International Student Assessment (PISA). Examples of challenges and solutions presented in the volume are often drawn from such large projects. <p> At the same time, we expect many of the methodological features discussed here also to apply for within-country comparative research as well. Thus we envisage chapters discussing question design, pretesting, translation, adaptation, data collection, documentation, harmonization, quality frameworks, and analysis to provide much of importance for within-country comparative researchers as well as for those involved in cross-national studies. <p> This introductory chapter is organized as follows. Section 1.2 briefly treats the growth and standing of comparative surveys. Section 1.3 indicates overlaps between multinational, multilingual, multicultural, and multiregional survey research and distinguishes between comparative research and surveys deliberately designed for comparative purposes. Section 1.4 considers the special nature of comparative surveys, and Section 1.5 how comparability may drive design decisions. Section 1.6 considers recent changes in comparative survey research methods and practice. The final section, 1.7, considers ongoing challenges and the current outlook. <p> <p> <b>1.2 COMPARATIVE SURVEY RESEARCH: GROWTH AND STANDING</b> <p> Almost without exception, those writing about comparative survey research—whether from the perspective of marketing, the social, economic and behavioral sciences, policy-making, educational testing, or health research—remark upon its "rapid," "ongoing," or "burgeoning" growth. And in each decade since World War TI, a marked "wave" of interest in conducting cross-national and cross-cultural survey research can be noted in one discipline or another (see contributions in Bulmer, 1998; Bulmer & Warwick, 198311 993; Gauthier, 2002; Hantrais, 2009; Hantrais & Mangen, 2007; Øyen, 1990; and Chapters 2 and 25, this volume). <p> Within the short span of some 50 years, multipopulation survey research has become accepted as not only useful and desirable but, indeed, as indispensable. In as much as international institutions and organizations—such as the European Commission, the Organization for Economic Co-operation and Development (OECD), the United Nations (UN), the United Nations Educational, Scientific and Cultural Organization (UNESCO), the World Bank, the International Monetary Fund (IMF), and the World Health Organization (WHO)—depend on multinational data to inform numerous activities, it has become ubiquitous and, in some senses, also commonplace. <p> <p> <b>1.3 TERMINOLOGY AND TYPES OF RESEARCH</b> <p> In this section we make a distinction which is useful for the special methodological focus of many chapters in this volume—between comparative research in general and deliberately designed comparative surveys. <p> <p> <b>1.3.1 Multipopulation Surveys: Multilingual, Multicultural, Multinational, and Multiregional</b> <p> Multipopulation studies can be conducted in one language; but most multipopulation research is nonetheless also multilingual. At the same time, cultural differences exist between groups that share a first language both within a country (e.g., the Welsh, Scots, Northern Irish, and English in the United Kingdom) and across countries (e.g., French-speaking nations/populations). Language difference (Czech versus Slovakian, Russian versus Ukrainian) is, therefore, not a necessary prerequisite for cultural difference, but it is a likely indicator of cultural difference. <p> Within-country research can be multilingual, as reflected in national research conducted in countries as different as the Philippines, the United States, Switzerland, Nigeria, or in French-speaking countries in Africa. Cross-national projects may thus often need to address within-country differences in language and culture in addition to across-country differences, both with respect to instrument versions and norms of communication. <p> Multiregional research may be either within- or across-country research and the term is used flexibly. Cross-national multiregional research may group countries considered to "belong together" in some respect, such as geographical location (the countries of Meso and Latin America), in demographica1 features (high or low birth or death rates, rural or urban populations), in terms of developmental theory (see Chapter 4, this volume) or in terms of income variability. Other multiregional research might be intent on covering a variety of specific populations in different locations or on ensuring application in a multitude of regions and countries. Within-country multiregional research might compare differences among populations in terms of north-south, east-west or urban-rural divisions. <p> <p> <b>1.3.2 Comparative by Design</b> <p> This volume focuses on methodological considerations for surveys that are deliberate] y planned for comparative research. These are to be understood as projects that deliberately design their instruments and possibly other components of the survey in order to compare different populations and that collect data from two or more different populations. In 1969, Stein Rokkan commented on the rarity of "deliberately designed cross-national surveys" (p. 20). Comparative survey research has grown tremendously over the last four decades and is ubiquitous rather than rare. However, Rokkan's warning that these surveys are not "surefire investments" still holds me; the success of any comparative survey requires to be demonstrated and cannot be assumed simply on the basis of protocols or specifications followed. Numerous chapters in this volume address how best to construct and assess different aspects of surveys designed for comparative research. <p> Comparative instruments are manifold in format and purpose: educational or psychological tests, diagnostic instruments for health, sports performance, needs or usability assessment tools; social science attitudinal, opinion and behavioral questionnaires; and market research instruments to investigate preferences in such things as size, shape, color, or texture. Several chapters also present comparative methodological studies. <p> Comparative surveys are conducted in a wide variety of modes, can be longitudinal, can compare different populations across countries or within countries, and can be any mix of these. Some of the studies referred to in the volume are longitudinal in terms of populations studied (panels) or in terms of the contents of the research project (programs of replication). Most of the methodological discussion here, however, focuses on synchronic, across-population research rather than on across-time perspectives (but see Lynn, 2009; Duncan, Kalton, Kasprzyk, & Singh, 1989; Smith, 2005). <p> Comparative surveys may differ considerably in the extent to which the deliberate design includes such aspects as sampling, the data collection process, documentation, or harmonization. In some cases, the instrument is the main component "designed" to result in comparable data, while many other aspects are decided at the local level (e.g., mode, sample design, interviewer assignment, and contact protocols). Even when much is decided at the local level, those involved in the project must implicitly consider these decisions compatible with the comparative goals of the study. <p> If we examine a range of large-scale cross-national studies conducted in the last few decades (see, for example, Chapters 25-31, this volume), marked differences can also be found in study design and implementation. Studies vary greatly in the level of coordination and standardization across the phases of the survey life cycle, for example, in their transparency and documentation of methods, and in their data collection requirements and approaches. <p> <p> <b>1.33 Comparative Uses of National Data</b> <p> Comparative research (of populations and locations) need not be based on data derived from surveys deliberately designed for that purpose. <p> A large body of comparative research in official statistics, for instance, is carried out using data from national studies designed for domestic purposes which are then also used in analyses across samples/populations/countries. Early cross-national social science research often consisted of such comparisons (cf. Gauthier, 2002; Mohler & Johnson, this volume; Rokkan, 1969; Scheuch, 1973; Verba, 1969). Official statistics agencies working at national and international levels (UNESCO Statistics; the European statistical agency, Eurostat; and national statistical agencies such as the German Statistisches Bundesarnt and Statistics Canada) often utilize such national data for comparative purposes, as do agencies producing international data on labor force statistics (International Labour Organization; ILO), on income, wealth, and poverty (Luxembourg Income Study; LIS), and on employment status (Luxembourg Employment Study; LES). Such agencies harmonize data from national studies and other sources because adequately rich and reliable data from surveys that were deliberately designed to produce cross-national datasets are not available for many topics. The harmonization strategies used to render outputs from national data comparable (ex-post output harmonization) are deliberately designed for that purpose (see, for example, Ehling, 2003); it is the national surveys themselves which are not comparative by design. A partnership between Eurostat and many national statistical offices has resulted in the European Statistical System, an initiative which aims to provide reliable and comparable statistics for all the European Union and the European Free Trade Association Member States on the basis of national data. <p> <i>(Continues...)</i> <p> <p> <!-- copyright notice --> <br></pre> <blockquote><hr noshade size='1'><font size='-2'> Excerpted from <b>Survey Methods in Multicultural, Multinational, and Multiregional Contexts</b> Copyright © 2010 by John Wiley & Sons, Inc.. Excerpted by permission of John Wiley & Sons. All rights reserved. No part of this excerpt may be reproduced or reprinted without permission in writing from the publisher.<br>Excerpts are provided by Dial-A-Book Inc. solely for the personal use of visitors to this web site.