Description of Courses

A Virtually Syntax Free Practical Introduction to Web Scraping for Survey and Social Science Researchers

SurvMeth 988.400 (.5 credit hours) to take this class for UM credit you must take SurvMeth 988.400 and 988.500, An Introduction to Big Data and Machine Learning for Survey Researchers and Social Scientists for a total of 1.0 credit hours.

Instructor: Trent D. Buskirk

This short course will offer a very practical introduction to web scraping geared at social scientists and survey researchers.  This course begins with an overview of web scraping discussing some basic technical jargon, types of web data and various methods for scraping.  Some websites are designed to be easily accessible by web crawlers or scraping algorithms while others require much more advanced, custom programming.  In this course we will illustrate how participants can discern these differences as well as presenting several motivating examples of the various ways web scraped data can be used throughout a study’s lifecycle from design to calibration to analysis.  We provide an extensive introduction to a suite of freeware programs that allow virtually syntax free, but customizable, web scraping capabilities.  The course concludes with specific focus on the tool where we demonstrate its capabilities and provide several, hands-on practical examples for participants to begin scraping several websites of increasing complexity.

If you wish to take this course for academic credit you must also enroll in An Introduction to Big Data and Machine Learning for Survey Researchers and Social Scientists.

Prerequisite: Having a trial account set up (this is a 7 day trial so please plan to have the license active during our course).  Details can be found here:

2019 Syllabus (PDF)


An Introduction to Big Data and Machine Learning for Survey Researchers and Social Scientists

SurvMeth 988.500 (.5 credit hours) to take this class for UM credit you must take SurvMeth 988.500 and 988.400, A Virtually Syntax Free Practical Introduction to Web Scraping for Survey and Social Science Researchers for a total of 1.0 credit hours.

Instructor: Trent D. Buskirk

The amount of data generated as a by-product in society is growing fast including data from satellites, sensors, transactions, social media and smartphones, just to name a few. Such data are often referred to as "big data", and can be used to create value in different areas such as health and crime prevention, commerce and fraud detection.  An emerging practice in many areas is to append or link big data sources with more specific and smaller scale sources that often contain much more limited information.  This practice has been used for some time by survey researchers in constructing frames by appending auxiliary information that is often not directly available on the frame, but can be obtained from an external source.   Using Big Data has the potential to go beyond the sampling phase for survey researchers and in fact has the potential to influence the social sciences in general.  Big Data is of interest for public opinion researchers and agencies that produce statistics to find alternative data sources either to reduce costs, to improve estimates or to produce estimates in a more timely fashion. However, Big Data pose several interesting and new challenges to survey researchers and social scientists among others who want to extract information from data. As Robert Groves (2012) pointedly commented, the era is “appropriately called Big Data and not Big Information”, because there is a lot of work for analysts before information can be gained from “auxiliary traces of some process that is going on in society.”  

This course offers participants a broad overview of big data sources, opportunities and examples motivated within the survey and social science contexts including the use of social media data, para data and other such sources.  This course also offers a detailed, practical introduction to four common machine learning methods that can be applied to big and small data alike at various aspects of a study’s lifecycle from design to nonresponse adjustments to propensity score matching to weighting and evaluation and analysis.  The machine learning methods will be demonstrated in R and we will provide several different examples of using these methods along with multiple packages in R that offer these methods. 

If you wish to take this course for academic credit you must also enroll in A Virtually Syntax Free Practical Introduction to Web Scrapping for Survey and Social Science Researchers.

Prerequisite: Basic proficency in R (i.e. how to load a package, launch it and basic R syntax knowledge)

2019 Syllabus (PDF)


Analysis Methods for Complex Sample Survey Data

SurvMeth 614 (3 credit hours)

Instructor: Yajuan Si, University of Michigan

This course provides an introduction to specialized software procedures that have been developed for the analysis of complex sample survey data. The course begins by considering the sampling designs of specific surveys: the National Comorbidity Survey-Replication (NCS-R), the National Health and Nutrition Examination Surveys (NHANES), and the Health and Retirement Study (HRS). Relevant design features of the NCS-R, NHANES and HRS include weights that take into account differences in probability of selection into the sample and differences in response rates, as well as stratification and clustering in the multistage sampling procedures used in identifying the sampled households and individuals.

Prerequisite: Two graduate-level courses in statistical methods, familiarity with basic sample design concepts, and familiarity with data analytic techniques such as linear and logistic regression.

Why take this course? 

  • To gain an understanding of modern methods and software for the secondary analysis of survey data collected from large complex samples
  • To have the opportunity for one-on-one interaction with the instructors when walking through analyses of survey data
  • To see several examples of applied statistical analyses of survey data
  • To have the experience of writing a scientific paper that presents an analysis of complex sample survey data, and getting expert feedback on that paper

2019 Syllabus (PDF)



Applied Sampling/Methods of Survey Sampling

SurvMeth 625 (3 credit hours)

Instructor: James Wagner, University of Michigan and Raphael Nishimura, University of Michigan

A fundamental feature of many sample surveys is a probability sample of subjects. Probability sampling requires rigorous application of mathematical principles to the selection process. Methods of Survey Sampling is a moderately advanced course in applied statistics, with an emphasis on the practical problems of sample design, which provides students with an understanding of principles and practice in skills required to select subjects and analyze sample data. Topics covered include stratified, clustered, systematic, and multi-stage sample designs, unequal probabilities and probabilities proportional to size, area and telephone sampling, ratio means, sampling errors, frame problems, cost factors, and practical designs and procedures. Emphasis is on practical considerations rather than on theoretical derivations, although understanding of principles requires review of statistical results for sample surveys. The course includes an exercise that integrates the different techniques into a comprehensive sample design.

Prerequisite: Two graduate-level courses in statistical methods.

2018 Syllabus (PDF)



Data Collection Methods 

SurvMeth 623 (3 credit hours, remote participation option available)

Instructors: Frederick Conrad, University of Michigan and Florian Keusch, University of Mannheim

This course reviews a range of survey data collection methods that are both interview-based (face-to-face and telephone) and self-administered (questionnaires that are mailed and online, i.e., web surveys). Mixed mode designs are also covered as well as several hybrid modes for collecting sensitive information e.g., self-administering the sensitive questions in what is otherwise a face-to-face interview. The course also covers newer methods such as mobile web and SMS (text message) interviews, and examines alternative data sources such as social media. It concentrates on the impact these techniques have on the quality of survey data, including error from measurement, nonresponse, and coverage, and assesses the tradeoffs between these error sources when researchers choose a mode or survey design. This is not a how-to-do-it course on survey data collection, but rather focuses on the error properties of key aspects of the data collection process.

Students will view recorded lectures and complete reading assignments in preparation for class discussion sessions which will occur twice per week, one hour per session.  Students are expected to attend all discussion sessions either in person or via BlueJeans.  Successful discussion sessions will occur through preparation and active participation by all participants enrolled in the course.  Students should have questions or discussion topics in mind for the class sessions.

Remote participation option:  It is not necessary to be physically in Ann Arbor to take the course.  Students who cannot be in Ann Arbor can enroll and join discussion sessions via  BlueJeans (

Once enrollment is confirmed via email, indicate if course attendance will be in person, in Ann Arbor or via BlueJeans.

Prerequisite: An introductory course in survey research methods or equivalent experience.  If joining remotely, participant must have computer, camera and headset available to join the class via BlueJeans (

2018 Syllabus (PDF)



Data Collection Using Wearables, Sensors, and Apps in the Social, Behavioral, and Health Sciences

SurvMeth 988.400 (1 credit hour)

Instructors: Heidi Guyer, University of Michigan and Florian Keusch, University of Mannheim

The recent proliferation of new mobile technology allows researchers to collect objective health and behavioral data at increased intervals, in real time, and may also reduce participant burden. In this course, we will provide examples of the utility of and integration of wearables, sensors, and apps in research settings. Examples will include the use of wearable health devices to measure activity, apps for ecological momentary assessment, and smartphone sensors to measure sound and movement, among others. Additionally, this course will consider the integration of these new technologies into existing surveys and the quality of the data collected from the total survey error perspective. We will discuss considerations for assessing coverage, participation, and measurement error when integrating wearables, sensors, and apps in a research setting as well as the costs and privacy considerations when collecting these types of data. Participants will work in groups to discuss a research study design using new technology and have the opportunity for hands-on practice with an activity tracker dataset.

Prerequisite: you must have your own laptop to participate in this class. 


2019 Syllabus (PDF)


Introduction to Data Collection Methods

SurvMeth 988.225 (1 credit hour)

Instructors: Florian Keusch, University of Mannheim

This 2-day workshop will introduce students to different methods of collecting data in the social sciences. Surveys are the most common form of collecting primary data in many disciplines, and this course will provide students with an overview of interview-administered (face-to-face and telephone) and self-administered (mail, web, mobile web, and SMS) survey data collection as well as the combination of multiple modes (mixed mode surveys). The course will in particular discuss the implication of survey design decisions on data quality. In addition, students will also receive an overview on alternative data sources (e.g., passive measurement, social media and administrative data) and how they can be used in combination with traditional survey data.

2019 Syllabus (PDF)



Introduction to the Health and Retirement Study (HRS) Workshop

Location: Institute for Social Research, Room 1070

Dates:  June 8-12, 2020

Time:  9:00am-4:00pm, Monday-Friday

Not for credit

Instructors: Amanda Sonnega, University of Michigan

The Health and Retirement Study ( Summer Workshop is intended to give participants an introduction to the study that will enable them to use the data for research. HRS is a large-scale longitudinal study with more than 20 years of data on the labor force participation and health transitions that individuals undergo toward the end of their work lives and in the years that follow. The HRS Summer Workshop features morning lectures on basic survey content, sample design, weighting, and restricted data files. Hands-on data workshops are held every afternoon in which participants learn to work with the data (including the user-friendly RAND version of the HRS data) under the guidance of HRS staff. Staff of the Gateway to Global Aging project (, which harmonizes data across HRS international sister studies, conduct an afternoon training. At the end of the week, students have the opportunity to present their research ideas to the class and HRS research faculty and obtain feedback. Topics include (but are not limited to) in depth information on HRS data about health insurance and medical care; biomarkers, physical measures, and genetic data; cognition; health and physical functioning; linkage to Medicare; employment, retirement, and pensions and linkage to Social Security records; psychosocial and well-being; family data; and international comparison data. The data training portion assumes some familiarity with SAS or STATA.

2018 Syllabus (PDF)



Introduction to Focus Group Interviewing Research Methods

SurvMeth 652 (1 credit hour)

Instructors: Richard Krueger and Mary Anne Casey, University of Minnesota


This course introduces the skills needed to conduct focus group interviews. Students will learn about the critical components of successful focus group research. They will develop a plan for a focus group study and then practice key skills. Attention will be placed on moderating, recruiting, developing questions, and analysis of focus groups. This course will be particularly applicable for those conducting focus group research in academic, non-profit, and government settings.

Course Topics:

The course will cover these skills:

  • Planning—When to use focus groups and designing a study
  • Recruiting—Identifying information-rich participants and getting them to show up
  • Hosting—Creating a permissive and nonthreatening environment
  • Moderating—The crucial first few minutes and moderating techniques
  • Developing questions—Characteristics of good focus group questions
  • Analyzing—Options for analysis
  • Reporting—Options for sharing what was learned

Course Format

The course format includes daily lectures along with opportunities to practice critical skills in small groups.

Why Take This Course?

Focus groups are used to understand issues, pilot test ideas, and evaluate programs. They also provide great insight when used in combination with surveys. Focus groups have been used to help design surveys, to pilot test surveys, and to understand survey findings. Take this course if you want to learn more about how focus groups might add to your research toolbox.

Prerequisite: An introductory course in research methods or equivalent experience.

2019 Syllabus (PDF)



Introduction to Applied Questionnaire Design

SurvMeth 988.206 (1 credit hour)

Instructor: Nora Cate Schaeffer, University of Wisconsin-Madison

This course provides students with practice applying principles of question design. Students leave the course with tools to use in diagnosing problems in survey questions and writing their own survey questions.  The lecture provides guidelines for writing and revising survey questions and using troubled questions from surveys as examples for revision. Each day's session combines lecture with group discussion and analysis.  For some class activities, students work in small groups to apply lecture material to identify problems in the survey questions and propose solutions. Assignments require that students write new questions or revise problematic questions and administer them to fellow students. Sessions consider both questions about events and behaviors and questions about subjective phenomena (such as attitudes, evaluations, and internal states).

20 Ways to Test Your Survey Questions is a course that complements well with this class.

2018 Syllabus (PDF)



Introduction to Questionnaire Design

SurvMeth 988.223 (1 credit hour)

Instructor:  Jessica Broome, Jessica Broome Research

This course provides an overview of the art and science of questionnaire design. Topics will include basic principles of questionnaire design; factual and non-factual questions; techniques for asking about sensitive topics; designing scales and response options; survey mode considerations; and an introduction to pre-testing surveys. The course will consist of both lectures and hands-on activities.

2018 Syllabus (PDF)



Introduction to Survey Methodology

SurvMeth 988.208 (1 credit hour)

Instructor: Emilia Peytcheva, RTI International

This 2-day course will introduce participants to the basic principles of survey design, presented within the Total Survey Error framework.  The course provides an introduction to the skills and resources needed to design and conduct a survey, covering topics such as sampling frames and designs, mode of data collection and their impact on survey estimates, cognitive processes involved in answering survey questions, best questionnaire design practices, and pretesting methods.

2018 Syllabus (PDF)



Introduction to Survey Research Techniques

SurvMeth 988.229 (2 credit hours)

Instructor: Tuba Suzer-Gurtekin, University of Michigan

Surveys continue to play an important role in addressing many kinds of problems about many kinds of populations stand alone or as part of an integrated information system.  Application of the scientific principles underlying surveys depends on good understanding of theories and empirical research from disciplines such as psychology, sociology, statistics and computer science.  A set of principles and empirical research will be introduced through the Total Surevy Error (TSE) framework.  The principles include problem and hypothesis formulation, study design, sampling, questionnaire design, interviewing techniques, pretesting, modes of data collection and data cleaning, management, and analysis.  Students will be trained to determine major steps in data collection design and implementation and to refer to literature to justify the steps.  The course will also discuss team and project management in the content of survey research, identifying skillsets and technical language required.  The course will also provide training in an important subset of skills needed to conduct a survey form beginning to end. 

2018 Syllabus (PDF)



Introduction to Survey Sampling

SurvMeth 988.219 (1 credit hour)

Instructor: Sunghee Lee, University of Michigan

This is a foundation course in sample survey methods and principles.  The instructors will present, in a non-technical manner, basic sampling techniques such as simple random sampling, systematic sampling, stratification, and cluster sampling.  The instructors will provide opportunities to implement sampling techniques in a series of exercises that accompany each topic. 

Participants should not expect to obtain sufficient background in this course to master survey sampling.  They can expect to become familiar with basic techniques well enough to converse with sampling statisticians more easily about sample design.



Mixed Methods Research Design and Data Collection Series:

SurvMeth 653 (1.5 credit hour) to take this class for UM credit you must take SurvMeth 653 Mixed Methods Research Design and Data Collection Series for a total of 1.5 credit hours.

Mixed Methods Research Design and Data Collection and Analysis

SurvMeth 653.001 (1 credit hour)

Instructors:  William Axinn, University of Michigan and Emily Treleaven, University of Michigan

This intensive course reviews mixed method research design principles, data collection strategies for combining multiple methods within a single research project, and strategies for analyzing data from multiple sources in a single study. We summarize the mixed method research process from the development of the research question to analysis and presentation of results. The methods examined include unstructured or in-depth interviews, semi-structured interviews, focus groups, survey interviews, observation, geographic information systems, archival research, social media and big/organic data analysis, and hybrid methods. Emphasis will be placed on the specific contribution of each method, as well as the integration of multiple alternative methods to advance specific research questions. This course is designed for those with a specific research question in mind – participants will be asked to design multi-method approaches to address a research question of their choice. By the end of this course, participants will have an overview of multi-method research that will enable them to design, understand, and evaluate multi-method approaches within a single project.

Prerequisite: An introductory course in survey research methods or equivalent experience.

Event History Calendar Data Collection Methods

SurvMeth 988.653 (.5 credit hours) to take this class for UM credit you must also take SurvMeth 653.001 Mixed Method Research Design, Data Collection and Analysis for a total of 1.5 credit hours.

Instructors: Dirgha Jibi Ghimire, University of Michigan and Heather Gatny, University of Michigan

This intensive course will offer an introduction to history calendar data collection methods. The course will consider both the context of longitudinal research designs and pure retrospective designs.  Methods will include approaches for the collection of dynamic measures of community, household and individual-level change over time. This course begins with an overview of principals of history calendar methods, integration of structured and semi structured methods of data collection techniques and types of data generated through the calendar methods.  In this course we will illustrate principals of calendar method with actual examples.  The course focuses on how to develop and implement calendars as data collection tools and encourages participants to explore critical issues concerning the applications of these methods.


2019 Syllabus (PDF)



Multi-Item Scale Development and Testing  

SurvMeth 988.220 (1 credit hour)

Instructors: Pamela Campanelli, UK Survey Methods Consultant 

Standardized multi-item scales are more common in some disciplines than others. This  course is designed to inspire participants from all disciplines that it is possible to develop your own high quality multi-item scales (or correctly adapt existing multi-item scales) and offers an introduction on how to do this. It covers the psychometric principles of question development while adding in principles of general questionnaire design. Focusing first on Classical Measurement Theory, participants design their own multi-item scales. This is followed by a group discussion of existing multi-item scales. The course then introduces some basic statistical tools for assessing the reliability and dimensionality of multi-item scales and participants get to practice evaluating some existing scales in a computer lab session. The course finishes with an introduction to Item Response Theory.

Prerequisite: There is no prerequisite, but a little knowledge about questionnaire design, multi-item scales and SPSS would be of value.

2019 Syllabus (PDF)



Probability and Non-Probability Sampling Methods

SurvMeth 988.224 (2 credit hours)  *Remote participation optionIt is not necessary to be physically in Ann Arbor to participate in these workshops. Students who cannot be in Ann Arbor can enroll and join sessions via  BlueJeans (  Once enrollment is confirmed via email, indicate if course attendance will be in person, in Ann Arbor or via BlueJeans

Instructors:  Jim Lepkowski and Sunghee Lee, University of Michigan

Probability and Non-probability Sampling Methods is a sampling course that differs from traditional sampling classes. First, this class gives an equal amount of attention to both probability and non-probability sampling methods as non-probability sampling cannot be discussed meaningfully without understanding probability sampling and these two methods offer distinctive advantages and disadvantages. Second, this class will combine theoretical/conceptual parts of sampling through lectures and practical applications of different approaches through lab sessions.

The course will start with examining probability sampling techniques and their properties, including simple random selection, systematic selection, cluster sampling, stratified sampling, and probability proportionate to size selection. Issues of weighting to compensate for unequal chances of selection and variance estimation for calculating confidence intervals are also examined.  Then the wide variety of non-probability sampling methods are examined, from panel-based convenience samples, to river samples, quota samples, respondent-driven samples, and other techniques. The properties of these samples are discussed, and assumptions needed to obtain estimates are examined. We will also examine these two approaches from the total survey error perspectives.

The lab sessions to be held after each class will combine R programming and group discussions on the topics that need to be considered when implementing various sampling approaches. Hands-on examples of frame preparation, sample draws, post-survey adjustments and analysis specific to design will be provided and discussed.

The course is not designed to provide the mastery of survey sampling. Rather, it provides materials that will accommodate participants to become familiar with advantages and disadvantages of the two methods and their implementation which will allow them to make informed design decisions.

2018 Syllabus (PDF)



Qualitative Methods: Overview and Semi-Structured Interviewing 

 SurvMeth 651 (1.5 credit hours)

Instructor: Nancy Riley, Bowdoin College

This course will focus on semi-structured, or in-depth, interviewing. This methodology is often most helpful in understanding complex social processes. The course will examine the goals, assumptions, process, and uses of interviewing and compare these methods to other related qualitative and quantitative methods in order to develop research designs appropriate to research goals. The course will cover all aspects of interviewing, including how to decide who to interview, how to ask good interview questions, and how to conduct successful interviews. Students will conduct interviews, and discuss the process and outcome of those interviews. We will examine the strengths and weaknesses of this methodology, particularly through discussion of some of the critiques of these methods.

Prerequisite: An introductory course in survey research methods or equivalent experience.

2019 Syllabus (PDF)



Responsive Survey Design:  A Research Education Program  

Below is the list of the individual courses offered as part of the RSD Program.  Participants can either register for individual courses or for a specifically designed RSD Course Sequence Option.

For more information on this program, please visit the RSD Program web site:

Not for academic credit workshop (*Remote participation option available)

RSD has financial support available to those who qualify

Responsive survey design (RSD) refers to a method for designing surveys that has been demonstrated to increase the quality and efficiency of survey data collection. RSD uses evidence from early phases of data collection to make design decisions for later phases. Beginning in the 2018 Summer Institute, we will offer a series of eleven one-day short courses in RSD techniques.

*Remote participation optionIt is not necessary to be physically in Ann Arbor to participate in these workshops. Students who cannot be in Ann Arbor can enroll and join sessions via  BlueJeans (  Once enrollment is confirmed via email, indicate if course attendance will be in person, in Ann Arbor or via BlueJeans.  Survey Methodology for Randomized Controlled Trails does not have the remote participation option.

These courses will include:

1.  Survey Methodology for Health Science Researchers/Linking Adaptive Interventions and RSD (one-day workshop)  1450 ISR

Instructor: Brady West

Topics covered: Randomized Controlled Trials (RCTs) are an important tool for tests of internal validity of causal claims in both health and social sciences.  In practice, however, inattention to crucial details of data collection methodology can compromise the internal validity test.  One crucial example is recruitment and retention of participants – though randomized to treatment, unequal reluctance to participate or unequal attrition from the RCT jeopardize the internal validity of comparisons within the RCT design.  Another crucial example is the interaction of treatment and measurement – if the measures themselves change in response to the RCT treatment, then observed treatment and control differences may reflect these measurement differences rather than treatment differences.  In both cases, specific tools from survey methodology can be used to maximize the internal validity test in the RCT design. This course will focus on the survey methodology topics most important for maintaining the internal validity of RCT studies and feature specific examples of applications to RCTs.  One set of tools will focus on maximizing participation and minimizing attrition of participants.  Core survey methodology tools for encouraging participation in both pre-treatment measurement and the treatment itself as well as tools for minimizing the loss of participants to follow-up measures will be featured.  These tools include incentives, tailoring refusal conversion, switching modes, and tracking strategies. Links to RSD will also be made. A second set of tools will focus on measurement construction to reduce chances of interaction with treatment. These tools include mode options, questionnaire design issues, and special instruments (such as life history calendars) to minimize reporting error.  Each portion of the course will feature examples applying each specific tool to RCT studies.


2.  Basic Concepts and Theoretical Background (one-day workshop)  1070 ISR

Instructor: James Wagner, Brady West, Andy Peytchev and Frauke Kreuter

This course will provide participants with an overview of the primary concepts underlying RSD. This will include discussion of the uncertainty in survey design, the role of paradata, or data describing the data collection process, in informing decisions, and potential RSD interventions. These interventions include timing and sequence of modes, techniques for efficiently deploying incentives, and combining two-phase sampling with other design changes. Interventions appropriate for face-to-face, telephone, web, mail and mixed-mode surveys will be discussed. Using the Total Survey Error (TSE) framework, the main concepts behind these designs will be explained with a focus on how these principles are designed to simultaneously control survey errors and survey costs. Examples of RSD in both large and small studies will be provided as motivation.  Small group exercises will help participants to think through some of the common questions that need to be answered when employing RSD.


3.  Case Studies in Responsive Design Research (one-day workshop) 1450 ISR

Instructor: Brady West, William Axinn, Joe Murphy and Barry Schouten

This course will explore several well-developed examples of RSD. Dr. West will serve as a moderator of the course, and also introduce a case study from the National Survey of Family Growth (NSFG). The instructors will then provide independent examples of the implementation of RSD in different international surveys. All case studies will be supplemented with discussions of issues regarding the development and implementation of RSD. Case studies will include the NSFG, the Relationship Dynamics and Social Life (RDSL) survey, the University of Michigan Campus Climate (UMCC) Survey, and the Netherlands Survey of Consumer Satisfaction, among others. This variety of case studies will reflect a diversity of survey conditions. The NSFG (West) is a cross-sectional survey that is run on a continuous basis with in-person interviewing. The RDSL (Axinn) is a panel survey that employed a mixed-mode approach to collecting weekly journal data from a panel of young women. The UMCC survey is a web survey of students at UM that employed multiple modes of contact across the phases of the design. The Netherlands Survey of Consumer Satisfaction (Schouten) is a mixed-mode survey combining web and mail survey data collection with telephone interviewing. The focus of the course will be on practical tools for implementing RSD in a variety of conditions, including small-scale surveys.


4.  Responsive Survey Design for Web Surveys (one-day workshop)  4036 ISR

Instructor: Stephanie Coffey, Scott Crawford and Julie Smith

Topics covered: Web surveys can be an inexpensive method for collecting data. This is especially true for designs that repeat measurement over several time periods. However, these relatively low-cost data collections may result in reduced data quality if the problem of nonresponse is ignored. This course will examine methods for using RSD to effectively deploy scarce resources in order to minimize the risk of nonresponse bias. Recent experience with the University of Michigan Campus Climate Survey and the National Survey of College Graduates is used to illustrate this point. These surveys are defined by phased designs and multiple modes of contact. This approach produced relatively high response rates and used alternative contact methods in later phases to recruit sample members from subgroups that were less likely to respond in earlier phases. In the case of the UM-CCS all of this was accomplished on a very small budget and with a small management team. Lessons from these experiences can be directly applied in many similar settings.


5.  Data Visualization for Active Monitoring  (one-day workshop)  1070 ISR

Instructor: Brad Edwards and Victoria Vignare

Topics covered: This course will cover basic concepts for the design and use of “dashboards” for monitoring survey data collection. We will begin with a detailed discussion of how to design dashboards from an RSD perspective. This will include concrete discussions of how relevant data may be collected and summarized across a variety of production environments. We will also discuss how these dashboards can be used to implement RSD interventions on an ongoing basis. We will demonstrate these points using examples from actual dashboards. We will briefly explore methods for modeling incoming paradata in order to detect outliers. In the afternoon, we will consider practical issues associated with the development of dashboards, including software alternatives. Finally, we will demonstrate how to update dashboards using data reflecting the results of ongoing fieldwork. Students will be provided with template spreadsheet dashboards as discussed earlier.


6.  Alternative Indicators Designed to Maximize Data Quality (one-day workshop)  1070 ISR

Instructor: Barry Schouten and Natalie Shlomo

Topics covered: The response rate has been shown to be a poor indicator for data quality with respect to nonresponse bias. Several alternatives have been proposed – the fraction of missing information (FMI), R-Indicators, subgroup response rates, etc. This course will explore the use of these indicators as guides for data collection when working within an RSD framework. We also explore optimization techniques that may be useful when designing a survey to maximize these alternative indicators. The consequences of optimizing a survey to other indicators will be explored. We will also consider how the response rate fits into this approach. We will end with a brief discussion of methods for post data collection evaluation of data quality.


7.  A Management Model for Responsive Survey Design (one-day workshop)

Instructor: Heidi Guyer, Joe Murphy and Shonda Kruger-Ndiaye

This course will cover issues associated with implementation of RSD to manage field work. Instructors will provide concrete instruction on active monitoring of key indicators across a variety of environments – small-scale surveys, large-scale surveys, and web, telephone, face-to-face and mixed-mode surveys. Methods for implementing RSD interventions in a diversity of production environments will be discussed. RSD will be presented within the framework of the principles of project management, with a particular focus on risk management. A checklist of steps for implementing RSD will be discussed in detail. This course will draw upon a semester-long graduate course in survey management, which includes sections on RSD.


8.  Using SMART Design in Responsive Survey Design (one-day workshop)  1450 ISR

Instructor: Danny Almirall, Ahnalee Brincks and Nick Seewald

Topics covered: The effective treatment and management of a wide variety of health disorders often requires individualized, sequential decision making whereby treatment is adapted over time based on the changing disease state or specific circumstances of the patient. Adaptive interventions (also known as dynamic treatment regimens) operationalize this type of individualized treatment decision making using a sequence of decision rules that specify whether, how, for whom, or when to alter the intensity, type, or delivery of pharmacological, behavioral, and/or psychosocial treatments. There has been a huge surge of scientific interest in constructing adaptive interventions via the sequential multiple assignment randomized trial (SMART) design. SMART is a type of multi-stage randomized trial design, developed specifically for the purpose of collecting high-quality data for building optimal adaptive interventions. SMARTs are still new to the great majority of behavioral and social science investigators. In this course, we will introduce adaptive interventions, SMART (including simple design principle, cutting-edge analytic methods (e.g., Q-Learning) for SMART data, and discuss how these ideas can guide responsive and adaptive survey designs.


9.  Implementing, Managing, and Analyzing Interventions in a Responsive Survey Design Framework (one-day workshop)

Instructor: Brady West  1070 ISR

Topics covered: This course will discuss a variety of potential RSD interventions. Many of these have been implemented experimentally, and the course will include evaluations of those experiments. The importance of experimental evaluations in early phases of RSD will be discussed. Methods for implementing interventions will also be discussed, including implementation of experiments aimed at evaluating new interventions. Strategies for implementing these interventions with both interviewer-mediated and self-administered (e.g., web and mail) surveys will be discussed. Methods for the evaluation of the results of the interventions (experimental and otherwise) will be considered. These evaluations will include measures of both costs and errors.


10.  Implementation of Responsive Survey Design in Federal Statistical Systems (one-day workshop)  4036 ISR

Instructor: Ben Reist and Stephanie Coffey

Topics covered: This course will provide an overview of challenges and successes experienced in the development of adaptive survey design at the U.S. Census Bureau and the National Agricultural Statistics Service, including illustrations from the National Survey of College Graduates, the National Health Interview Survey and the Survey of Income and Program Participation. The presentation will include a brief history of the evolution of adaptive design capabilities at these agencies. We also discuss the development of a protocol for adaptive survey design that guides implementation and transparent documentation. The case studies covered will show applications of AD in surveys with different designs (cross-section vs. longitudinal, single vs. multi-mode) and different cost/quality objectives.  We discuss successes and failures in these applications and factors that will shape future uses of adaptive design.



20 Ways to Test Your Survey Questions   

SurvMeth 988.221 (1 credit hour) 

Instructor: Pamela Campanelli, UK Survey Methods Consultant

Testing your survey questionnaire is absolutely essential for ensuring a high quality survey. There has been a proliferation of question testing methods (both new methods and variations of existing methods). This course covers 20 different question testing techniques and combines information from the survey methodological literature with practical advice and hands on practice. The course covers methods for testing quantitative interview-based, web and mail questionnaires looking both at standard methods as well as new and unusual techniques. Overall the course covers: standard field test, interviewer rating form and variations, behaviour coding (classical, sequence-based and shortened), expert review, systematic form appraisal (including two online programmes), respondent debriefing (including vignettes and web probing), cognitive interviewing (short introduction), focus groups, split ballot tests, usability testing, studying item nonresponse and response distribution patterns, reliability and validity, record check studies, paradata, an introduction to analysis based methods (latent class, multi-trait-multi-method procedure and item response theory), eye tracking,  computational linguistics, and crowd sourcing).

Introduction to Applied Questionnaire Design is a course that complements well with this class.

Prerequisite: There is no prerequisite, but some knowledge of questionnaire design is of value.

2019 Syllabus (PDF)



Web Survey Design

SurvMeth 988.213 (1 credit hour)

Instructors: Mick Couper, University of Michigan

The course focuses on the design of Web survey instruments and procedures. The course covers all aspects of instrument design for Web surveys, including the appropriate use of various input tools (e.g., radio buttons, check boxes) for Web surveys, general formatting and layout issues, movement through the instrument (action buttons, navigation, error messages), and so on. The course draws on empirical results from experiments on alternative design approaches as well as practical experience in the design and implementation of a wide variety of Web surveys. The course also covers the challenges of designing Web surveys for mobile devices (smartphones and tablets), and the collection and use of paradata to evaluate Web survey instruments. The course does not focus on the technical aspects of Web survey implementation, such as hardware, software or programming. It also does not focus on general survey design or questionnaire design – participants are expected to have some familiarity with surveys.



Workshop in Survey Sampling Techniques

SurvMeth 616 (6 credit hours)

Instructors: Steve Heeringa, Jim Lepkowski and Raphael Nishimura, University of Michigan

The Workshop in Sampling Techniques is a component of the Sampling Program for Survey Statisticians. The workshop can only be taken in conjunction with the sampling methods courses, Methods of Survey Sampling and Analysis of Complex Sample Survey Data. The workshop allows students the opportunity to implement methods studied in the companion methods courses such as segmenting and listing in area sampling; selection of a national sample of the U.S.; stratification; controlled selection; telephone sampling; national samples for developing countries; and sampling with microcomputers.

The workshop is a required class for the Sampling Program for Survey Statisticians (SPSS). The SPSS is an eight-week program. It consists of three courses: a methods course (SurvMeth 612), a course on the analysis of complex sample survey data (SurvMeth 614), and a hands-on daily workshop (SurvMeth 616). Students enrolled in these three courses are considered Fellows in the Program. The methods and the analysis courses may be taken without being a Fellow. However, the workshop cannot be taken alone. Fellows receive a certificate upon successful completion of the program.

Syllabus 2018 (PDF)