Description of Courses

This course provides practical methods and tools to analyze complex survey data with a hands-on introduction to the use of specialized statistical software procedures. The course focuses on case studies with specific large-scale national surveys: the National Comorbidity Survey-Replication (NCS-R), the National Health and Nutrition Examination Surveys (NHANES), and the Health and Retirement Study (HRS). Relevant design features of the NCS-R, NHANES and HRS include survey weights that take into account differences in probability of selection into the sample and differences in response rates, as well as stratification and clustering in the multistage sampling procedures used in identifying the sampled households and individuals. After introducing essential concepts related to complex sample designs, the course will turn to the construction of survey weights, estimation of sampling variance, descriptive analysis, regression analysis, and finally special topics in the analysis of survey data. Participants can expect to work on homework exercises, computer lab exercises, and a final analysis project.

Why take this course? 

• To gain an understanding of modern methods and software for the secondary analysis of survey data collected from large complex samples
• To have the opportunity for one-on-one interaction with the instructors when walking through analyses of survey data
• To see various examples of applied statistical analyses of survey data
• To have the experience of writing a scientific paper that presents an analysis of complex sample survey data, and getting expert feedback on that paper

The recent proliferation of mobile technology allows researchers to collect objective health and behavioral data at increased intervals, in real time, and may also reduce participant burden. In this course, we will provide examples of the utility of and integration of wearables, sensors, and apps in research settings. Examples will include the use of wearable health devices to measure activity, apps for ecological momentary assessment, and smartphone sensors to measure sound and movement, among others. Additionally, this course will consider the integration of these new technologies into existing surveys and the quality of the data collected from the total survey error perspective. We will discuss considerations for assessing coverage, participation, and measurement error when integrating wearables, sensors, and apps in a research setting as well as the costs and privacy considerations when collecting these types of data. Participants will work in groups to discuss a research study design using new technology and have the opportunity for hands-on practice with sensor data.

Participants should have prior knowledge of question design before attending.

This course is designed to follow on from Introduction to Questionnaire Design or Writing Questions for Surveys. Now instead of looking at question comprehension from a cognitive side, the linguistic side will be explored including online tools. Factual questions will be revisited but with the goal of exploring different types of respondent memory problems and their solutions, while also covering time anomalies in surveys and quasi facts. Subjective questions will be revisited to understand attitude consistency and inconsistency, to look at the feasibility of changing attitudes to change behavioral intentions to change behaviors and to cover the popular topic of satisfaction and other customer experience metrics. Alternatives to questionnaires will also be covered such as event history calendars, internet enabled devices, factorial surveys and multi-item scales. The course concludes with ways to translate survey questions and evaluate the translation. The course will be interactive with the goal of making it as close to in-person training as possible. There also will be workshops throughout. Pamela is happy to chat with participants about their own questionnaires.

This course introduces the skills needed to conduct focus group interviews. Students will learn about the critical components of successful focus group research. They will develop a plan for a focus group study and then practice key skills. Attention will be placed on recent developments in conducting virtual focus groups via Zoom. This course will be particularly applicable for those conducting focus group research in academic, non-profit, and government settings.

Course Topic

The course will cover these skills:

• Conducting in-person and virtual focus groups
• How to plan and design a focus group study
• Identifying information-rich participants and getting them to show up
• Beginning the focus group - the crucial first few minutes and moderating techniques
• Developing questions—Characteristics of good focus group questions
• Analyzing—Options for analysis

Course Format

The course will be presented on Zoom over a period of 2 days.

Why Take This Course?

Focus groups are used to understand issues, pilot test ideas, and evaluate programs. They also provide great insight when used in combination with surveys. Focus groups have been used to help design surveys, to pilot test surveys, and to understand survey findings. Take this course if you want to learn more about how focus groups might add to your research toolbox.

This course will begin to empower students with an understanding of the importance and basic tenets of rigorous questionnaire design, as well as practice designing an appropriate instrument for a real world problem. Students will watch course videos independently, and work on a questionnaire for a topic of their choosing. Four live online meetings (Tuesdays and Thursdays from 2-3:30 PM EST) will take a workshop format where students will ask questions, share their own questionnaires in progress, and give feedback to classmates.

This course covers the basic principles of survey design and methods and introduces the necessary components of a good quality survey.   The course employs the Total Survey Error framework to discuss sampling frames and designs, modes of data collection and their effects on survey errors, the cognitive processes involved in answering survey questions and their impact on questionnaire design, pretesting methods and post-data collection processing.  The goal of the course is to give an introduction to the skills and resources needed to design and conduct a survey

In this two-week course, students will learn a variety of natural language processing methods for analyzing and extracting meaning from text data. The course will start with an introduction to text data, including text preprocessing and exploratory methods. The topics that follow will include machine learning models used for topic modeling, clustering, classification, sentiment analysis, and word embeddings. Students will also be introduced to web scraping. Considerations to both long and short texts of various subject matter. Class examples will be demonstrated primarily in R. This course assumes a bachelors-level background in Statistics or related field and knowledge of R or Python; no prior knowledge of text analysis is assumed.

The Health and Retirement Study (hrs.isr.umich.edu) workshop is intended to give participants an introduction to the study that will enable them to get started using the data for research. HRS is a large-scale longitudinal study with more than 20 years of data on the labor force participation and health transitions that individuals undergo toward the end of their work lives and in the years that follow. This online workshop is intended for users who have little to no experience using HRS data.

Content lectures delivered by HRS co-investigators and content area experts on basic survey content, sample design, weighting, and restricted data files will be available on the course website for viewing ahead of time. During the week of the workshop, each content lecturer will participate in a Zoom meeting with the class to answer questions about their lecture. The majority of each day will be devoted to data labs in which participants will gain experience using the data, with a strong focus on introductory data management and simple data analysis.

The data lab portion assumes some familiarity with Stata. We will provide a temporary site license for Stata and an introductory Stata training module that participants can complete ahead of the workshop to be well prepared. The programming code for the labs will also be provided in SAS.

This course provides an introduction to supervised statistical learning techniques such
as decision trees, random forests and boosting and discusses their potential application
in the social sciences. These methods focus on predicting an outcome Y based on
some learned function f(X) and therefore facilitate new research perspectives in
comparison with traditional regression models, which primarily focus on causation.
Predictive methods also provide a valuable extension to the empirical social scientists’
toolkit as new data sources become more prominent. In addition to introducing
supervised learning methods, the course will include practical sessions to demonstrate
how to tune and evaluate prediction models using the statistical programming language
R.

A fundamental feature of many sample surveys is a probability sample of subjects. Probability sampling requires rigorous application of mathematical principles to the selection process. Methods of Survey Sampling is a moderately advanced course in applied statistics, with an emphasis on the practical problems of sample design, which provides students with an understanding of principles and practice in skills required to select subjects and analyze sample data. Topics covered include stratified, clustered, systematic, and multi-stage sample designs, unequal probabilities and probabilities proportional to size, area probability sampling, ratio means, sampling errors, frame problems, cost factors, and practical designs and procedures. Emphasis is on practical considerations rather than on theoretical derivations, although understanding of principles requires review of statistical results for sample surveys. The course includes an exercise that integrates the different techniques into a comprehensive sample design.

Why take this course? 

• To understand the basic ideas, concepts and principles of probability sampling from an applied perspective
• To be able to identify and appropriately apply sampling techniques to survey design problems
• To be able to compute the sample size for a variety of sample designs
• To understand and be able to assess the impact of the sample design on survey estimates
• To learn how to design and select a probability sample involving complex sampling techniques in a survey project, and receive expert feedback on a sampling report

This course will focus on semi-structured, or in-depth, interviewing, with a brief introduction to other qualitative methods, including observation. Semi-structured interviewing is often most helpful in understanding complex social processes.  We will examine the goals, assumptions, process, and uses of interviewing and compare these methods to other related qualitative and quantitative methods in order to develop research designs appropriate to research goals. The course will cover all aspects of interviewing, including how to decide who to interview, how to ask good interview questions, and how to conduct successful interviews. Students will conduct interviews, and discuss the process and outcome of those interviews. We will examine the strengths and weaknesses of this methodology, particularly through discussion of some of the critiques of these methods.

This course will provide participants with an overview of the primary concepts underlying RSD. This will include discussion of the uncertainty in survey design, the role of paradata, or data describing the data collection process, in informing decisions, and potential RSD interventions. These interventions include timing and sequence of modes, techniques for efficiently deploying incentives, and combining two-phase sampling with other design changes. Interventions appropriate for face-to-face, telephone, web, mail and mixed-mode surveys will be discussed. Using the Total Survey Error (TSE) framework, the main concepts behind these designs will be explained with a focus on how these principles are designed to simultaneously control survey errors and survey costs. Examples of RSD in both large and small studies will be provided as motivation.  Small group exercises will help participants to think through some of the common questions that need to be answered when employing RSD. Not for academic credit. 

This first webinar in a two-part series on implementing interventions in a responsive design framework will discuss a variety of potential RSD interventions. Many of these have been implemented experimentally, and the course will include evaluations of those experiments. The importance of experimental evaluations in early phases of RSD will be discussed. Methods for implementing interventions will also be discussed, including implementation of experiments aimed at evaluating new interventions. Strategies for implementing these interventions with both interviewer-mediated and self-administered (e.g., web and mail) surveys will be discussed. Methods for the evaluation of the results of the interventions (experimental and otherwise) will be considered. These evaluations will include measures of both costs and errors. Not for academic credit.  

This course is intended as an introduction to the science behind survey research and will be taught at an undergraduate level. Our primary reason for designing this course is to share our excitement about survey methodology as a research field. In this course, we focus on principles and theories as opposed to a strictly hands on approach to creating surveys. This approach to the study of surveys will allow you to apply the same methods to a very broad range of research topics, e.g., public opinion and politics, health, drug use, consumer behavior, customer satisfaction and market research among many others. In addition, the course will present basic statistical concepts and techniques in sample design, data collection, and reporting, as well as explanations of how error can be introduced into the survey process. At the end of this course, you will have a basic understanding of what needs to go into conducting a high-quality survey and will be exposed to resources that will allow you build on the knowledge of survey methodology you have learned in the course.

The Workshop in Sampling Techniques is a component of the Sampling Program for Survey Statisticians. The workshop can only be taken in conjunction with the Methods of Survey Sampling and Analysis of Complex Sample Survey Data courses. The workshop allows students the opportunity to implement methods studied in the companion methods courses such as segmenting and listing in area sampling; selection of a national sample of the U.S.; stratification; controlled selection; telephone sampling; national samples for developing countries; and sampling with computers.

The workshop is a required class for the Sampling Program for Survey Statisticians (SPSS). The SPSS is an eight-week program. It consists of three courses: a methods course (SurvMeth 612), a course on the analysis of complex sample survey data (SurvMeth 614), and a hands-on daily workshop (SurvMeth 616). Students enrolled in these three courses are considered Fellows in the Program. The methods and the analysis courses may be taken without being a Fellow. However, the workshop cannot be taken alone. Fellows receive a certificate upon successful completion of the program.

COURSE OBJECTIVES
• Introduce a structural analysis of parts of a survey question
• Introduce cognitive interviewing as a method for testing survey questions
• Describe guidelines for diagnosing problems in survey questions and writing new survey questions
• Focus on the structure and wording of survey questions, whether for interviewer-administered or self- administered instruments
• Provide an opportunity to apply the guidelines and principles during in-class exercises
• Focus on improving individual questions and sets of questions.
• Summarize research that underlies key decisions in writing survey questions.

DESCRIPTION

This workshop distills research about survey questions to principles that can be applied to write survey questions that are clear and obtain reliable answers. The workshop provides students with tools to use in diagnosing problems in survey questions and in writing their own survey questions. Sessions combine lecture with group exercises and discussion. The lecture provides guidelines for writing and revising survey questions and illustrates how to revise troubled questions. Assignments require that students analyze problematic questions, revise them, and administer them to fellow students. Sessions consider both questions about events and behaviors and questions about subjective phenomena (such as attitudes, evaluations, and internal
states).

WHO SHOULD ATTEND
Individuals who will be writing or reviewing survey questions or survey instruments or analyzing survey data. This course gives practical guidance to those who have written survey questions but who are not familiar with research on question design, those who are just beginning to design survey instruments, and those who use survey data but do not themselves design survey instruments.