Description of Courses

This course provides practical methods and tools to analyze complex survey data with a hands-on introduction to the use of specialized statistical software procedures. The course focuses on case studies with specific large-scale national surveys: the National Comorbidity Survey-Replication (NCS-R), the National Health and Nutrition Examination Surveys (NHANES), and the Health and Retirement Study (HRS). Relevant design features of the NCS-R, NHANES and HRS include survey weights that take into account differences in probability of selection into the sample and differences in response rates, as well as stratification and clustering in the multistage sampling procedures used in identifying the sampled households and individuals. After introducing essential concepts related to complex sample designs, the course will turn to the construction of survey weights, estimation of sampling variance, descriptive analysis, regression analysis, and finally special topics in the analysis of survey data. Participants can expect to work on homework exercises, computer lab exercises, and a final analysis project.

Why take this course? 

• To gain an understanding of modern methods and software for the secondary analysis of survey data collected from large complex samples
• To have the opportunity for one-on-one interaction with the instructors when walking through analyses of survey data
• To see various examples of applied statistical analyses of survey data
• To have the experience of writing a scientific paper that presents an analysis of complex sample survey data, and getting expert feedback on that paper

The recent proliferation of mobile technology allows researchers to collect objective health and behavioral data at increased intervals, in real time, and may also reduce participant burden. In this course, we will provide examples of the utility of and integration of wearables, sensors, and apps in research settings. Examples will include the use of wearable health devices to measure activity, apps for ecological momentary assessment, and smartphone sensors to measure sound and movement, among others. Additionally, this course will consider the integration of these new technologies into existing surveys and the quality of the data collected from the total survey error perspective. We will discuss considerations for assessing coverage, participation, and measurement error when integrating wearables, sensors, and apps in a research setting as well as the costs and privacy considerations when collecting these types of data. Participants will work in groups to discuss a research study design using new technology and have the opportunity for hands-on practice with sensor data.

The Health and Retirement Study (hrsonline.isr.umich.edu) Summer Workshop is intended to give participants an introduction to the study that will enable them to use the data for research. HRS is a large-scale longitudinal study with more than 20 years of data on the labor force participation and health transitions that individuals undergo toward the end of their work lives and in the years that follow.

Disability Data in the Health and Retirement Study (HRS) is a 3-day online (Zoom) workshop intended to give participants an overview of the disability data resources in the HRS. Content lectures delivered by HRS co-investigators and content area experts will cover a range of topics including:

• The various ways that disability, disabling conditions, health, and functioning are measured in the HRS;
• How measures of disability have changed as the survey has evolved;
• ●        How the HRS captures disability benefit receipt in the survey and through and linkage with data from the Social Security Administration;
• ●        How disability-related topics like employer accommodations, assistive technology and personal assistance are measured in the survey.

In addition to presentations on these topics, the workshop will feature labs focused on working with work disability measures in Section M and disability spell data in the RAND HRS.

Students will have the opportunity to present research ideas and receive feedback from the workshop faculty and other students.

The course is designed for those with experience using HRS data or for those who have taken the introductory HRS workshop. The data training portion assumes some familiarity with STATA.

This one-week course introduces students to the design and implementation of online survey data collection instruments. The course is both hands-on and conceptual. It begins by discussing what is unique about web surveys and when their use is most appropriate, followed by an introduction to survey errors that can affect the quality of web survey data. Small groups of students will each develop a research problem and a questionnaire to address their problem, designed for online administration. They will pretest the question wording, program the questionnaire using a web survey development platform (no programming experience is required), and assess users’ (respondents’) experience while interacting with the web-based instrument. Students will also develop basic plans for data collection and analysis. Finally, each group will present its problem, online questionnaire, evaluation, and plans to the rest of the class.  

Why take this course?

· To gain an understanding of what should go into creating a web-based questionnaire

· To gain experience weighing the pros and cons of different web questionnaire features

· To gain experience building a web questionnaire on a standard platform

· To gain experience evaluating survey questions and their usability in an online questionnaire

Participants should have prior knowledge of questionnaire design before attending.

This course is designed to follow on from Introduction to Questionnaire Design or Writing Questions for Surveys. Now instead of looking at question comprehension from a cognitive side, the linguistic side will be explored including online tools. Factual questions will be revisited but with the goal of exploring different types of respondent memory problems. Solutions include basic aids to improve memory and alternative methods: decomposition, calendars, event history calendars, internet enabled devices, wearables, apps and sensors, and additional tasks on mobile phones. Also covered are the effects of telescoping and quasi-facts. Subjective questions will be revisited to understand attitude consistency and inconsistency, to look at the feasibility of changing attitudes to change behavioral intentions to change behaviors and to cover the popular topic of satisfaction and other customer experience metrics. Alternatives methods to attitude measurement will also be covered: factorial surveys and multi-item scales. The course concludes with multi-cultural issues raised by Rincón and a mini appendix on ways to translate survey questions and evaluate the translation. The course will be interactive with the goal of making it as close to in-person training as possible. There also will be workshops throughout. Pamela is happy to chat with participants about their own questionnaires.”

The course will be presented in two sections. Section 1 (June 11) will be an overview of focus group interviewing research. In this session we will cover the basics of in-person focus group interviewing. Topics would include planning, developing questions, recruiting, moderating and analyzing the focus group study. Special attention is placed on conducting focus group research in the academic, public and non-profit environments. Section 2 (June 12) will build on Session 1 and introduce the strategies for conducting on-line focus group research. The course will introduce protocol for planning, conducting and analyzing results from on-line groups using Zoom. Course Topic The course will cover these skills:

• Conducting in-person and virtual focus groups
• How to plan and design a focus group study
• Identifying information-rich participants and getting them to show up
• Beginning the focus group – the crucial first few minutes and moderating techniques
• Developing questions—Characteristics of good focus group questions
• Analyzing—Options for analysis

Course Format The course will be presented on Zoom over a period of 2 days. Why Take This Course? Focus groups are used to understand issues, pilot test ideas, and evaluate programs. They also provide great insight when used in combination with surveys. Focus groups have been used to help design surveys, to pilot test surveys, and to understand survey findings. Take this course if you want to learn more about how focus groups might add to your research toolbox.

This course approaches questionnaire design from a practical, hands-on perspective. Students will draft their own questionnaires, starting with a definition of research objectives, and provide feedback on their peers’ questions. As students walk through this process, coursework will cover the importance and basic tenets of rigorous design, and provide frameworks and tools for thinking about some of the common challenges in questionnaire development.

This course covers the basic principles of survey design and methods and introduces the necessary components of a good quality survey.   The course employs the Total Survey Error framework to discuss sampling frames and designs, modes of data collection and their effects on survey errors, the cognitive processes involved in answering survey questions and their impact on questionnaire design, pretesting methods and post-data collection processing.  The goal of the course is to give an introduction to the skills and resources needed to design and conduct a survey

In this two-week course, students will learn a variety of natural language processing methods for analyzing and extracting meaning from text data. The course will start with an introduction to text data, including text preprocessing and exploratory methods. The topics that follow will include machine learning models used for topic modeling, clustering, classification, sentiment analysis, and word embeddings. Students will also be introduced to web scraping. Considerations to both long and short texts of various subject matter. Class examples will be demonstrated primarily in R. This course assumes a bachelors-level background in Statistics or related field and knowledge of R or Python; no prior knowledge of text analysis is assumed.

The Health and Retirement Study (hrs.isr.umich.edu) workshop is intended to give participants an introduction to the study that will enable them to get started using the data for research. HRS is a large-scale longitudinal study with more than 20 years of data on the labor force participation and health transitions that individuals undergo toward the end of their work lives and in the years that follow. This online workshop is intended for users who have little to no experience using HRS data.

Content lectures delivered by HRS co-investigators and content area experts on basic survey content, sample design, weighting, and restricted data files will be available on the course website for viewing ahead of time. During the week of the workshop, each content lecturer will participate in a Zoom meeting with the class to answer questions about their lecture. The majority of each day will be devoted to data labs in which participants will gain experience using the data, with a strong focus on introductory data management and simple data analysis.

The data lab portion assumes some familiarity with Stata. We will provide a temporary site license for Stata and an introductory Stata training module that participants can complete ahead of the workshop to be well prepared. The programming code for the labs will also be provided in SAS.

This course provides an introduction to supervised statistical learning techniques such
as decision trees, random forests and boosting and discusses their potential application
in the social sciences. These methods focus on predicting an outcome Y based on
some learned function f(X) and therefore facilitate new research perspectives in
comparison with traditional regression models, which primarily focus on causation.
Predictive methods also provide a valuable extension to the empirical social scientists’
toolkit as new data sources become more prominent. In addition to introducing
supervised learning methods, the course will include practical sessions to demonstrate
how to tune and evaluate prediction models using the statistical programming language
R.

A fundamental feature of many sample surveys is a probability sample of subjects. Probability sampling requires rigorous application of mathematical principles to the selection process. Methods of Survey Sampling is a moderately advanced course in applied statistics, with an emphasis on the practical problems of sample design, which provides students with an understanding of principles and practice in skills required to select subjects and analyze sample data. Topics covered include stratified, clustered, systematic, and multi-stage sample designs, unequal probabilities and probabilities proportional to size, area probability sampling, ratio means, sampling errors, frame problems, cost factors, and practical designs and procedures. Emphasis is on practical considerations rather than on theoretical derivations, although understanding of principles requires review of statistical results for sample surveys. The course includes an exercise that integrates the different techniques into a comprehensive sample design.

Why take this course? 

• To understand the basic ideas, concepts and principles of probability sampling from an applied perspective
• To be able to identify and appropriately apply sampling techniques to survey design problems
• To be able to compute the sample size for a variety of sample designs
• To understand and be able to assess the impact of the sample design on survey estimates
• To learn how to design and select a probability sample involving complex sampling techniques in a survey project, and receive expert feedback on a sampling report

This course reviews multiple methods of data collection and presents study designs for combining multiple methods within a single research project. The course focuses on the integration of survey methods with multiple alternative methods to achieve a single data collection approach using the strengths of some methods to compensate for weaknesses in other methods. The methods examined include unstructured or in-depth interviews, semi-structured interviews, focus groups, survey interviews, observation, geographic information systems, archival research, social media analysis and hybrid methods. Emphasis will be placed on the specific contribution of each method, as well as the use of combined methods to advance specific research questions. This course is designed for those with a specific research question in mind. Throughout the course, participants will be asked to design multi-method approaches to a research question of their choice. By the end of this course, participants will have an overview of multi-method research that will enable them to design, understand, and evaluate multi-method approaches within a single project.

This course will focus on semi-structured, or in-depth, interviewing, with a brief introduction to other qualitative methods, including observation. Semi-structured interviewing is often most helpful in understanding complex social processes.  We will examine the goals, assumptions, process, and uses of interviewing and compare these methods to other related qualitative and quantitative methods in order to develop research designs appropriate to research goals. The course will cover all aspects of interviewing, including how to decide who to interview, how to ask good interview questions, and how to conduct successful interviews. Students will conduct interviews, and discuss the process and outcome of those interviews. We will examine the strengths and weaknesses of this methodology, particularly through discussion of some of the critiques of these methods.

This course will provide participants with an overview of the primary concepts underlying RSD. This will include discussion of the uncertainty in survey design, the role of paradata, or data describing the data collection process, in informing decisions, and potential RSD interventions. These interventions include timing and sequence of modes, techniques for efficiently deploying incentives, and combining two-phase sampling with other design changes. Interventions appropriate for face-to-face, telephone, web, mail and mixed-mode surveys will be discussed. Using the Total Survey Error (TSE) framework, the main concepts behind these designs will be explained with a focus on how these principles are designed to simultaneously control survey errors and survey costs. Examples of RSD in both large and small studies will be provided as motivation.  Small group exercises will help participants to think through some of the common questions that need to be answered when employing RSD. Not for academic credit. 

This webinar to learn about implementing interventions in a responsive design framework will discuss a variety of potential RSD interventions. Many of these have been implemented experimentally, and the course will include evaluations of those experiments. The importance of experimental evaluations in early phases of RSD will be discussed. Methods for implementing interventions will also be discussed, including implementation of experiments aimed at evaluating new interventions. Strategies for implementing these interventions with both interviewer-mediated and self-administered (e.g., web and mail) surveys will be discussed. Methods for the evaluation of the results of the interventions (experimental and otherwise) will be considered. These evaluations will include measures of both costs and errors. Not for academic credit.  

This course is intended as an introduction to the science behind survey research and will be taught at an undergraduate level. Our primary reason for designing this course is to share our excitement about survey methodology as a research field. In this course, we focus on principles and theories as opposed to a strictly hands on approach to creating surveys. This approach to the study of surveys will allow you to apply the same methods to a very broad range of research topics, e.g., public opinion and politics, health, drug use, consumer behavior, customer satisfaction and market research among many others. In addition, the course will present basic statistical concepts and techniques in sample design, data collection, and reporting, as well as explanations of how error can be introduced into the survey process. At the end of this course, you will have a basic understanding of what needs to go into conducting a high-quality survey and will be exposed to resources that will allow you build on the knowledge of survey methodology you have learned in the course.

The Workshop in Sampling Techniques is a component of the Sampling Program for Survey Statisticians. The workshop can only be taken in conjunction with the Methods of Survey Sampling and Analysis of Complex Sample Survey Data courses. The workshop allows students the opportunity to implement methods studied in the companion methods courses such as segmenting and listing in area sampling; selection of a national sample of the U.S.; stratification; controlled selection; telephone sampling; national samples for developing countries; and sampling with computers.

The workshop is a required class for the Sampling Program for Survey Statisticians (SPSS). The SPSS is an eight-week program. It consists of three courses: a methods course (SurvMeth 612), a course on the analysis of complex sample survey data (SurvMeth 614), and a hands-on daily workshop (SurvMeth 616). Students enrolled in these three courses are considered Fellows in the Program. The methods and the analysis courses may be taken without being a Fellow. However, the workshop cannot be taken alone. Fellows receive a certificate upon successful completion of the program.

COURSE OBJECTIVES
• Introduce a structural analysis of parts of a survey question
• Introduce cognitive interviewing as a method for testing survey questions
• Describe guidelines for diagnosing problems in survey questions and writing new survey questions
• Focus on the structure and wording of survey questions, whether for interviewer-administered or self- administered instruments
• Provide an opportunity to apply the guidelines and principles during in-class exercises
• Focus on improving individual questions and sets of questions.
• Summarize research that underlies key decisions in writing survey questions.

DESCRIPTION

This workshop distills research about survey questions to principles that can be applied to write survey questions that are clear and obtain reliable answers. The workshop provides students with tools to use in diagnosing problems in survey questions and in writing their own survey questions. Sessions combine lecture with group exercises and discussion. The lecture provides guidelines for writing and revising survey questions and illustrates how to revise troubled questions. Assignments require that students analyze problematic questions, revise them, and administer them to fellow students. Sessions consider both questions about events and behaviors and questions about subjective phenomena (such as attitudes, evaluations, and internal
states).

WHO SHOULD ATTEND
Individuals who will be writing or reviewing survey questions or survey instruments or analyzing survey data. This course gives practical guidance to those who have written survey questions but who are not familiar with research on question design, those who are just beginning to design survey instruments, and those who use survey data but do not themselves design survey instruments.