Description of Courses

This short course will offer a very practical introduction to web scraping geared at social scientists and survey researchers.  This course begins with an overview of web scraping discussing some basic technical jargon, types of web data and various methods for scraping.  Some websites are designed to be easily accessible by web crawlers or scraping algorithms while others require much more advanced, custom programming.  In this course we will illustrate how participants can discern these differences as well as presenting several motivating examples of the various ways web scraped data can be used throughout a study’s lifecycle from design to calibration to analysis.  We provide an extensive introduction to a suite of freeware programs that allow virtually syntax free, but customizable, web scraping capabilities.  The course concludes with specific focus on the import.io tool where we demonstrate its capabilities and provide several, hands-on practical examples for participants to begin scraping several websites of increasing complexity.  

The amount of data generated as a by-product in society is growing fast including data from satellites, sensors, transactions, social media and smartphones, just to name a few. Such data are often referred to as "big data", and can be used to create value in different areas such as health and crime prevention, commerce and fraud detection.  An emerging practice in many areas is to append or link big data sources with more specific and smaller scale sources that often contain much more limited information.  This practice has been used for some time by survey researchers in constructing frames by appending auxiliary information that is often not directly available on the frame, but can be obtained from an external source.   Using Big Data has the potential to go beyond the sampling phase for survey researchers and in fact has the potential to influence the social sciences in general.  Big Data is of interest for public opinion researchers and agencies that produce statistics to find alternative data sources either to reduce costs, to improve estimates or to produce estimates in a more timely fashion. However, Big Data pose several interesting and new challenges to survey researchers and social scientists among others who want to extract information from data. As Robert Groves (2012) pointedly commented, the era is “appropriately called Big Data and not Big Information”, because there is a lot of work for analysts before information can be gained from “auxiliary traces of some process that is going on in society.” This course offers participants a broad overview of big data sources, opportunities and examples motivated within the survey and social science contexts including the use of social media data, para data and other such sources.  This course also offers a detailed, practical introduction to four common machine learning methods that can be applied to big and small data alike at various aspects of a study’s lifecycle from design to nonresponse adjustments to propensity score matching to weighting and evaluation and analysis.  The machine learning methods will be demonstrated in R and we will provide several different examples of using these methods along with multiple packages in R that offer these methods.  

This course provides practical methods and tools to analyze complex survey data with a hands-on introduction to the use of specialized statistical software procedures. The course focuses on case studies with specific large-scale national surveys: the National Comorbidity Survey-Replication (NCS-R), the National Health and Nutrition Examination Surveys (NHANES), and the Health and Retirement Study (HRS). Relevant design features of the NCS-R, NHANES and HRS include survey weights that take into account differences in probability of selection into the sample and differences in response rates, as well as stratification and clustering in the multistage sampling procedures used in identifying the sampled households and individuals. After introducing essential concepts related to complex sample designs, the course will turn to the construction of survey weights, estimation of sampling variance, descriptive analysis, regression analysis, and finally special topics in the analysis of survey data. Participants can expect to work on homework exercises, computer lab exercises, and a final analysis project. Why take this course? 

• To gain an understanding of modern methods and software for the secondary analysis of survey data collected from large complex samples
• To have the opportunity for one-on-one interaction with the instructors when walking through analyses of survey data
• To see various examples of applied statistical analyses of survey data
• To have the experience of writing a scientific paper that presents an analysis of complex sample survey data, and getting expert feedback on that paper

Prerequisite: Two graduate-level courses in statistical methods, familiarity with basic sample design concepts, and familiarity with data analytic techniques such as linear and logistic regression. *Remote participation option:  It is not necessary to physically be in Ann Arbor to participate in this course. Students who cannot be in Ann Arbor can enroll and join sessions via BlueJeans (https://www.bluejeans.com). Once enrollment is confirmed via email, please indicate if course attendance will be in person in Ann Arbor) or via BlueJeans. Individual access to Stata, SAS or R software is required for remote participation.

The recent proliferation of mobile technology allows researchers to collect objective health and behavioral data at increased intervals, in real time, and may also reduce participant burden. In this course, we will provide examples of the utility of and integration of wearables, sensors, and apps in research settings. Examples will include the use of wearable health devices to measure activity, apps for ecological momentary assessment, and smartphone sensors to measure sound and movement, among others. Additionally, this course will consider the integration of these new technologies into existing surveys and the quality of the data collected from the total survey error perspective. We will discuss considerations for assessing coverage, participation, and measurement error when integrating wearables, sensors, and apps in a research setting as well as the costs and privacy considerations when collecting these types of data. Participants will work in groups to discuss a research study design using new technology and have the opportunity for hands-on practice with sensor data.

This two-day course provides an overview and guided exploration of cutting edge methods for collecting and analyzing web data. The course is divided into two parts. Part I provides a crash course (introduction through guided examples) in collecting, wrangling, and analyzing unstructured (via scraping) and structured (via APIs) web data. Guided and hands-on examples will focus on various digital media (e.g., New York Times API) and public opinion (e.g., Reddit) data sources and will include analytical examples of text mining and machine learning. Part II of the course provides a deep-dive into Twitter data, including an extensive overview of the popular {rtweet} package (by the package author himself) and numerous guided examples and walk-throughs featuring cutting edge techniques in sentiment analysis, neural networks, bot detection, network analysis, etc. Programming language: R Software libraries: httr, rvest, healthforum*, rtweet*, tweetbotornot2*, congress116, xgboost, igraph, quanteda, text2vec, textfeatures*, wactor*, tidyverse, data.table *Authored or co-authored by course instructor

This course is designed to follow on from Introduction to Questionnaire Design. Now instead of looking at question comprehension from a cognitive side, the linguistic side will be explored including online tools. Factual questions will be revisited but with the goal of exploring different types of respondent memory problems and their solutions, while also covering time anomalies in surveys and quasi facts. Subjective questions will be revisited to understand attitude consistency and inconsistency, to look at the feasibility of changing attitudes to change behavioral intentions to change behaviors and to cover the popular topic of satisfaction and other customer experience metrics. Alternatives to questionnaires will also be covered such as event history calendars, internet enabled devices, GPS, factorial surveys and multi-item scales. The course concludes with ways to translate survey questions and evaluate the translation. The course with be interactive and there will be workshops throughout. Pamela is also happy to chat with participants about their own questionnaires.

This 2-day workshop will introduce students to different methods of collecting data in the social sciences. Surveys are the most common form of collecting primary data in many disciplines, and this course will provide students with an overview of interview-administered (face-to-face and telephone) and self-administered (mail, web, mobile web, and SMS) survey data collection as well as the combination of multiple modes (mixed mode surveys). The course will in particular discuss the implication of survey design decisions on data quality. In addition, students will also receive an overview on alternative data sources (e.g., passive measurement, social media and administrative data) and how they can be used in combination with traditional survey data.

Introduction: This course introduces the skills needed to conduct focus group interviews. Students will learn about the critical components of successful focus group research. They will develop a plan for a focus group study and then practice key skills. Attention will be placed on moderating, recruiting, developing questions, and analysis of focus groups. This course will be particularly applicable for those conducting focus group research in academic, non-profit, and government settings. Course Topics: The course will cover these skills:

• Planning—When to use focus groups and designing a study
• Recruiting—Identifying information-rich participants and getting them to show up
• Hosting—Creating a permissive and nonthreatening environment
• Moderating—The crucial first few minutes and moderating techniques
• Developing questions—Characteristics of good focus group questions
• Analyzing—Options for analysis
• Reporting—Options for sharing what was learned

Course Format The course format includes daily lectures along with opportunities to practice critical skills in small groups. Why Take This Course? Focus groups are used to understand issues, pilot test ideas, and evaluate programs. They also provide great insight when used in combination with surveys. Focus groups have been used to help design surveys, to pilot test surveys, and to understand survey findings. Take this course if you want to learn more about how focus groups might add to your research toolbox. Prerequisite: An introductory course in research methods or equivalent experience.

This course provides an overview of the art and science of questionnaire design. Topics will include basic principles of questionnaire design; factual and non-factual questions; techniques for asking about sensitive topics; designing scales and response options; survey mode considerations; and an introduction to pre-testing surveys. The course will consist of both lectures and hands-on activities, as well as an optional third day for students who are working on their own surveys to apply course learnings to their instruments.

This 2-day course will introduce participants to the basic principles of survey design, presented within the Total Survey Error framework.  The course provides an introduction to the skills and resources needed to design and conduct a survey, covering topics such as sampling frames and designs, mode of data collection and their impact on survey estimates, cognitive processes involved in answering survey questions, best questionnaire design practices, and pretesting methods.

This is a foundation course in sample survey methods and principles. The instructors will present, in a non-technical manner, basic sampling techniques such as simple random sampling, systematic sampling, stratification, and cluster sampling. The instructors will provide opportunities to implement sampling techniques in a series of exercises that accompany each topic. Participants should not expect to obtain sufficient background in this course to master survey sampling. They can expect to become familiar with basic techniques well enough to converse with sampling statisticians more easily about sample design.

The Health and Retirement Study (hrsonline.isr.umich.edu) Summer Workshop is intended to give participants an introduction to the study that will enable them to use the data for research. HRS is a large-scale longitudinal study with more than 20 years of data on the labor force participation and health transitions that individuals undergo toward the end of their work lives and in the years that follow. The HRS Summer Workshop features morning lectures on basic survey content, sample design, weighting, and restricted data files. Hands-on data workshops are held every afternoon in which participants learn to work with the data (including the user-friendly RAND version of the HRS data) under the guidance of HRS staff. Staff of the Gateway to Global Aging project (G2Aging.org), which harmonizes data across HRS international sister studies, conduct an afternoon training. At the end of the week, students have the opportunity to present their research ideas to the class and HRS research faculty and obtain feedback. Topics include (but are not limited to) in depth information on HRS data about health insurance and medical care; biomarkers, physical measures, and genetic data; cognition; health and physical functioning; linkage to Medicare; employment, retirement, and pensions and linkage to Social Security records; psychosocial and well-being; family data; and international comparison data. The data training portion assumes some familiarity with SAS or STATA.

A fundamental feature of many sample surveys is a probability sample of subjects. Probability sampling requires rigorous application of mathematical principles to the selection process. Methods of Survey Sampling is a moderately advanced course in applied statistics, with an emphasis on the practical problems of sample design, which provides students with an understanding of principles and practice in skills required to select subjects and analyze sample data. Topics covered include stratified, clustered, systematic, and multi-stage sample designs, unequal probabilities and probabilities proportional to size, area probability sampling, ratio means, sampling errors, frame problems, cost factors, and practical designs and procedures. Emphasis is on practical considerations rather than on theoretical derivations, although understanding of principles requires review of statistical results for sample surveys. The course includes an exercise that integrates the different techniques into a comprehensive sample design. Why take this course? 

• To understand the basic ideas, concepts and principles of probability sampling from an applied perspective
• To be able to identify and appropriately apply sampling techniques to survey design problems
• To be able to compute the sample size for a variety of sample designs
• To understand and be able to assess the impact of the sample design on survey estimates
• To learn how to design and select a probability sample involving complex sampling techniques in a survey project, and receive expert feedback on a sampling report

Prerequisite: Two graduate-level courses in statistical methods. *Remote participation option:  It is not necessary to physically be in Ann Arbor to participate in this course. Students who cannot be in Ann Arbor can enroll and join sessions via BlueJeans (https://www.bluejeans.com). Once enrollment is confirmed via email, please indicate if course attendance will be in person in Ann Arbor) or via BlueJeans.

This course will focus on semi-structured, or in-depth, interviewing, with a brief introduction to other qualitative methods, including observation. Semi-structured interviewing is often most helpful in understanding complex social processes.  We will examine the goals, assumptions, process, and uses of interviewing and compare these methods to other related qualitative and quantitative methods in order to develop research designs appropriate to research goals. The course will cover all aspects of interviewing, including how to decide who to interview, how to ask good interview questions, and how to conduct successful interviews. Students will conduct interviews, and discuss the process and outcome of those interviews. We will examine the strengths and weaknesses of this methodology, particularly through discussion of some of the critiques of these methods.

This course will provide participants with an overview of the primary concepts underlying RSD. This will include discussion of the uncertainty in survey design, the role of paradata, or data describing the data collection process, in informing decisions, and potential RSD interventions. These interventions include timing and sequence of modes, techniques for efficiently deploying incentives, and combining two-phase sampling with other design changes. Interventions appropriate for face-to-face, telephone, web, mail and mixed-mode surveys will be discussed. Using the Total Survey Error (TSE) framework, the main concepts behind these designs will be explained with a focus on how these principles are designed to simultaneously control survey errors and survey costs. Examples of RSD in both large and small studies will be provided as motivation.  Small group exercises will help participants to think through some of the common questions that need to be answered when employing RSD. RSD has financial support available to those who qualify. Not for academic credit.  *Remote participation option:  Students will join sessions via Bluejeans (https://www.bluejeans.com). If you are present in Ann Arbor, it will be possible to attend the course in a classroom.  For more information on this program, please visit the RSD Program web site: https://rsdprogram.si.isr.umich.edu/.

Web surveys can be an inexpensive method for collecting data. This is especially true for designs that repeat measurement over several time periods. However, these relatively low-cost data collections may result in reduced data quality if the problem of nonresponse is ignored. This webinar will examine methods for using RSD to effectively deploy scarce resources in order to minimize the risk of nonresponse bias. Recent experience with the University of Michigan Campus Climate Survey and the National Survey of College Graduates is used to illustrate this point. These surveys are defined by phased designs and multiple modes of contact. This approach produced relatively high response rates and used alternative contact methods in later phases to recruit sample members from subgroups that were less likely to respond in earlier phases. In the case of the UM-CCS all of this was accomplished on a very small budget and with a small management team. Lessons from these experiences can be directly applied in many similar settings. RSD has financial support available to those who qualify. Not for academic credit.  *Remote participation option:  Students will join sessions via Bluejeans (https://www.bluejeans.com). If you are present in Ann Arbor, it will be possible to attend the course in a classroom.  For more information on this program, please visit the RSD Program web site: https://rsdprogram.si.isr.umich.edu/.

This webinar will explore several well-developed examples of RSD. Dr. West will serve as a moderator of the webinar, and also introduce a case study from the National Survey of Family Growth (NSFG). The instructors will then provide independent examples of the implementation of RSD in different international surveys using face-to-face and telephone modes of data collection. All case studies will be supplemented with discussions of issues regarding the development and implementation of RSD. Case studies will include the NSFG, the Relationship Dynamics and Social Life (RDSL) survey, and the Netherlands Survey of Consumer Satisfaction, among others. This variety of case studies will reflect a diversity of survey conditions. The NSFG (West) is a cross-sectional survey that is run on a continuous basis with in-person interviewing. The RDSL (Axinn) is a small-scale panel survey that employed a mixed-mode approach to collecting weekly journal data from a panel of young women. The Netherlands Survey of Consumer Satisfaction (Schouten) is a mixed-mode survey combining web and mail survey data collection with telephone interviewing. The focus of the course will be on practical tools for implementing RSD in a variety of conditions, including small-scale surveys. RSD has financial support available to those who qualify. Not for academic credit.  *Remote participation option:  Students will join sessions via Bluejeans (https://www.bluejeans.com). If you are present in Ann Arbor, it will be possible to attend the course in a classroom.  For more information on this program, please visit the RSD Program web site: https://rsdprogram.si.isr.umich.edu/.

This second part of a two-part webinar series on data quality indicators will give participants a chance to work through hands-on examples of computing and interpreting the data quality indicators introduced in the first part of the series. Example code will be provided and discussed in detail as students are applying it to real production data. RSD has financial support available to those who qualify. Not for academic credit.  *Remote participation option:  Students will join sessions via Bluejeans (https://www.bluejeans.com). If you are present in Ann Arbor, it will be possible to attend the course in a classroom.  For more information on this program, please visit the RSD Program web site: https://rsdprogram.si.isr.umich.edu/.

This first part of a two-part webinar series on data quality indicators will provide an overview of statistical approaches to evaluating data quality. The response rate has been shown to be a poor indicator for data quality with respect to nonresponse bias. Several alternatives have been proposed – the fraction of missing information (FMI), R-Indicators, subgroup response rates, etc. This webinar will explore the use of these indicators as guides for data collection when working within an RSD framework. We also explore optimization techniques that may be useful when designing a survey to maximize these alternative indicators. The consequences of optimizing a survey to other indicators will be explored. We will also consider how the response rate fits into this approach. We will end with a brief discussion of methods for post data collection evaluation of data quality. RSD has financial support available to those who qualify. Not for academic credit.   *Remote participation option:  Students will join sessions via Bluejeans (https://www.bluejeans.com). If you are present in Ann Arbor, it will be possible to attend the course in a classroom.  For more information on this program, please visit the RSD Program web site: https://rsdprogram.si.isr.umich.edu/.

This first webinar in a two-part webinar series on data visualization for production monitoring will cover basic concepts for the design and use of “dashboards” for monitoring survey data collection. We will begin with a detailed discussion of how to design dashboards from an RSD perspective. This will include concrete discussions of how relevant data may be collected and summarized across a variety of production environments. We will also discuss how these dashboards can be used to implement RSD interventions on an ongoing basis. RSD has financial support available to those who qualify. Not for academic credit.  *Remote participation option:  Students will join sessions via Bluejeans (https://www.bluejeans.com). If you are present in Ann Arbor, it will be possible to attend the course in a classroom.  For more information on this program, please visit the RSD Program web site: https://rsdprogram.si.isr.umich.edu/.

This second webinar in a two-part webinar series on data visualization for production monitoring will demonstrate concepts from the first webinar using examples from actual dashboards. We will briefly explore methods for modeling incoming paradata in order to detect outliers. We will then consider practical issues associated with the development of dashboards, including software alternatives. Finally, we will demonstrate how to update dashboards using data reflecting the results of ongoing fieldwork. Participants will be provided with template spreadsheet dashboards for their own applications. RSD has financial support available to those who qualify. Not for academic credit.   *Remote participation option:  Students will join sessions via Bluejeans (https://www.bluejeans.com). If you are present in Ann Arbor, it will be possible to attend the course in a classroom.  For more information on this program, please visit the RSD Program web site: https://rsdprogram.si.isr.umich.edu/.

This first webinar in a two-part series on implementing interventions in a responsive design framework will discuss a variety of potential RSD interventions. Many of these have been implemented experimentally, and the course will include evaluations of those experiments. The importance of experimental evaluations in early phases of RSD will be discussed. Methods for implementing interventions will also be discussed, including implementation of experiments aimed at evaluating new interventions. Strategies for implementing these interventions with both interviewer-mediated and self-administered (e.g., web and mail) surveys will be discussed. Methods for the evaluation of the results of the interventions (experimental and otherwise) will be considered. These evaluations will include measures of both costs and errors. RSD has financial support available to those who qualify. Not for academic credit.   *Remote participation option:  Students will join sessions via Bluejeans (https://www.bluejeans.com). If you are present in Ann Arbor, it will be possible to attend the course in a classroom.  For more information on this program, please visit the RSD Program web site: https://rsdprogram.si.isr.umich.edu/.

This second webinar in a two-part series on implementing interventions in a responsive design framework will walk participants through several real-world examples of interventions that have been applied to real surveys. Participants will also be able to work on small-group exercises designed to develop original interventions in different survey contexts. RSD has financial support available to those who qualify. Not for academic credit.   *Remote participation option:  Students will join sessions via Bluejeans (https://www.bluejeans.com). If you are present in Ann Arbor, it will be possible to attend the course in a classroom.  For more information on this program, please visit the RSD Program web site: https://rsdprogram.si.isr.umich.edu/.

This four-hour webinar will focus on the survey methodology topics most important for understanding the objectives of responsive survey design and its applications. One set of tools will focus on maximizing participation and minimizing attrition of survey participants.  Core survey methodology tools for encouraging participation will be featured.  These tools include incentives, tailoring refusal conversion, switching modes, and tracking strategies. A second set of tools will focus on measurement construction. These tools include mode options, questionnaire design issues, and special instruments (such as life history calendars) to minimize reporting error.  Each portion of the course will feature examples applying each specific tool to various real studies. RSD has financial support available to those who qualify. Not for academic credit. *Remote participation option:  Students will join sessions via Bluejeans (https://www.bluejeans.com). If you are present in Ann Arbor, it will be possible to attend the course in a classroom.  For more information on this program, please visit the RSD Program web site: https://rsdprogram.si.isr.umich.edu/.

This is the first of two webinars that will introduce participants to a general framework for evaluating and maximizing data quality when working with data from a variety of different study designs. In this first webinar, we will introduce a general framework for evaluating total data quality (TDQ), considering concepts related to sampling, nonresponse, measurement, processing, and data analysis. We will then discuss how to apply this framework to different types of data sources, including designed data (such as surveys) and found / organic data (which arise following some organic process, e.g., consumer transactions), focusing on various metrics for evaluating total data quality. RSD has financial support available to those who qualify. Not for academic credit workshop.   *Remote participation option:  Students will join sessions via Bluejeans (https://www.bluejeans.com). If you are present in Ann Arbor, it will be possible to attend the course in a classroom.  For more information on this program, please visit the RSD Program web site: https://rsdprogram.si.isr.umich.edu/.

This is the second of two webinars on the total data quality framework. In this webinar, we will continue our discussion on measuring total data quality. The focus will then turn to tools and techniques for maximizing total data quality (such as responsive and adaptive survey design for designed survey data, weighting approaches, and tools for repairing linkage error). We will present a series of examples considering data from real studies, where the concepts introduced will be applied to vet the total quality of the data sets analyzed. Small-group exercises will be used to give participants hands-on experience with applying some of the concepts discussed to assess data quality. RSD has financial support available to those who qualify. Not for academic credit. *Remote participation option:  Students will join sessions via Bluejeans (https://www.bluejeans.com). If you are present in Ann Arbor, it will be possible to attend the course in a classroom.  For more information on this program, please visit the RSD Program web site: https://rsdprogram.si.isr.umich.edu/.

This course focuses on design and implementation considerations for different phases of the survey lifecycle when conducting surveys internationally or outside of one’s home country. Overview and considerations related to ten topics are discussed: Total Survey Error framework, project stakeholders, triple constraints, bids and contracts, sampling and sample management, questionnaire and instrument design, cross-cultural differences, translation and adaptation, pretesting and cognitive interviews, interviewers and data collection, and interviewer monitoring.  Class will meet for online discussion on July 6, 13, 20 and 27 from 11:00am-12:00pm, Eastern Standard Time.

Inquire at isr-summer@umich.edu if you are interested in taking this course for academic credit

*Remote participation

The Workshop in Sampling Techniques is a component of the Sampling Program for Survey Statisticians. The workshop can only be taken in conjunction with the Methods of Survey Sampling and Analysis of Complex Sample Survey Data courses. The workshop allows students the opportunity to implement methods studied in the companion methods courses such as segmenting and listing in area sampling; selection of a national sample of the U.S.; stratification; controlled selection; telephone sampling; national samples for developing countries; and sampling with computers. The workshop is a required class for the Sampling Program for Survey Statisticians (SPSS). The SPSS is an eight-week program. It consists of three courses: a methods course (SurvMeth 612), a course on the analysis of complex sample survey data (SurvMeth 614), and a hands-on daily workshop (SurvMeth 616). Students enrolled in these three courses are considered Fellows in the Program. The methods and the analysis courses may be taken without being a Fellow. However, the workshop cannot be taken alone. Fellows receive a certificate upon successful completion of the program.

COURSE OBJECTIVES
• Introduce a structural analysis of parts of a survey question
• Introduce cognitive interviewing as a method for testing survey questions
• Describe guidelines for diagnosing problems in survey questions and writing new survey questions
• Focus on the structure and wording of survey questions, whether for interviewer-administered or self- administered instruments
• Provide an opportunity to apply the guidelines and principles during in-class exercises
• Focus on improving individual questions and sets of questions.
• Summarize research that underlies key decisions in writing survey questions.

DESCRIPTION

This workshop distills research about survey questions to principles that can be applied to write survey questions that are clear and obtain reliable answers. The workshop provides students with tools to use in diagnosing problems in survey questions and in writing their own survey questions. Sessions combine lecture with group exercises and discussion. The lecture provides guidelines for writing and revising survey questions and illustrates how to revise troubled questions. Assignments require that students analyze problematic questions, revise them, and administer them to fellow students. Sessions consider both questions about events and behaviors and questions about subjective phenomena (such as attitudes, evaluations, and internal states).

WHO SHOULD ATTEND
Individuals who will be writing or reviewing survey questions or survey instruments or analyzing survey data. This course gives practical guidance to those who have written survey questions but who are not familiar with research on question design, those who are just beginning to design survey instruments, and those who use survey data but do not themselves design survey instruments.