Description of Courses

This short course will offer a very practical introduction to data gathering geared at social scientists and survey researchers.  This course begins with an overview of web scraping discussing some basic technical jargon, types of web data and various methods for scraping.  The course also includes a discussion and illustration of Application Programming Interfaces (APIs) use for gathering web data when they are available.   Some websites are designed to be easily accessible by web crawlers or scraping algorithms while others require much more advanced, custom programming.  And some web data can be accessed using an API that is provided by the website.    In this course we will illustrate how participants can discern these differences as well as presenting several motivating examples of the various ways web scraped data can be used throughout a study’s lifecycle from design to calibration to analysis.  We provide an extensive introduction to a suite of freeware programs that allow virtually syntax free, but customizable, web scraping capabilities.  We contrast this type of gathered data access to APIs for some websites like Zillow or Twitter and discuss pros and cons of using web scraping or APIs to gather this type of web data. The course concludes with specific focus on the import.io tool where we demonstrate its capabilities and provide several, hands-on practical examples for participants to begin scraping several websites of increasing complexity.  We will also illustrate API calls in R for Zillow, the Census and others as time permits.

 

The amount of data generated as a by-product in society is growing fast including data from satellites, sensors, transactions, social media and smartphones, just to name a few. Such data are often referred to as "big data", and can be used to create value in different areas such as health and crime prevention, commerce and fraud detection.  An emerging practice in many areas is to append or link big data sources with more specific and smaller scale sources that often contain much more limited information.  This practice has been used for some time by survey researchers in constructing frames by appending auxiliary information that is often not directly available on the frame, but can be obtained from an external source.   Using Big Data has the potential to go beyond the sampling phase for survey researchers and in fact has the potential to influence the social sciences in general.  Big Data is of interest for public opinion researchers and agencies that produce statistics to find alternative data sources either to reduce costs, to improve estimates or to produce estimates in a more timely fashion. However, Big Data pose several interesting and new challenges to survey researchers and social scientists among others who want to extract information from data. As Robert Groves (2012) pointedly commented, the era is “appropriately called Big Data and not Big Information”, because there is a lot of work for analysts before information can be gained from “auxiliary traces of some process that is going on in society.”

This course offers participants a broad overview of big data sources, opportunities and examples motivated within the survey and social science contexts including the use of social media data, para data and other such sources.  This course also offers a detailed, practical introduction to four common machine learning methods that can be applied to big and small data alike at various aspects of a study’s lifecycle from design to nonresponse adjustments to propensity score matching to weighting and evaluation and analysis.  The machine learning methods will be demonstrated in R and we will provide several different examples of using these methods along with multiple packages in R that offer these methods.

 

This course provides practical methods and tools to analyze complex survey data with a hands-on introduction to the use of specialized statistical software procedures. The course focuses on case studies with specific large-scale national surveys: the National Comorbidity Survey-Replication (NCS-R), the National Health and Nutrition Examination Surveys (NHANES), and the Health and Retirement Study (HRS). Relevant design features of the NCS-R, NHANES and HRS include survey weights that take into account differences in probability of selection into the sample and differences in response rates, as well as stratification and clustering in the multistage sampling procedures used in identifying the sampled households and individuals. After introducing essential concepts related to complex sample designs, the course will turn to the construction of survey weights, estimation of sampling variance, descriptive analysis, regression analysis, and finally special topics in the analysis of survey data. Participants can expect to work on homework exercises, computer lab exercises, and a final analysis project.

Why take this course? 

• To gain an understanding of modern methods and software for the secondary analysis of survey data collected from large complex samples
• To have the opportunity for one-on-one interaction with the instructors when walking through analyses of survey data
• To see various examples of applied statistical analyses of survey data
• To have the experience of writing a scientific paper that presents an analysis of complex sample survey data, and getting expert feedback on that paper

The recent proliferation of mobile technology allows researchers to collect objective health and behavioral data at increased intervals, in real time, and may also reduce participant burden. In this course, we will provide examples of the utility of and integration of wearables, sensors, and apps in research settings. Examples will include the use of wearable health devices to measure activity, apps for ecological momentary assessment, and smartphone sensors to measure sound and movement, among others. Additionally, this course will consider the integration of these new technologies into existing surveys and the quality of the data collected from the total survey error perspective. We will discuss considerations for assessing coverage, participation, and measurement error when integrating wearables, sensors, and apps in a research setting as well as the costs and privacy considerations when collecting these types of data. Participants will work in groups to discuss a research study design using new technology and have the opportunity for hands-on practice with sensor data.

The Health and Retirement Study (hrsonline.isr.umich.edu) Summer Workshop is intended to give participants an introduction to the study that will enable them to use the data for research. HRS is a large-scale longitudinal study with more than 20 years of data on the labor force participation and health transitions that individuals undergo toward the end of their work lives and in the years that follow.

The 5-day workshop will be online again this year. Content lectures delivered by HRS co-investigators and content area experts on basic survey content, sample design, weighting, and restricted data files will be available on the course website for viewing any time. In addition, each content lecturer will participate in a Zoom meeting with the class to answer questions about their lecture. The majority of each day will be devoted to data training workshops in which participants learn to work with the data including the user-friendly RAND version of the HRS data and the Gateway to Global Aging project data, which harmonizes data across HRS international sister studies.

The data training portion assumes some familiarity with SAS or STATA.

This one-week course introduces students to the design and implementation of online survey data collection instruments. The course is both hands-on and conceptual. It begins by discussing what is unique about web surveys and when their use is most appropriate, followed by an introduction to survey errors that can affect the quality of web survey data. Small groups of students will each develop a research problem and a questionnaire to address their problem, designed for online administration. They will pretest the question wording, program the questionnaire using a web survey development platform (no programming experience is required), and assess users’ (respondents’) experience while interacting with the web-based instrument. Students will also develop basic plans for data collection and analysis. Finally, each group will present its problem, online questionnaire, evaluation, and plans to the rest of the class.  

Why take this course?

· To gain an understanding of what should go into creating a web-based questionnaire

· To gain experience weighing the pros and cons of different web questionnaire features

· To gain experience building a web questionnaire on a standard platform

· To gain experience evaluating survey questions and their usability in an online questionnaire

Participants should have prior knowledge of question design before attending.

This course is designed to follow on from Introduction to Questionnaire Design. Now instead of looking at question comprehension from a cognitive side, the linguistic side will be explored including online tools. Factual questions will be revisited but with the goal of exploring different types of respondent memory problems and their solutions, while also covering time anomalies in surveys and quasi facts. Subjective questions will be revisited to understand attitude consistency and inconsistency, to look at the feasibility of changing attitudes to change behavioral intentions to change behaviors and to cover the popular topic of satisfaction and other customer experience metrics. Alternatives to questionnaires will also be covered such as event history calendars, internet enabled devices, factorial surveys and multi-item scales. The course concludes with ways to translate survey questions and evaluate the translation. The course will be interactive with the goal of making it as close to in-person training as possible. There also will be workshops throughout. Pamela is happy to chat with participants about their own questionnaires.

This course introduces the skills needed to conduct focus group interviews. Students will learn about the critical components of successful focus group research. They will develop a plan for a focus group study and then practice key skills and receive individualized feedback from instructors. Attention will be placed on recent developments in conducting online focus groups (e.g., using Zoom as a platform) as well as an overview of in-person focus group interviews. This course will be particularly applicable for those conducting focus group research in academic, non-profit, and government settings.

Course Topic

The course will cover these skills:

Conducting in-person and virtual focus groups

How to plan and design a focus group study

Identifying information-rich participants and getting them to show up

Beginning the focus group - the crucial first few minutes and moderating techniques

Developing questions—Characteristics of good focus group questions

Analyzing—Options for analysis

Course Format

The course will be presented online on Zoom over a period of 2 days.

Why Take This Course?

Focus groups are used to understand issues, pilot test ideas, evaluate programs, and to uncover underlying emotions and processes driving human behaviors and choices. They also provide great insight when used in combination with surveys. Focus groups have been used to help design surveys, to pilot test surveys, and to understand survey findings. Take this course if you want to learn more about how focus groups might add to your research toolbox.

This course will begin to empower students with an understanding of the importance and basic tenets of rigorous questionnaire design, as well as practice designing an appropriate instrument for a real world problem. Students will watch course videos independently, and work on a questionnaire for a topic of their choosing. Four live online meetings (Tuesdays and Thursdays from 2-3:30 PM EST) will take a workshop format where students will ask questions, share their own questionnaires in progress, and give feedback to classmates.

This course covers the basic principles of survey design and methods and introduces the necessary components of a good quality survey.   The course employs the Total Survey Error framework to discuss sampling frames and designs, modes of data collection and their effects on survey errors, the cognitive processes involved in answering survey questions and their impact on questionnaire design, pretesting methods and post-data collection processing.  The goal of the course is to give an introduction to the skills and resources needed to design and conduct a survey

This is a foundation course in sample survey methods and principles. The instructors will present, in a non-technical manner, basic sampling techniques such as simple random sampling, systematic sampling, stratification, and cluster sampling. The instructors will provide opportunities to implement sampling techniques in a series of exercises that accompany each topic.

Participants should not expect to obtain sufficient background in this course to master survey sampling. They can expect to become familiar with basic techniques well enough to converse with sampling statisticians more easily about sample design.

The Health and Retirement Study (hrsonline.isr.umich.edu) Summer Workshop is intended to give participants an introduction to the study that will enable them to use the data for research. HRS is a large-scale longitudinal study with more than 20 years of data on the labor force participation and health transitions that individuals undergo toward the end of their work lives and in the years that follow.

The 5-day workshop will be online again this year. Content lectures delivered by HRS co-investigators and content area experts on basic survey content, sample design, weighting, and restricted data files will be available on the course website for viewing any time. In addition, each content lecturer will participate in a Zoom meeting with the class to answer questions about their lecture. The majority of each day will be devoted to data training workshops in which participants learn to work with the data including the user-friendly RAND version of the HRS data and the Gateway to Global Aging project data, which harmonizes data across HRS international sister studies.

The data training portion assumes some familiarity with SAS or STATA.

A fundamental feature of many sample surveys is a probability sample of subjects. Probability sampling requires rigorous application of mathematical principles to the selection process. Methods of Survey Sampling is a moderately advanced course in applied statistics, with an emphasis on the practical problems of sample design, which provides students with an understanding of principles and practice in skills required to select subjects and analyze sample data. Topics covered include stratified, clustered, systematic, and multi-stage sample designs, unequal probabilities and probabilities proportional to size, area probability sampling, ratio means, sampling errors, frame problems, cost factors, and practical designs and procedures. Emphasis is on practical considerations rather than on theoretical derivations, although understanding of principles requires review of statistical results for sample surveys. The course includes an exercise that integrates the different techniques into a comprehensive sample design.

Why take this course? 

• To understand the basic ideas, concepts and principles of probability sampling from an applied perspective
• To be able to identify and appropriately apply sampling techniques to survey design problems
• To be able to compute the sample size for a variety of sample designs
• To understand and be able to assess the impact of the sample design on survey estimates
• To learn how to design and select a probability sample involving complex sampling techniques in a survey project, and receive expert feedback on a sampling report

This course will focus on semi-structured, or in-depth, interviewing, with a brief introduction to other qualitative methods, including observation. Semi-structured interviewing is often most helpful in understanding complex social processes.  We will examine the goals, assumptions, process, and uses of interviewing and compare these methods to other related qualitative and quantitative methods in order to develop research designs appropriate to research goals. The course will cover all aspects of interviewing, including how to decide who to interview, how to ask good interview questions, and how to conduct successful interviews. Students will conduct interviews, and discuss the process and outcome of those interviews. We will examine the strengths and weaknesses of this methodology, particularly through discussion of some of the critiques of these methods.

This course will provide participants with an overview of the primary concepts underlying RSD. This will include discussion of the uncertainty in survey design, the role of paradata, or data describing the data collection process, in informing decisions, and potential RSD interventions. These interventions include timing and sequence of modes, techniques for efficiently deploying incentives, and combining two-phase sampling with other design changes. Interventions appropriate for face-to-face, telephone, web, mail and mixed-mode surveys will be discussed. Using the Total Survey Error (TSE) framework, the main concepts behind these designs will be explained with a focus on how these principles are designed to simultaneously control survey errors and survey costs. Examples of RSD in both large and small studies will be provided as motivation.  Small group exercises will help participants to think through some of the common questions that need to be answered when employing RSD. RSD has financial support available to those who qualify. Not for academic credit. 

Web surveys can be an inexpensive method for collecting data. This is especially true for designs that repeat measurement over several time periods. However, these relatively low-cost data collections may result in reduced data quality if the problem of nonresponse is ignored. This webinar will examine methods for using RSD to effectively deploy scarce resources in order to minimize the risk of nonresponse bias. Recent experiences with the University of Michigan Campus Climate Survey (UM-CCS), the National Survey of College Graduates (NSCG), and the Residential Energy Consumption Survey (RECS) are used to illustrate this point. These surveys are all defined by phased designs and multiple modes of contact. This approach improves survey outcomes, including response rates, representativeness, and cost by using alternative contact methods in later phases to recruit sample members from subgroups that were less likely to respond in earlier phases. These surveys demonstrate the benefit of RSD in web surveys across a variety of different samples sizes, and both small and large budgets and management teams. As a result, lessons from these experiences can be directly applied in many similar settings. RSD has financial support available to those who qualify. Not for academic credit. 

This webinar will explore several well-developed examples of RSD. Dr. West will serve as a moderator of the webinar, and also introduce a case study from the National Survey of Family Growth (NSFG). The instructors will then provide independent examples of the implementation of RSD in different international surveys using face-to-face and telephone modes of data collection. All case studies will be supplemented with discussions of issues regarding the development and implementation of RSD. Case studies will include the NSFG, the Relationship Dynamics and Social Life (RDSL) survey, and the Netherlands Survey of Consumer Satisfaction, among others. This variety of case studies will reflect a diversity of survey conditions. The NSFG (West) is a cross-sectional survey that is run on a continuous basis with in-person interviewing. The RDSL (West) is a small-scale panel survey that employed a mixed-mode approach to collecting weekly journal data from a panel of young women. The Netherlands Survey of Consumer Satisfaction (Schouten) is a mixed-mode survey combining web and mail survey data collection with telephone interviewing. The National Longitudinal Study of Adolescent to Adult Health (AddHealth; Murphy) employs adaptive design in a longitudinal framework, using web, mail, telephone, and face-to-face modes of data collection.  The focus of the course will be on practical tools for implementing RSD in a variety of conditions, including small-scale surveys. RSD has financial support available to those who qualify. Not for academic credit. 

This second part of a two-part webinar series on data quality indicators will give participants a chance to work through hands-on examples of computing and interpreting the data quality indicators introduced in the first part of the series. Example code will be provided and discussed in detail as students are applying it to real production data. RSD has financial support available to those who qualify. Not for academic credit. 

This first part of a two-part webinar series on data quality indicators will provide an overview of statistical approaches to evaluating data quality. The response rate has been shown to be a poor indicator for data quality with respect to nonresponse bias. Several alternatives have been proposed – the fraction of missing information (FMI), R-Indicators, subgroup response rates, etc. This webinar will explore the use of these indicators as guides for data collection when working within an RSD framework. We also explore optimization techniques that may be useful when designing a survey to maximize these alternative indicators. The consequences of optimizing a survey to other indicators will be explored. We will also consider how the response rate fits into this approach. We will end with a brief discussion of methods for post data collection evaluation of data quality. RSD has financial support available to those who qualify. Not for academic credit.  

This first webinar in a two-part webinar series on data visualization for production monitoring will cover basic concepts for the design and use of “dashboards” for monitoring survey data collection. We will begin with a detailed discussion of how to design dashboards from an RSD perspective. This will include concrete discussions of how relevant data may be collected and summarized across a variety of production environments. We will also discuss how these dashboards can be used to implement RSD interventions on an ongoing basis. RSD has financial support available to those who qualify. Not for academic credit. 

This second webinar in a two-part webinar series on data visualization for production monitoring will demonstrate concepts from the first webinar using examples from actual dashboards. We will briefly explore methods for modeling incoming paradata in order to detect outliers. We will then consider practical issues associated with the development of dashboards, including software alternatives. Finally, we will demonstrate how to update dashboards using data reflecting the results of ongoing fieldwork. Participants will be provided with template spreadsheet dashboards for their own applications. RSD has financial support available to those who qualify. Not for academic credit.  

This first webinar in a two-part series on implementing interventions in a responsive design framework will discuss a variety of potential RSD interventions. Many of these have been implemented experimentally, and the course will include evaluations of those experiments. The importance of experimental evaluations in early phases of RSD will be discussed. Methods for implementing interventions will also be discussed, including implementation of experiments aimed at evaluating new interventions. Strategies for implementing these interventions with both interviewer-mediated and self-administered (e.g., web and mail) surveys will be discussed. Methods for the evaluation of the results of the interventions (experimental and otherwise) will be considered. These evaluations will include measures of both costs and errors. RSD has financial support available to those who qualify. Not for academic credit.  

This second webinar in a two-part series on implementing interventions in a responsive design framework will walk participants through several real-world examples of interventions that have been applied to real surveys. Participants will also be able to work on small-group exercises designed to develop original interventions in different survey contexts. RSD has financial support available to those who qualify. Not for academic credit.  

This four-hour webinar will focus on the survey methodology topics most important for understanding the objectives of responsive survey design and its applications. One set of tools will focus on maximizing participation and minimizing attrition of survey participants.  Core survey methodology tools for encouraging participation will be featured.  These tools include incentives, tailoring refusal conversion, switching modes, and tracking strategies. A second set of tools will focus on measurement construction. These tools include mode options, questionnaire design issues, and special instruments (such as life history calendars) to minimize reporting error.  Each portion of the course will feature examples applying each specific tool to various real studies. RSD has financial support available to those who qualify. Not for academic credit.

This is the first of two webinars that will introduce participants to a general framework for evaluating and maximizing data quality when working with data from a variety of different study designs. In this first webinar, we will introduce a general framework for evaluating total data quality (TDQ), considering concepts related to sampling, nonresponse, measurement, processing, and data analysis. We will then discuss how to apply this framework to different types of data sources, including designed data (such as surveys) and found / organic data (which arise following some organic process, e.g., consumer transactions), focusing on various metrics for evaluating total data quality. RSD has financial support available to those who qualify. Not for academic credit workshop.  

This is the second of two webinars on the total data quality framework. In this webinar, we will continue our discussion on measuring total data quality. The focus will then turn to tools and techniques for maximizing total data quality (such as responsive and adaptive survey design for designed survey data, weighting approaches, and tools for repairing linkage error). We will present a series of examples considering data from real studies, where the concepts introduced will be applied to vet the total quality of the data sets analyzed. Small-group exercises will be used to give participants hands-on experience with applying some of the concepts discussed to assess data quality. RSD has financial support available to those who qualify. Not for academic credit.

This course focuses on design and implementation considerations for different phases of the survey lifecycle when conducting surveys internationally or outside of one’s home country. Overview and considerations related to ten topics are discussed: Total Survey Error framework, project stakeholders, triple constraints, bids and contracts, sampling and sample management, questionnaire and instrument design, cross-cultural differences, translation and adaptation, pretesting and cognitive interviews, interviewers and data collection, and interviewer monitoring.  Class will meet for online discussion on June 30, July 7, 14 and 21 from 11:00am-12:00pm, Eastern Standard Time.

Inquire at isr-summer@umich.edu if you are interested in taking this course for academic credit

*Remote participation

This course is intended as an introduction to the science behind survey research and will be taught at an undergraduate level. Our primary reason for designing this course is to share our excitement about survey methodology as a research field. In this course, we focus on principles and theories as opposed to a strictly hands on approach to creating surveys. This approach to the study of surveys will allow you to apply the same methods to a very broad range of research topics, e.g., public opinion and politics, health, drug use, consumer behavior, customer satisfaction and market research among many others. In addition, the course will present basic statistical concepts and techniques in sample design, data collection, and reporting, as well as explanations of how error can be introduced into the survey process. At the end of this course, you will have a basic understanding of what needs to go into conducting a high-quality survey and will be exposed to resources that will allow you build on the knowledge of survey methodology you have learned in the course.

The Workshop in Sampling Techniques is a component of the Sampling Program for Survey Statisticians. The workshop can only be taken in conjunction with the Methods of Survey Sampling and Analysis of Complex Sample Survey Data courses. The workshop allows students the opportunity to implement methods studied in the companion methods courses such as segmenting and listing in area sampling; selection of a national sample of the U.S.; stratification; controlled selection; telephone sampling; national samples for developing countries; and sampling with computers.

The workshop is a required class for the Sampling Program for Survey Statisticians (SPSS). The SPSS is an eight-week program. It consists of three courses: a methods course (SurvMeth 612), a course on the analysis of complex sample survey data (SurvMeth 614), and a hands-on daily workshop (SurvMeth 616). Students enrolled in these three courses are considered Fellows in the Program. The methods and the analysis courses may be taken without being a Fellow. However, the workshop cannot be taken alone. Fellows receive a certificate upon successful completion of the program.

COURSE OBJECTIVES
• Introduce a structural analysis of parts of a survey question
• Introduce cognitive interviewing as a method for testing survey questions
• Describe guidelines for diagnosing problems in survey questions and writing new survey questions
• Focus on the structure and wording of survey questions, whether for interviewer-administered or self- administered instruments
• Provide an opportunity to apply the guidelines and principles during in-class exercises
• Focus on improving individual questions and sets of questions.
• Summarize research that underlies key decisions in writing survey questions.

DESCRIPTION

This workshop distills research about survey questions to principles that can be applied to write survey questions that are clear and obtain reliable answers. The workshop provides students with tools to use in diagnosing problems in survey questions and in writing their own survey questions. Sessions combine lecture with group exercises and discussion. The lecture provides guidelines for writing and revising survey questions and illustrates how to revise troubled questions. Assignments require that students analyze problematic questions, revise them, and administer them to fellow students. Sessions consider both questions about events and behaviors and questions about subjective phenomena (such as attitudes, evaluations, and internal
states).

WHO SHOULD ATTEND
Individuals who will be writing or reviewing survey questions or survey instruments or analyzing survey data. This course gives practical guidance to those who have written survey questions but who are not familiar with research on question design, those who are just beginning to design survey instruments, and those who use survey data but do not themselves design survey instruments.