This short course will offer a very practical introduction to data gathering geared at social scientists and survey researchers. This course begins with an overview of web scraping discussing some basic technical jargon, types of web data and various methods for scraping. The course also includes a discussion and illustration of Application Programming Interfaces (APIs) use for gathering web data when they are available. Some websites are designed to be easily accessible by web crawlers or scraping algorithms while others require much more advanced, custom programming. And some web data can be accessed using an API that is provided by the website. In this course we will illustrate how participants can discern these differences as well as presenting several motivating examples of the various ways web scraped data can be used throughout a study’s lifecycle from design to calibration to analysis. We provide an extensive introduction to a suite of freeware programs that allow virtually syntax free, but customizable, web scraping capabilities. We contrast this type of gathered data access to APIs for some websites like Zillow or Twitter and discuss pros and cons of using web scraping or APIs to gather this type of web data. The course concludes with specific focus on the import.io tool where we demonstrate its capabilities and provide several, hands-on practical examples for participants to begin scraping several websites of increasing complexity. We will also illustrate API calls in R for Zillow, the Census and others as time permits.
Prerequisite: To take this class for UM credit you must take SurvMeth 988.204-A and 988.204-B, An Introduction to Big Data and Machine Learning for Survey Researchers and Social Scientists for a total of 1.0 credit hour.Having a trial import.io account set up (this is a 7 day trial so please plan to have the license active during our course). Details can be found here: https://www.import.io/signup/.
The amount of data generated as a by-product in society is growing fast including data from satellites, sensors, transactions, social media and smartphones, just to name a few. Such data are often referred to as "big data", and can be used to create value in different areas such as health and crime prevention, commerce and fraud detection. An emerging practice in many areas is to append or link big data sources with more specific and smaller scale sources that often contain much more limited information. This practice has been used for some time by survey researchers in constructing frames by appending auxiliary information that is often not directly available on the frame, but can be obtained from an external source. Using Big Data has the potential to go beyond the sampling phase for survey researchers and in fact has the potential to influence the social sciences in general. Big Data is of interest for public opinion researchers and agencies that produce statistics to find alternative data sources either to reduce costs, to improve estimates or to produce estimates in a more timely fashion. However, Big Data pose several interesting and new challenges to survey researchers and social scientists among others who want to extract information from data. As Robert Groves (2012) pointedly commented, the era is “appropriately called Big Data and not Big Information”, because there is a lot of work for analysts before information can be gained from “auxiliary traces of some process that is going on in society.”
This course offers participants a broad overview of big data sources, opportunities and examples motivated within the survey and social science contexts including the use of social media data, para data and other such sources. This course also offers a detailed, practical introduction to four common machine learning methods that can be applied to big and small data alike at various aspects of a study’s lifecycle from design to nonresponse adjustments to propensity score matching to weighting and evaluation and analysis. The machine learning methods will be demonstrated in R and we will provide several different examples of using these methods along with multiple packages in R that offer these methods.
Prerequisite: To take this class for UM credit you must take SurvMeth 988.204-B and 988.204-A, A Virtually Syntax Free Practical Introduction to Web Scraping for Survey and Social Science Researchers for a total of 1.0 credit hours.Basic proficency in R (i.e. how to load a package, launch it and basic R syntax knowledge)
This course provides practical methods and tools to analyze complex survey data with a hands-on introduction to the use of specialized statistical software procedures. The course focuses on case studies with specific large-scale national surveys: the National Comorbidity Survey-Replication (NCS-R), the National Health and Nutrition Examination Surveys (NHANES), and the Health and Retirement Study (HRS). Relevant design features of the NCS-R, NHANES and HRS include survey weights that take into account differences in probability of selection into the sample and differences in response rates, as well as stratification and clustering in the multistage sampling procedures used in identifying the sampled households and individuals. After introducing essential concepts related to complex sample designs, the course will turn to the construction of survey weights, estimation of sampling variance, descriptive analysis, regression analysis, and finally special topics in the analysis of survey data. Participants can expect to work on homework exercises, computer lab exercises, and a final analysis project.
Why take this course?
To gain an understanding of modern methods and software for the secondary analysis of survey data collected from large complex samples
To have the opportunity for one-on-one interaction with the instructors when walking through analyses of survey data
To see various examples of applied statistical analyses of survey data
To have the experience of writing a scientific paper that presents an analysis of complex sample survey data, and getting expert feedback on that paper
Prerequisite: Two graduate-level courses in statistical methods, familiarity with basic sample design concepts, and familiarity with data analytic techniques such as linear and logistic regression.
The recent proliferation of mobile technology allows researchers to collect objective health and behavioral data at increased intervals, in real time, and may also reduce participant burden. In this course, we will provide examples of the utility of and integration of wearables, sensors, and apps in research settings. Examples will include the use of wearable health devices to measure activity, apps for ecological momentary assessment, and smartphone sensors to measure sound and movement, among others. Additionally, this course will consider the integration of these new technologies into existing surveys and the quality of the data collected from the total survey error perspective. We will discuss considerations for assessing coverage, participation, and measurement error when integrating wearables, sensors, and apps in a research setting as well as the costs and privacy considerations when collecting these types of data. Participants will work in groups to discuss a research study design using new technology and have the opportunity for hands-on practice with sensor data.
This one-week course introduces students to the design and implementation of online survey data collection instruments. The course is both hands-on and conceptual. It begins by discussing what is unique about web surveys and when their use is most appropriate, followed by an introduction to survey errors that can affect the quality of web survey data. Small groups of students will each develop a research problem and a questionnaire to address their problem, designed for online administration. They will pretest the question wording, program the questionnaire using a web survey development platform (no programming experience is required), and assess users’ (respondents’) experience while interacting with the web-based instrument. Students will also develop basic plans for data collection and analysis. Finally, each group will present its problem, online questionnaire, evaluation, and plans to the rest of the class.
Why take this course?
· To gain an understanding of what should go into creating a web-based questionnaire
· To gain experience weighing the pros and cons of different web questionnaire features
· To gain experience building a web questionnaire on a standard platform
· To gain experience evaluating survey questions and their usability in an online questionnaire
Participants should have prior knowledge of question design before attending.
This course is designed to follow on from Introduction to Questionnaire Design. Now instead of looking at question comprehension from a cognitive side, the linguistic side will be explored including online tools. Factual questions will be revisited but with the goal of exploring different types of respondent memory problems and their solutions, while also covering time anomalies in surveys and quasi facts. Subjective questions will be revisited to understand attitude consistency and inconsistency, to look at the feasibility of changing attitudes to change behavioral intentions to change behaviors and to cover the popular topic of satisfaction and other customer experience metrics. Alternatives to questionnaires will also be covered such as event history calendars, internet enabled devices, factorial surveys and multi-item scales. The course concludes with ways to translate survey questions and evaluate the translation. The course will be interactive with the goal of making it as close to in-person training as possible. There also will be workshops throughout. Pamela is happy to chat with participants about their own questionnaires.
This course will begin to empower students with an understanding of the importance and basic tenets of rigorous questionnaire design, as well as practice designing an appropriate instrument for a real world problem. Students will watch course videos independently, and work on a questionnaire for a topic of their choosing. Four live online meetings (Tuesdays and Thursdays from 2-3:30 PM EST) will take a workshop format where students will ask questions, share their own questionnaires in progress, and give feedback to classmates.
This course covers the basic principles of survey design and methods and introduces the necessary components of a good quality survey. The course employs the Total Survey Error framework to discuss sampling frames and designs, modes of data collection and their effects on survey errors, the cognitive processes involved in answering survey questions and their impact on questionnaire design, pretesting methods and post-data collection processing. The goal of the course is to give an introduction to the skills and resources needed to design and conduct a survey
The Health and Retirement Study (hrsonline.isr.umich.edu) Summer Workshop is intended to give participants an introduction to the study that will enable them to use the data for research. HRS is a large-scale longitudinal study with more than 20 years of data on the labor force participation and health transitions that individuals undergo toward the end of their work lives and in the years that follow. The HRS Summer Workshop features morning lectures on basic survey content, sample design, weighting, and restricted data files. Hands-on data workshops are held every afternoon in which participants learn to work with the data (including the user-friendly RAND version of the HRS data) under the guidance of HRS staff. Staff of the Gateway to Global Aging project (G2Aging.org), which harmonizes data across HRS international sister studies, conduct an afternoon training. At the end of the week, students have the opportunity to present their research ideas to the class and HRS research faculty and obtain feedback. Topics include (but are not limited to) in depth information on HRS data about health insurance and medical care; biomarkers, physical measures, and genetic data; cognition; health and physical functioning; linkage to Medicare; employment, retirement, and pensions and linkage to Social Security records; psychosocial and well-being; family data; and international comparison data. The data training portion assumes some familiarity with SAS or STATA.
A fundamental feature of many sample surveys is a probability sample of subjects. Probability sampling requires rigorous application of mathematical principles to the selection process. Methods of Survey Sampling is a moderately advanced course in applied statistics, with an emphasis on the practical problems of sample design, which provides students with an understanding of principles and practice in skills required to select subjects and analyze sample data. Topics covered include stratified, clustered, systematic, and multi-stage sample designs, unequal probabilities and probabilities proportional to size, area probability sampling, ratio means, sampling errors, frame problems, cost factors, and practical designs and procedures. Emphasis is on practical considerations rather than on theoretical derivations, although understanding of principles requires review of statistical results for sample surveys. The course includes an exercise that integrates the different techniques into a comprehensive sample design.
Why take this course?
To understand the basic ideas, concepts and principles of probability sampling from an applied perspective
To be able to identify and appropriately apply sampling techniques to survey design problems
To be able to compute the sample size for a variety of sample designs
To understand and be able to assess the impact of the sample design on survey estimates
To learn how to design and select a probability sample involving complex sampling techniques in a survey project, and receive expert feedback on a sampling report
Prerequisite: Two graduate-level courses in statistical methods.
This course will focus on semi-structured, or in-depth, interviewing, with a brief introduction to other qualitative methods, including observation. Semi-structured interviewing is often most helpful in understanding complex social processes. We will examine the goals, assumptions, process, and uses of interviewing and compare these methods to other related qualitative and quantitative methods in order to develop research designs appropriate to research goals. The course will cover all aspects of interviewing, including how to decide who to interview, how to ask good interview questions, and how to conduct successful interviews. Students will conduct interviews, and discuss the process and outcome of those interviews. We will examine the strengths and weaknesses of this methodology, particularly through discussion of some of the critiques of these methods.
This course will provide participants with an overview of the primary concepts underlying RSD. This will include discussion of the uncertainty in survey design, the role of paradata, or data describing the data collection process, in informing decisions, and potential RSD interventions. These interventions include timing and sequence of modes, techniques for efficiently deploying incentives, and combining two-phase sampling with other design changes. Interventions appropriate for face-to-face, telephone, web, mail and mixed-mode surveys will be discussed. Using the Total Survey Error (TSE) framework, the main concepts behind these designs will be explained with a focus on how these principles are designed to simultaneously control survey errors and survey costs. Examples of RSD in both large and small studies will be provided as motivation. Small group exercises will help participants to think through some of the common questions that need to be answered when employing RSD.
RSD has financial support available to those who qualify.
Not for academic credit.
Web surveys can be an inexpensive method for collecting data. This is especially true for designs that repeat measurement over several time periods. However, these relatively low-cost data collections may result in reduced data quality if the problem of nonresponse is ignored. This webinar will examine methods for using RSD to effectively deploy scarce resources in order to minimize the risk of nonresponse bias. Recent experience with the University of Michigan Campus Climate Survey and the National Survey of College Graduates is used to illustrate this point. These surveys are defined by phased designs and multiple modes of contact. This approach produced relatively high response rates and used alternative contact methods in later phases to recruit sample members from subgroups that were less likely to respond in earlier phases. In the case of the UM-CCS all of this was accomplished on a very small budget and with a small management team. Lessons from these experiences can be directly applied in many similar settings.
RSD has financial support available to those who qualify.
Not for academic credit.
This webinar will explore several well-developed examples of RSD. Dr. West will serve as a moderator of the webinar, and also introduce a case study from the National Survey of Family Growth (NSFG). The instructors will then provide independent examples of the implementation of RSD in different international surveys using face-to-face and telephone modes of data collection. All case studies will be supplemented with discussions of issues regarding the development and implementation of RSD. Case studies will include the NSFG, the Relationship Dynamics and Social Life (RDSL) survey, and the Netherlands Survey of Consumer Satisfaction, among others. This variety of case studies will reflect a diversity of survey conditions. The NSFG (West) is a cross-sectional survey that is run on a continuous basis with in-person interviewing. The RDSL (West) is a small-scale panel survey that employed a mixed-mode approach to collecting weekly journal data from a panel of young women. The Netherlands Survey of Consumer Satisfaction (Schouten) is a mixed-mode survey combining web and mail survey data collection with telephone interviewing. The National Longitudinal Study of Adolescent to Adult Health (AddHealth; Murphy) employs adaptive design in a longitudinal framework, using web, mail, telephone, and face-to-face modes of data collection. The focus of the course will be on practical tools for implementing RSD in a variety of conditions, including small-scale surveys.
RSD has financial support available to those who qualify.
Not for academic credit.
This second part of a two-part webinar series on data quality indicators will give participants a chance to work through hands-on examples of computing and interpreting the data quality indicators introduced in the first part of the series. Example code will be provided and discussed in detail as students are applying it to real production data.
RSD has financial support available to those who qualify.
Not for academic credit.
This first part of a two-part webinar series on data quality indicators will provide an overview of statistical approaches to evaluating data quality. The response rate has been shown to be a poor indicator for data quality with respect to nonresponse bias. Several alternatives have been proposed – the fraction of missing information (FMI), R-Indicators, subgroup response rates, etc. This webinar will explore the use of these indicators as guides for data collection when working within an RSD framework. We also explore optimization techniques that may be useful when designing a survey to maximize these alternative indicators. The consequences of optimizing a survey to other indicators will be explored. We will also consider how the response rate fits into this approach. We will end with a brief discussion of methods for post data collection evaluation of data quality.
RSD has financial support available to those who qualify.
Not for academic credit.
This first webinar in a two-part webinar series on data visualization for production monitoring will cover basic concepts for the design and use of “dashboards” for monitoring survey data collection. We will begin with a detailed discussion of how to design dashboards from an RSD perspective. This will include concrete discussions of how relevant data may be collected and summarized across a variety of production environments. We will also discuss how these dashboards can be used to implement RSD interventions on an ongoing basis.
RSD has financial support available to those who qualify.
Not for academic credit.
This second webinar in a two-part webinar series on data visualization for production monitoring will demonstrate concepts from the first webinar using examples from actual dashboards. We will briefly explore methods for modeling incoming paradata in order to detect outliers. We will then consider practical issues associated with the development of dashboards, including software alternatives. Finally, we will demonstrate how to update dashboards using data reflecting the results of ongoing fieldwork. Participants will be provided with template spreadsheet dashboards for their own applications.
RSD has financial support available to those who qualify.
Not for academic credit.
This first webinar in a two-part series on implementing interventions in a responsive design framework will discuss a variety of potential RSD interventions. Many of these have been implemented experimentally, and the course will include evaluations of those experiments. The importance of experimental evaluations in early phases of RSD will be discussed. Methods for implementing interventions will also be discussed, including implementation of experiments aimed at evaluating new interventions. Strategies for implementing these interventions with both interviewer-mediated and self-administered (e.g., web and mail) surveys will be discussed. Methods for the evaluation of the results of the interventions (experimental and otherwise) will be considered. These evaluations will include measures of both costs and errors.
RSD has financial support available to those who qualify.
Not for academic credit.
This second webinar in a two-part series on implementing interventions in a responsive design framework will walk participants through several real-world examples of interventions that have been applied to real surveys. Participants will also be able to work on small-group exercises designed to develop original interventions in different survey contexts.
RSD has financial support available to those who qualify.
Not for academic credit.
This four-hour webinar will focus on the survey methodology topics most important for understanding the objectives of responsive survey design and its applications. One set of tools will focus on maximizing participation and minimizing attrition of survey participants. Core survey methodology tools for encouraging participation will be featured. These tools include incentives, tailoring refusal conversion, switching modes, and tracking strategies. A second set of tools will focus on measurement construction. These tools include mode options, questionnaire design issues, and special instruments (such as life history calendars) to minimize reporting error. Each portion of the course will feature examples applying each specific tool to various real studies.
RSD has financial support available to those who qualify.
Not for academic credit.
This is the first of two webinars that will introduce participants to a general framework for evaluating and maximizing data quality when working with data from a variety of different study designs. In this first webinar, we will introduce a general framework for evaluating total data quality (TDQ), considering concepts related to sampling, nonresponse, measurement, processing, and data analysis. We will then discuss how to apply this framework to different types of data sources, including designed data (such as surveys) and found / organic data (which arise following some organic process, e.g., consumer transactions), focusing on various metrics for evaluating total data quality.
RSD has financial support available to those who qualify.
Not for academic credit workshop.
This is the second of two webinars on the total data quality framework. In this webinar, we will continue our discussion on measuring total data quality. The focus will then turn to tools and techniques for maximizing total data quality (such as responsive and adaptive survey design for designed survey data, weighting approaches, and tools for repairing linkage error). We will present a series of examples considering data from real studies, where the concepts introduced will be applied to vet the total quality of the data sets analyzed. Small-group exercises will be used to give participants hands-on experience with applying some of the concepts discussed to assess data quality.
RSD has financial support available to those who qualify.
Not for academic credit.
This course focuses on design and implementation considerations for different phases of the survey lifecycle when conducting surveys internationally or outside of one’s home country. Overview and considerations related to ten topics are discussed: Total Survey Error framework, project stakeholders, triple constraints, bids and contracts, sampling and sample management, questionnaire and instrument design, cross-cultural differences, translation and adaptation, pretesting and cognitive interviews, interviewers and data collection, and interviewer monitoring. Class will meet for online discussion on July 6, 13, 20 and 27 from 11:00am-12:00pm, Eastern Standard Time.
Inquire at isr-summer@umich.edu if you are interested in taking this course for academic credit
This course is intended as an introduction to the science behind survey research and will be taught at an undergraduate level. Our primary reason for designing this course is to share our excitement about survey methodology as a research field. In this course, we focus on principles and theories as opposed to a strictly hands on approach to creating surveys. This approach to the study of surveys will allow you to apply the same methods to a very broad range of research topics, e.g., public opinion and politics, health, drug use, consumer behavior, customer satisfaction and market research among many others. In addition, the course will present basic statistical concepts and techniques in sample design, data collection, and reporting, as well as explanations of how error can be introduced into the survey process. At the end of this course, you will have a basic understanding of what needs to go into conducting a high-quality survey and will be exposed to resources that will allow you build on the knowledge of survey methodology you have learned in the course.
International and comparative population research is a key cornerstone of population science and demography. International and comparative research is essential: 1. to learn the variations in population dynamics across different populations; 2. to predict the future of global population trends; and 3. to test hypotheses across widely varying context and determine the limits on forces producing population change. Our five-day intensive program on international and comparative population research begins with a one day review of the field and deep-dive into data creation for this science. Chitwan Valley Family Study (CVFS) will be used as a featured example and compared to other international population studies as appropriate for the topics.
The Workshop in Sampling Techniques is a component of the Sampling Program for Survey Statisticians. The workshop can only be taken in conjunction with the Methods of Survey Sampling and Analysis of Complex Sample Survey Data courses. The workshop allows students the opportunity to implement methods studied in the companion methods courses such as segmenting and listing in area sampling; selection of a national sample of the U.S.; stratification; controlled selection; telephone sampling; national samples for developing countries; and sampling with computers.
The workshop is a required class for the Sampling Program for Survey Statisticians (SPSS). The SPSS is an eight-week program. It consists of three courses: a methods course (SurvMeth 612), a course on the analysis of complex sample survey data (SurvMeth 614), and a hands-on daily workshop (SurvMeth 616). Students enrolled in these three courses are considered Fellows in the Program. The methods and the analysis courses may be taken without being a Fellow. However, the workshop cannot be taken alone. Fellows receive a certificate upon successful completion of the program.
COURSE OBJECTIVES • Introduce a structural analysis of parts of a survey question • Introduce cognitive interviewing as a method for testing survey questions • Describe guidelines for diagnosing problems in survey questions and writing new survey questions • Focus on the structure and wording of survey questions, whether for interviewer-administered or self- administered instruments • Provide an opportunity to apply the guidelines and principles during in-class exercises • Focus on improving individual questions and sets of questions. • Summarize research that underlies key decisions in writing survey questions.
DESCRIPTION
This workshop distills research about survey questions to principles that can be applied to write survey questions that are clear and obtain reliable answers. The workshop provides students with tools to use in diagnosing problems in survey questions and in writing their own survey questions. Sessions combine lecture with group exercises and discussion. The lecture provides guidelines for writing and revising survey questions and illustrates how to revise troubled questions. Assignments require that students analyze problematic questions, revise them, and administer them to fellow students. Sessions consider both questions about events and behaviors and questions about subjective phenomena (such as attitudes, evaluations, and internal states).
WHO SHOULD ATTEND Individuals who will be writing or reviewing survey questions or survey instruments or analyzing survey data. This course gives practical guidance to those who have written survey questions but who are not familiar with research on question design, those who are just beginning to design survey instruments, and those who use survey data but do not themselves design survey instruments.
