Claim OwnershipTotal Survey Design
Subscribed: 7Played: 22
Subscribe
© 2026 Total Survey Design
Description
Total Survey Design is a podcast for explaining the complexities of survey design. This podcast serves a diverse audience, including academics, small business owners, nonprofits, industry professionals, and students. Each season features episodes covering topics from survey utility to sample sizes, and question design to total survey error. Episode content includes insightful discussions, expert interviews, and special event coverage to enhance your survey skills and understanding.
36 Episodes
Reverse
In this episode, I explore research by Lena Le and Thom Allen at Washington State University on the use of personal appeals in survey contact letters — the persuasive statements that encourage people to participate, like explaining how results will be used or telling respondents their opinion matters. Analyzing 190 surveys, their data shows response rates climb from 21.6% with no appeals to 51.6% with nine. I also ran my own statistical analysis on the data, finding each additional appeal is...
In this episode, Dr. Azdren Coma explores how the Dunning-Kruger effect distorts self-reported skill ratings in hiring and surveys. A seemingly straightforward question like “On a scale of 0 to 10, how skilled are you in Microsoft Excel?” can produce misleading data because undefined scale points allow respondents to interpret them differently. Beginners often overestimate their abilities due to limited awareness of complexity, while more skilled individuals rate themselves more conservativel...
In this episode, I will be talking about open-ended questions. Along with closed-ended questions, open-ended questions are the other major type of question used in surveys. This episode covers the purpose of open-ended questions, their pros and cons, the different types of open-ended questions, several examples, and finally some tips and best practices for implementing them effectively in surveys. The episode also includes an important ethical cautionary tale, as well as a warning about the ...
What happens when you are faced with a list of options but do not have a strong preference? In this episode, I explore how the simple order of a list can nudge a person's choice. I move beyond the idea of favorite flavors to look at the undecided voter and the hungry diner. I examine why being first on a ballot can sway an election by as much as 10 percent in some local races. I discuss why these order effects are most dangerous in non-partisan elections where voters may not recognize every ...
This episode is a live guest lecture recorded for a Marketing Research class at Washington State University. The guest lecture is meant to introduce complete newcomers to general ideas and considerations for survey design. I also offer a few general tips on survey questionnaire design. Support the show Contact us at: totalsurveydesign@gmail.com Find us online at: instagram.com/totalsurveydesign/ https://taplink.cc/totalsurveydesign
In this episode of the Total Survey Design Podcast, I explore whether to include a neutral midpoint option in survey questions or force respondents to pick a side. I define forced-choice formats, from even-numbered scales and binary picks to pairwise comparisons. I discuss key evidence like the 2019 Pew Research experiment, which found forced yes/no questions increased reporting on sensitive topics compared to select-all-that-apply lists. I present strong arguments on both sides before explai...
Should you ever force respondents to answer a survey question? In this episode of Total Survey Design, I examine forced questions. While these features are often used under the assumption that more complete data means better data, forcing responses frequently leads to worse outcomes by increasing survey breakoffs, encouraging satisficing, and undermining respondents’ trust. I outline more effective and respectful alternatives, encouraging responses without coercion. Support the show Con...
On this episode of the Total Survey Design podcast, I put generative AI to the test by asking it to critique and improve the Microsoft Word Feedback survey: a short, everyday questionnaire that many of us have seen after using the program. I compare the suggestions from Grok, ChatGPT, and Gemini, highlight where they agree, where they differ, and share my own thoughts on what they got right and what they missed. I recap the original survey (which takes just 50 seconds to describe), dive into...
On this bonus episode of the Total Survey Design podcast, I follow up on Trump and Rogan’s polling critiques with a listener email highlighting nonresponse bias, the real issue behind low response rates, where responders often differ systematically from non-responders. I explain how rigorous weighting corrects for this bias, why it’s not foolproof, and how strong weighting helped make the 2024 election polls some of the most accurate in years, with top aggregators and pollsters like AtlasInt...
On this episode of the Total Survey Design podcast, I examine an episode of the Joe Rogan Experience podcast, #2219, which aired on October 25th, 2024, where he discusses survey sampling with President Donald Trump. They both express skepticism about the reliability of polls for making broad national claims. They also refute the ability to make predictions from small samples and criticize polling methodology. In this episode, I talk about what they got right and what they got wrong, diving in...
In this episode, Azdren creates a feedback survey from scratch, showing the process from idea formation, to messy first draft of the questionnaire, to final product. To see the final version of the questionnaire and to give your valuable feedback on this podcast, please visit https://tinyurl.com/totalsurveydesign Support the show Contact us at: totalsurveydesign@gmail.com Find us online at: instagram.com/totalsurveydesign/ https://taplink.cc/totalsurveydesign
This episode of the Total Survey Design podcast demystifies the statistical technique of survey weighting (or post-stratification), explaining how it corrects sample imbalances by giving "quiet groups" more influence. The hosts define weighting, detail when it's an effective correction tool for probability samples, and provide practical advice on avoiding common pitfalls like over-adjustment that can hurt data precision. Support the show Contact us at: totalsurveydesign@gmail.com Find us onli...
This episode explains the important distinction between response rate and nonresponse error in survey research. Drawing on principles from Dillman et al.(2014), it clarifies that a high response rate does not necessarily mean low nonresponse error, and vice versa. Through two practical examples, the episode illustrates how systematic differences between respondents and nonrespondents -- not the number of responses -- determine the presence of nonresponse error. The episode emphasizes that eva...
In this episode, the hosts speak with Mark Miazga, Director of University Survey and Assessment Services at the University of Minnesota, about his journey into survey research. Mark shares how his upbringing in rural Wisconsin and a background in law and nonprofit work shaped his perspective on data and public service. The conversation covers his transition from corporate law to survey methodology, his work managing complex institutional surveys, and how empathy and practical experience influ...
In this episode, we talk with Robert Santos, Director of the U.S. Census Bureau from 2022 to 2025, about his professional journey and philosophy toward data and public service. Rob shares how his upbringing in a Mexican American community in San Antonio and how his early exposure to music and math shaped his path into statistics. The conversation explores his commitment to equity in data, the importance of cultural understanding in survey design, and his reflections on his time at the Census....
In this episode, I talk about a survey I didn’t plan to analyze—one that popped up the moment I opened Microsoft Word. Microsoft asked for feedback, and well… I had some. I break down the flaws in their opening question, including directional bias, vague wording, and unlabeled scale points. I also reflect on how these design choices impact data quality and what could have been done better. It’s a reminder that even the most common surveys we see in the wild from even the biggest tech giants c...
In this episode of Total Survey Design, Dr. Azdren Coma and Dr. Seon Yup Lee interview Dr. Jon Krosnick, a leading expert on political behavior, survey methodology, and public opinion. Dr. Krosnick is a professor at Stanford University who has advised the U.S. Census Bureau, led national studies on voting behavior and climate change, and published extensively on how people form attitudes and make decisions. Recorded at the historic St. Louis Union Station Hotel during the 2025 AAPOR conferen...
In this episode I sat down with Thom to talk about surveys. We talked about his time at the Social and Economic Sciences Research Center, working with Don Dillman on various projects including the 2000 U.S. Census, some of his favorite projects, my dissertation survey, the application of the Tailored Design Method on a wide variety of cultures, and much more. Support the show Contact us at: totalsurveydesign@gmail.com Find us online at: instagram.com/totalsurveydesign/ https://taplink.cc/tota...
In this episode, we do a small retrospective on the first season. We also look ahead to see what is to come. We also give a call to action, seeking your support to ensure that this project continues to exist into the future. If you would like to support us, you can do so by visiting our Ko-fi page at: https://ko-fi.com/totalsurveydesign Thank you. Support the show Contact us at: totalsurveydesign@gmail.com Find us online at: instagram.com/totalsurveydesign/ https://taplink.cc/totalsurveyd...
In this episode, we explore essential considerations for crafting effective surveys. We begin by emphasizing the importance of defining the purpose of your survey, which serves as a guide through each step of its creation. We discuss how understanding your main goal—whether it's measuring customer satisfaction or academic variables—shapes your questions and improves data quality. Additionally, we highlight the significance of knowing your target population, budget, and timeframe. By the...
Comments
Download from Google Play
Download from App Store
United States