ReproducibiliTea Podcast

In this episode, Will is joined by Lianne Wolsink, PhD candidate at Ruhr University Bochum and current steering committee member of ReproducibiliTea. Will and Lianne discuss the ReproducibiliTea reading lists, created to help journal clubs do deep dives on metascience topics. Lianne created reading lists on replication and science communication; Will created an introductory reading list on Open Science, preregistration, and theory in psychology. Reading lists: https://rpt-rl.netlify.app Books mentioned: Science Fictions by Stuart J Ritchie Against Method by Paul Feyerabend Nobody's Fool by Daniel Simons and Christopher Chabris

Show notes: OSF Preprints | Let's talk about it: positivism and critical theory in dialogue https://osf.io/preprints/osf/t2ywh_v1 Caliban & the Witch by Silvia Federici Pollution is Colonialism by Max Liboiron Against Method by Paul Feyerabend Lecture on Against Method by Tomas Petricek, University of Cambridge : https://youtu.be/jzD1O_gastA S2E10: Limits of ReproducibiliTea for discussion and resources of a feminist lens on open science https://soundcloud.com/reproducibilitea/s1e10-limits The battle for your brain by Nita Farahany Quokka app: https://palm-lab.github.io/QualCA Quokka video guide: https://www.youtube.com/watch?v=9jJK70W0hAI

In this episode, Sarah and Will chat to Josh de Leeuw from Vassar College and the creator of jsPsych. We chat about the history of jsPsych, the unseen process behind creating open-access scientific software, and the current challenges facing software developers in the open scholarship movement. jsPsych is a javaScript framework for creating online experiments, and is always looking for people to contribute to the codebase: https://jspsych.org. Follow Josh de Leeuw on Bluesky: https://bsky.app/profile/joshdeleeuw.bsky.social

Why is Indigenous health research so crucial, especially for adolescents? What are the systemic challenges researchers face, and how can we push for openness, transparency, and equity in health research? In this conversation, hosts Queen Saikia and Sarah Sauvé speak with leading experts from New Zealand - Prof. Sue Crengle (University of Otago), Prof. Terryann Clarke (University of Auckland), and Dr. Andrew Sise (University of Otago) - to explore the current landscape of Indigenous health research and the methodologies used in these investigations. They also discuss key issues around reproducibility, openness, and transparency, the discrepancies in research systems, and how we can drive meaningful change. This episode is a must-listen for anyone passionate about health equity, open research, and improving research culture. Listen now and join the conversation in the comments!

In this episode, Will is joined by Jamie Moffa, a doctoral student in systems neuroscience at Washington University in St. Louis. Jamie has been thinking and working in the science communication space, especially via the In Plain English podcast, which is aimed at bringing scientific knowledge and understanding to the general public. Show Notes: We think about this paper: Volk, S. C. (2024). Assessing the Outputs, Outcomes, and Impacts of Science Communication: A Quantitative Content Analysis of 128 Science Communication Projects. Science Communication, 10755470241253858. https://journals.sagepub.com/doi/10.1177/10755470241253858 Will mentions this paper by C Thi. Nguyen: Nguyen, C. T. (2021). The seductions of clarity. Royal Institute of Philosophy Supplements, 89, 227-255. https://philarchive.org/rec/NGUTSO-2 Will mentions this paper about color constancy and Crocs randomly...: Wallisch, P., & Karlovich, M. (2019). Disagreeing about Crocs and socks: Creating profoundly ambiguous color displays. arXiv preprint arXiv:1908.05736. Follow and reach out to Jamie, especially if you'd like to contribute to the In Plain English podcast! Jamie Moffa – https://copitslab.wustl.edu/people/jamie-moffa/ In Plain English Podcast – https://inplainenglishpod.org/ Our science communicator highlights: Nature and Nurture Podcast by Adam Omary –https://podcasters.spotify.com/pod/show/NatureNurture Cass Eris – https://www.youtube.com/casseris Dr Neurofourier – https://www.youtube.com/c/Neurofourier SciShow by Complexly (Hank and John Green): https://www.youtube.com/@SciShow Science Night Podcast – https://www.scinight.com/episodes Ed Yong (no longer at the Atlantic!) – https://edyong.me/ The Violinist's Thumb by Sam Kean – https://samkean.com/books/the-violinists-thumb/ You can find Will on BlueSky: https://bsky.app/profile/willngiam.bsky.social If you'd like to find out more about ReproducibiliTea, our grassroots initiative to build community in Open Science across institutions, check out https://reproducibilitea.org.

In this episode, Will and Helena are joined by Emmanuel Boakye and Lamis Elkheir to share their experiences as scientists and Open Science advocates in the Global South and how they started the African Reproducibility Network (AREN). African Reproducibility Network Website: https://africanrn.org/ Twitter: https://twitter.com/africanrepro Lamis Elkheir LinkedIn: https://sd.linkedin.com/in/lamis-elkheir-b5844092 Twitter: https://twitter.com/lamiselkheir?lang=en Emmanuel Boakye LinkedIn: https://gh.linkedin.com/in/emmaboakye Twitter: https://twitter.com/thescientistgh

In this episode, we welcome Queen Saikia as a host of the podcast! She and Will Ngiam are joined by Jonny Coates, Associate Director of ASAPBio, a non-profit organisation seeking to Accelerate Science and Publication in Biology. The topic of conversation is preprint review and peer review. Enjoy! Show notes: ASAPBio: https://asapbio.org/ PREreview: https://prereview.org/

We welcome back the ReproducibiliTea Podcast with Will and Helena chatting to Nafisa Jadavji and Nele Haelterman about Reproducibility for Everyone (R4E), a community-led initiative to run reproducibility workshops. Show notes: Repro4Everyone - https://repro4everyone.org

Today, Sarah, Will and Jan sit down to discuss the last season of ReproducibiliTea. We talk about Will's terrible terrible taste in puns, Jan's terrible taste in pizza, and Sarah's fall into FORRT(.org). While we are wrapping up Season 3, stay tuned for a few more fun & exciting episodes we have planned for this year! Not many footnotes this time: Sarah mentioned sysmus: https://sites.google.com/view/sysmus/home For more info go to ReproducibiliTea.org For comments, questions, tips and tricks, and praise use our feedback form: forms.gle/H6jgUzbbpyauLxUC8

Today, Will sits down with Max Korbmacher, Thomas Rhys Evans, and Flavio Azevedo, some of the authors of the paper "The replication crisis has led to positive structural, procedural, and community changes" to talk about the paper, FORRT, and Open Science communities. Show notes: The paper we discuss for this episode: Korbmacher, M., Azevedo, F., Pennington, C. R., Hartmann, H., Pownall, M., Schmidt, K., ... & Evans, T. (2023). The replication crisis has led to positive structural, procedural, and community changes. Nature Communications Psychology, 1(1), 3. https://www.nature.com/articles/s44271-023-00003-2 FORRT – The Framework for Open and Reproducible Research Training: https://forrt.org Getting involved with FORRT: https://forrt.org/about/get-involved/ Charlotte Pennington’s new book: A Student's Guide to Open Science: Using the Replication Crisis to Reform Psychology https://www.mheducation.co.uk/a-student-s-guide-to-open-science-using-the-replication-crisis-to-reform-psychology-9780335251162-emea-group UK Reproducibility Network: https://www.ukrn.org/ Project Teaching Integrity in Empirical Research (TIER): https://www.projecttier.org/ Reproducibility Wiki: https://replication.uni-goettingen.de/ Paper Trail: https://thepapertrailjc.squarespace.com/ Brain Imaging Data Structure (BIDS): https://bids.neuroimaging.io/ Collaborative Replication Education Project (CREP): https://www.crep-psych.org/ The Center for Open Science: https://www.cos.io/ Nowhere Lab: http://nowherelab.com/ Advancing Big-team Reproducible Science through Increased Representation (ABRIR): https://abrirpsy.org/ Open Life Science: https://openlifesci.org/ Turing Way: https://the-turing-way.netlify.app/index.html For more info go to ReproducibiliTea.org For comments, questions, tips and tricks use our feedback form: forms.gle/H6jgUzbbpyauLxUC8

Today, Will talks to David Reinstein about scientific publishing and The Unjournal. The Unjournal: https://www.unjournal.org The Unjournal on LinkedIn: https://www.linkedin.com/company/unjournal/ To get the latest updates: https://bit.ly/ujupdates To apply for positions at the Unjournal: https://bit.ly/Ujwork Will and David’s extended notes for the episode: https://docs.google.com/document/d/13rL6mq71GD6gPv5wBHr_lfX6YSKkLK9n6EAVfGhZhCk/edit?usp=sharing For more info go to https://ReproducibiliTea.org For comments, questions, tips and tricks use our feedback form: forms.gle/H6jgUzbbpyauLxUC8

Today, Will sits down with Björn Jorges and Sabrina Hansmann-Roth to discuss the role of academic societies in the science reform movement. The poster session: https://www.visionsciences.org/2023-pre-data-collection-poster-session-satellite/ Korbmacher, M., Azevedo, F., Pennington, C., Hartmann, H., Pownall, M., Schmidt, K., ... & Evans, T. (2023). The replication crisis has led to positive structural, procedural, and community changes. Communications Psychology. https://gala.gre.ac.uk/id/eprint/42730842730_EVANS_The_replication_crisis_has_led_to_positive_structural_procedural_and_community_changes.pdf As always: For more info go to ReproducibiliTea.org For comments, questions, tips and tricks use our feedback form: forms.gle/H6jgUzbbpyauLxUC8

Today Sarah chats to Zoltan Dienes Live from SIPS! Zoltan's keynote: https://www.youtube.com/watch?v=BxdGXLOC1Cc Reviewing labor: https://researchintegrityjournal.biomedcentral.com/articles/10.1186/s41073-021-00118-2?ref=refind Peer Community In Registered Reports: https://rr.peercommunityin.org/ Flourishing Science Think Tank paper: https://mindrxiv.org/4zrmd There are a few little audio problems in this weeks episode, so special shoutout to our transcript, which is here: https://docs.google.com/document/d/1zd4o40QLn0o5q1Eyp966EoDdvu5XtT6O/edit?usp=sharing&ouid=115964780222242468834&rtpof=true&sd=true For more info go to ReproducibiliTea.org For comments, questions, tips and tricks use our feedback form: forms.gle/H6jgUzbbpyauLxUC8

Today, Sarah is joined by Agata Bochynska and Matthew Good from the University of Oslo's Open Research Team to talk about how vital libraries are to Open Science. Links from this episode: QualiFAIR: https://www.uv.uio.no/ils/english/about/organization/tlvlab/qualifair/ TIER2: https://tier2-project.eu/ Carpentries: https://carpentries.org/ ReproducibiliTea at UiO: https://www.ub.uio.no/english/libraries/dsc/open-repro-research/reproducibilitea/index.html Preprint mentioned by Agata in the last segment: https://osf.io/kcvra/ Agata Twitter: https://twitter.com/AgataBochynska Agata mastodon: https://fediscience.org/@agata Digital Scholarship Centre website: https://www.ub.uio.no/english/libraries/dsc/ For more info go to ReproducibiliTea.org For comments, questions, tips and tricks use our feedback form: forms.gle/H6jgUzbbpyauLxUC8

Today, Sarah and Will discuss the invisible workload of making open science. The paper on invisible workload: https://journal.trialanderror.org/pub/the-invisible-workload/release/1 The replication crisis has led to positive structural, procedural, and community changes: https://osf.io/r6cvx/ For more info go to ReproducibiliTea.org For comments, questions, tips and tricks use our feedback form: forms.gle/H6jgUzbbpyauLxUC8

Today special guest Nora Serres talks with Sarah Sauve about Registered Report and how cool Bayesian statistics are. Show notes: Appetezer paper: Bayes factor design analysis: Planning for compelling evidence https://link.springer.com/article/10.3758/s13423-017-1230-y Nora’s thesis: https://www.duo.uio.no/bitstream/handle/10852/95323/5/Master-thesis_NS.pdf On registered reports: https://www.cos.io/initiatives/registered-reports https://osf.io/preprints/metaarxiv/x7aqr https://www.nature.com/srep/journal-policies/registered-reports PCI Registered Reports: https://rr.peercommunityin.org/PCIRegisteredReports/help/guide_for_authors On Bayesian/statistics: Bayesian Spectacles: https://www.bayesianspectacles.org/ Understanding Psychology as a Science by Zoltan Dienes: https://link.springer.com/book/9780230542303 For more info go to ReproducibiliTea.org For comments, questions, tips and tricks use our feedback form: forms.gle/H6jgUzbbpyauLxUC8 The transcript is available here: https://docs.google.com/document/d/1CCk__36WxOxuuFc1JX8lou03HvGVRci4/edit?usp=sharing&ouid=115964780222242468834&rtpof=true&sd=true

Today, Will and Jan sit down with special guest and author of the paper "A large-scale study on research code quality and execution" Ana Trisovic to talk about the reproducibility of analysis code. Also windows. Links from this episode: Ana's paper: https://www.nature.com/articles/s41597-022-01143-6 Gratitude package: https://github.com/Pakillo/grateful For more info go to ReproducibiliTea.org For comments, questions, tips and tricks use our feedback form: forms.gle/H6jgUzbbpyauLxUC8

! Important Update ! Since recording, UCU have announced that they will "ringfence £250,000 for members facing deductions for undertaking forms of ASOS, including the marking and assessment boycott." This is reassuring news for workers at universities where the VCs have threatened 50-100% pay deductions. But still no new offers from employers. Today, Sophia and Jan are sitting down with @IlsePit(@fediscience.org) and @KatarinaWarren (k-almeidawarren.com) to chat about the ongoing UCU strikes in the UK, and how it is for ECRs to take industrial actions. "The University and College Union (UCU) represents over 120,000 academics, lecturers, trainers, instructors, researchers, managers, administrators, computer staff, librarians, technicians, professional staff and postgraduates in universities, colleges, prisons, adult education and training organisations across the UK" (https://www.ucu.org.uk/article/1685/About-UCU) You can support the UCU here: https://www.ucu.org.uk/ For more info go to https://ReproducibiliTea.org For comments, questions, tips and tricks use our feedback form: forms.gle/H6jgUzbbpyauLxUC8

Will, Helena and Jan talk about how we assess research and that it is weird we never really learned how to do Peer Review. Papers we discussed: Responsible Research Assessment Should Prioritize Theory Development and Testing Over Ticking Open Science Boxes: https://psyarxiv.com/ad74m/ A consensus-based tool for evaluating threats to the validity of empirical Research: https://psyarxiv.com/fc8v3 Will's blog post on peer review: https://williamngiam.github.io/blog/my_peer_review_process The tool for checking p-Values: statcheck.io The tool for checking mean values: https://journals.sagepub.com/doi/10.1177/1948550616673876 For more info go to ReproducibiliTea.org. Do you have questions? Comments? Wanna say hi? You can reach us here: https://forms.gle/H6jgUzbbpyauLxUC8

We mentioned lots of resources this week, brace yourselves! Books that changed our worlds How To Read a Paper: The Basics of Evidence-Based Medicine by Trisha Greenhalgh https://www.amazon.co.uk/How-Read-Paper-Evidence-Based-Medicine/dp/1444334360 How to Win Campaigns by Chris Rose https://www.amazon.co.uk/How-Win-Campaigns-Steps-Success/dp/1853839620 Pollution is Colonialism by Max Liboiron https://www.dukeupress.edu/pollution-is-colonialism A Student’s Guide to Open Science by Charlotte Pennington https://www.mheducation.co.uk/a-student-s-guide-to-open-science-using-the-replication-crisis-to-reform-psychology-9780335251162-emea-group The Art of Statistics: Learning from Data by David Spiegelhalter https://www.penguin.co.uk/books/294857/the-art-of-statistics-by-spiegelhalter-david/9780241258767 Tools we mention: FORRT Academic Wheel of Privilege: https://www.youtube.com/watch?v=mzEdTyA06cU&ab_channel=FORRTproject Ioannidis et al. (2019): https://journals.plos.org/plosbiology/article?id=10.1371/journal.pbio.3000384 who provide a means to adjust research impact for self-citations using a newly developed tool Baccini et al. (2019): https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0221212 calculate an “inwardness” metric which shows bias to researchers’ own country Citation Diversity Statement: tools that can help authors assess the citation diversity statistics of their papers (see, for example, https://github.com/dalejn/cleanBib#instructions) Jane Lawrence Sumner’s Gender Balance Assessment Tool (GBAT) https://jlsumner.shinyapps.io/syllabustool/ Groups/projects working on highlighting under-cited work: ABRIR (https://abrirpsy.org/), Marginalia (https://www.marginaliascience.com/) and FORRT (https://forrt.org/neurodiversity/) Other relevant links/resources: Evidence-Based Toxicology https://www.tandfonline.com/journals/tebt20 Evidence-Based Toxicology Collaboration https://www.ebtox.org/ Living a Feminist Life by Sara Ahmed Peer review episodes of Secret Feminist Agenda: https://secretfeministagenda.com/2020/04/07/bonus-episode-season-3-peer-review-of-secret-feminist-agenda/ and https://secretfeministagenda.com/2020/04/07/bonus-episode-response-to-review-of-season-3/ Algorithms are racist & sexist: Noble, S. U. (2018). Algorithms of oppression: How search engines reinforce racism. nyu Press. #MHAWS: Mirya Holman's Aggressive Winning Scholars Newsletter: https://miryaholman.substack.com/ MetaDocencia - https://www.metadocencia.org/