The Glass Frog Podcast

'Tis the season for reflection, so Jen and Rebecca use Episode 6 as an opportunity to talk about what they learned this season and reflect on ways in which they can get better at their work, particularly with respect to approaching their work through a lens of inclusiveness. We hope you benefit from hearing all the ways in which we can improve in our work and hope our episodes this season can prompt some reflection for you as well. Friends, this is our season 4 finale. Thank you for coming with us on this ride. To reiterate, we love the episode suggestions we receive from our listeners, so please keep them coming in. Until next season, stay well!

We had a great chat with Heather Krause, Founder of We All Count Project for Data Equity. Heather is a statistician by training and is presently using her skills and passion to improve equity in data science. She practiced data science in an international setting for many years before realizing many of her own practices were problematic, both reflecting and reinforcing western ways of thinking and often de-legitimizing the values of the people and groups she was studying. Rather than blow up her career, Heather thankfully imagined a new way of doing data science that would marry rigor with equity in the planning, execution, and communication of quantitative research. She now works with a range of clients from the public and nonprofit sectors, as well as academia and other research institutions, to help them identify areas for improvement and tangible practices to improve their own research. We talk with Heather about what data equity means (and, for that matter, what data science means), the many hats she wears not only as a data scientist herself but as a therapist for researchers who are learning that their long-held practices and traditions may be inequitable in both obvious and not-so-obvious ways, and her philosophy that there is no trade off between rigor and equity in data science (and in fact they should be mutually reinforcing!). It was a great conversation, and we know our listeners will appreciate Heather's candor and thoughtfulness. Friends, Heather brings The Goods. First of all, bless her soul, she's a disciplined blogger. Her website is also chock full of tools and resources which you can use to inform your approach to data equity. If you're just now starting to think about approaching your work through the lens of equity, we suggest you start with these resources. The website also offers a Data Equity Framework which researchers can use to think through some of the thorny issues that arise when planning a study, collecting data, analyzing data, and reporting on results. Check out the tools and framework, and if you're interested in digging deeper, please reach out to Heather for more information!

We talk this week with Martha Brown and Debbie Gowensmith about trauma-informed evaluation. It was a great chat, but, before we jump in, here's a little background on our guests. Our first guest is Martha Brown, president and founder of RJAE Consulting. A well-known evaluator and trainer, Martha's expertise is in the area of trauma-informed practices and restorative justice. She joins us alongside Debbie Gowensmith, vice-president of Groundswell Services, Inc., a consulting firm that specializes in strengthening equity-building movements. Debbie's expertise is in community-based research and program evaluation, and she also has a background in program management, which she brings to bear on her evaluation work. We found our guests after reading Debbie's January 2021 AEA365 post on using Photovoice to support trauma-informed evaluation. Debbie introduced us to Martha, and the rest is history. As we mention in the introduction of the episode, we really only scratch the surface in this conversation and would probably benefit from another couple hours with Martha and Debbie! We consider ourselves newcomers to this work, so Debbie and Martha patiently covered some "Trauma 101" material with us before we moved on to discussing trauma-informed evaluation. We talk about different evaluation approaches and methodologies that are better suited for trauma-informed work, but really Martha and Debbie encourage evaluators to assume a trauma-informed lens irrespective of their methodology. Whether we're doing Photovoice or field work or surveys, we should be mindful of the prior experiences of both research participants and our staff and be careful not to engage in practices that re-traumatize individuals or communities. Debbie and Martha referenced and shared a few resources, which we found helpful and will therefore pass along to our loyal listeners: SAMHSA’s Concept of Trauma and Guidance for a Trauma-informed Approach (SAMHSA is Substance Abuse and Mental Health Services Administration, an operating division of the Department of Health and Human Services) SAMHSA’s National Registry of Evidence-based Programs and Practices (NREPP) CDC's site on Adverse Childhood Experiences (ACEs) We really hope you enjoy the conversation as much as we did! FYI, if you'd like to reach out to and/or learn from Martha, you can find her at: For training/workshops in Trauma Informed (TI) Evaluation for your organization or conference, email martha@rjaeconsulting.com For information regarding how Hamai Consulting can support TI systems change and help organization become TI, email martha@hamaiconsulting.com

Episode 3 brings us Jennifer Taylor and Sarah Goletz from the Indiana Area Health Education Center (AHEC) to talk about ripple effect mapping (REM). We knew very little about REM, so we sat like schoolchildren at the proverbial feet of Jennifer and Sarah as they patiently walked us through the basics. If you too are unfamiliar with REM, it's a method that combines elements of appreciative inquiry, mind mapping, and participatory evaluation to ascertain how participants and others involved in or affected by a program describe the program's outcomes and impact. It can also be used to identify a program's unintended consequences (both positive and negative) and inform recommendations. This page on the University of Minnesota Extension School's website has some great resources that help explain how it all works. At a high level, it involves gathering 12 to 20 stakeholders in a room, guiding them through a process where they talk in pairs about elements of the program they appreciate or find impactful, and then facilitating a discussion wherein participants share out their ideas while facilitators organize and map these ideas on a large sheet of paper, whiteboard, or projector at the front of the room. The result is a cool map that visually depicts a program's impacts (see example here). Jennifer and  Sarah stumbled upon REM in their own work at AHEC, and now they not only use REM in their own research, they also happily teach others how to do so. Though COVID has put a temporary damper on some of this work, they were able to host a workshop in Fall 2020 and are happy to serve as evangelists for the method in any way they can. If you have questions, hit them up!

Friends, we have a doozy of an episode. Andy Johnson from Inspire to Change  joined us to talk about arts-based evaluation, creative evaluation, and applied neuroaesthetics. We could have talked to Andy all day, but professional norms dictated that we had to allow him to leave the meeting after our scheduled time expired. Bummer, but if we can have him back, we will! Andy is an artist-turned-evaluator, so he brings an interesting perspective to his work with nonprofits. We wanted to chat with him because of this season's theme around inclusivity. Andy and his team have really unique ways of ensuring that their approach to building hypotheses and theories about program include all perspectives, and also that their way of presenting data reflects the many ways in which people prefer to access and interpret data in the real world. We discuss about some of the unique ways in which evaluators integrate the arts into their work, with respect to both methodology and presentation. Andy introduced us to the visual matrix method, which is a way of using visual art to elicit research participants’ thoughts, sometimes subconscious thoughts, on a particular topic by having them freely respond to art work. We also talked about verbatim theater as a way of presenting qualitative data in a way that is accessible both to people who participated in the research and to consumers of the research. We added links here in case you want to check it out. As a quick plug, Andy’s team is going to offer two classes this fall -- 1) creative evaluation for internal evaluators and 2) writing for social justice. You can visit the events page of their website to learn more about them.

We are excited to be kicking off Season 4 of the Glass Frog Podcast with  evaluator and author Chari Smith! Chari is the founder of Evaluation into Action,  which partners with nonprofit professionals to create realistic and  meaningful program evaluation processes. She teaches workshops aimed at  helping nonprofit professionals understand the value of program  evaluation, and recently published her first book that helps early  career evaluators, nonprofit professionals, and students learn how to  create and implement an evaluation plan. The book, Nonprofit Program Evaluation Made Simple: Get Your Data. Show Your Impact. Improve Your Practice, is on shelves, so order your copy today! Chari is a proud member of the Oregon Program Evaluators Network  (OPEN) and the International Society of Evaluation Education (ISEE). She  is also an active member of the American Evaluation Association (AEA)  and serves on their Nonprofit and Foundations Topic Interest Group (TIG)  Leadership Council. As of summer 2021, she is also an adjunct faculty  member for the University of Portland’s Pamplin School of Business  Nonprofit MBA program. Chari is passionate about her work and brings her  enthusiasm, insight, and expertise to each project. Chari is also available to speak on a range of topics that will both  inspire and inform. If you have an upcoming conference or event, please  reach out to her through her website. Feel free to view her opening plenary session at the American Evaluation Association conference 2020. Participate in a workshop with Chari! On Thursday,  July 29 at 9:00 AM PST, Chari will be giving a workshop titled  Measurable Outcomes Made Simple. If you work with nonprofits or want to  learn a more collaborative approach to creating outcomes, please join  Chari. You can learn more and register for the workshop here. You can connect with Chari on Twitter (@evalenthusiast), LinkedIn, or by e-mail at chari@evaluationintoaction.com.  We talk with Chari about her artistic interests as well. She wrote a song about evaluation (!), which you can hear on her website. You can also learn more about her creative endeavors using the links below: Chari writes music, plays and musicals. Learn more about her creative adventures at: charismith.com Evaluation as Jazz – Comfort Pandemic Music with Michael Quinn Patton, original music by Chari Smith. Portland Mini Musical Festival 2020: Book Club. Book, Music, and Lyrics by Chari Smith.  A group of women in varying stages of their adult lives, come together  under the pretense of discussing a book. The book never gets discussed,  but wine is imbibed, secrets are shared and support is given. It is a  heartfelt tribute to the power of friendship among women, no matter what  their age. (Please note, this video opens with a song that is for the  entire performance of all 6 mini musicals. Book Club starts right after  the song and is about 20 minutes.)

We are back for season 4! We have a great lineup this season, and can’t wait to learn from our guests alongside our listeners! The  first episode of the season will post on Monday, July 5. We’re joined  by evaluator and recent author Chari Smith. We talk with Chari about her  new book, her approach to building a culture of evaluation with her  clients, and (on a tangent we couldn’t resist) her background in and  passion for music and theater. Chari was the perfect guest for  the first episode of the season, which we’re using to loosely explore  the theme of inclusion. How can we be more inclusive of others’  perspectives when doing evaluation? How can we be more inclusive of  program participants’ lived experiences when developing evaluation plans  and collecting data? How can we as evaluators be more open to others’  experiences and, more importantly, include those voices to make our work  better? Chari takes an intentionally inclusive approach to building  capacity, and we learned a lot from our conversation with her. Throughout  the season, we will continue to drop episodes on the first Monday of  every month. As in the past, if you have questions or topics you want to  see explored on the podcast, please reach out at  https://glassfrog.us/contact-us/. We love hearing from our listeners!

Welcome to the final episode of Season 3 of The Glass Frog Podcast!  This episode we got to talk with evaluator, consultant, cartoonist, and  all around nice guy Chris Lysy. We’ve been following Chris on and off  for a few years now through his blog. He recently published a book of evaluation-themed cartoons, Evaluation Illustrated, and we took that as sign that we should have him on our show. We talked with Chris about digital evaluation, the non-predictable  and non-linear dimensions of his career as an evaluator, and of course  about his cartoons. If you haven’t checked out the book, you should. (It would be a great holiday gift for your evaluator friends and family members!) A few of our favorites are below. Chris has an active newsletter and also a new Patreon where he shares more cartoons and keeps you up-to-speed on his work. Subscribe to both for a  regular dose of humor and thoughtfulness amidst these crazy times. This is our final episode of the season. We’re already planning for  Season 4, so, listeners, please don’t hesitate to reach out if you have  ideas! It’s been fun being on this journey with you!

The Glass Frog Podcast is back for its third season! Amidst the chaos and heartbreak of 2020, we're hoping that the podcast can offer our listeners not just a chance to learn something new, but a brief, monthly respite from all of the disorder. We're really glad you're joining us. Our first episode of the season drops on Monday, July 6. We're kicking off the season by talking to Stefanie Singer, who manages a program called Everyday Arts for Special Education (EASE) at Urban Arts Partnership in New York City. Stefanie shares her experience training special education teachers on how to use simple, arts-integrated activities to encourage social-emotional and academic growth among their students. We'll continue to post episodes the first Monday of every month. They’ll be released through our website and on iTunes and Google Play, so please subscribe if you want to stay up to date each month. Thanks to everyone who continues to reach out with ideas and comments! Please keep the feedback coming! If you have ideas for future episodes, please free to reach out to us via the Contact Us page of our website. See you next week! The post Season 3 Trailer! appeared first on Glass Frog Solutions.

In this episode, Jen and Rebecca continue their discussion about their project on the study of the ancillary benefits of education programs. (If you haven’t seen the first part of this episode, you can check it out here. It will get you up to speed on the project and the teacher residency part of the […] The post Jen and Rebecca talk about auxiliary staffing programs appeared first on Glass Frog Solutions.

In this episode, Jen and Rebecca talk for the first time about one of their own projects: a study of the ancillary benefits of education programs. That may sound wonky, so let us explain. All nonprofit education programs have primary goals — some aim to help teachers in the classroom, some train teachers, some provide […] The post Jen and Rebecca talk about teacher residency programs appeared first on Glass Frog Solutions.

We had a chance to talk with Teresa McCaffrey of the Educational Partnership Center (EPC) at University of California, Santa Cruz this month. EPC oversees a collection of 10 programs aimed at preparing underrepresented students for college. Altogether, they serve 11,000 students in the Santa Cruz region. Though each program has a different goal, EPC […] The post Collective Impact Models with Teresa McCaffrey appeared first on Glass Frog Solutions.

This month, we talk with Kathy Haynie, Director of Haynie Research and Evaluation, about evaluating STEM education programs. Kathy has worked in the STEM evaluation field for almost 20 years and was gracious enough to let us pick her brain on her background and approach to evaluation, as well as what she (and the field […] The post Evaluating STEM Education Programs with Kathy Haynie appeared first on Glass Frog Solutions.

This episode, we are excited to bring you the wisdom of Madhawa “Mads” Palihapitiya. Originally from Sri Lanka, Mads currently teaches at University of Massachusetts – Boston and is an expert in conflict resolution and violence prevention, which he came to study as a result of growing up during Sri Lanka’s Civil War. Mads eventually began working as an evaluator in communities around Boston, which is when he started integrating photography into his evaluation work. Jen came across Mads’ photography work while reading the AEA 365 blog, as she is wont to do in her leisure time. His article, “Images as Data for Needs Assessments and Evaluation,” and his discussion of art and images in evaluation piqued her interest in his work. Unsurprisingly, Mads was a great guest. If you listen to the podcast, you’ll immediately see why. Like our episode with Stefanie Singer of EASE last month, he reminds us of the role that the arts can play in revealing our shared humanity and helping research participants express their truth. We really enjoyed talking to Mads and we think you’re really going to enjoy the episode as well! The post Photography and Evaluation with Madhawa Palihapitiya appeared first on Glass Frog Solutions.

Welcome to Season 3! We're kicking off our season by talking with Stefanie Singer, who is the Program Director for Everyday Arts for Special Education (EASE), a New York City-based program operated by Urban Arts Partnership. EASE is an arts integration program that trains teachers on how to use simple arts integration activities and methods to improve outcomes for elementary-aged students. The program was designed to serve special education students, but as we discuss in the interview, the activities can benefit all students. We really wanted to talk to Stef because we haven't talked much about teacher professional development programs on this podcast, nor have we highlighted any programs that explicitly address developing social-emotional skills among students with significant disabilities. Stef has a great perspective on the challenges and stresses that teachers face (even before the pandemic introduced a whole new slew of challenges and stresses!), and talks about EASE's role in helping teachers relax in the classroom and let their students have fun. EASE has a bunch of fun activities, some of which you can now try at home with your own kids (or friends - we're not judging). Check them out on the EASE at Home website. You can follow Urban Arts Partnership on Twitter at @UrbanArtsOrg or on Facebook at @urbanartspartnership. The post Arts Integration with Stefanie Singer appeared first on Glass Frog Solutions.