đŻ Evaluation! A make or break thing in the digital health. I sat down with Dr Shay Soremekun from the London School of Hygiene & Tropical Medicine to talk about the very hot topic of evidence generation and evaluation. Everyone's talking about LLM evals, technical performance, benchmarking. But ultimately people care about impact. Yet impact is rarely a neat, linear path to a yes/no answer. Anyone who's actually implemented something in the field knows about all the other contributing factors, the daily challenges and how hard it is to move the needle on big clinical outcomes with robust clinical evaluation.To understand why it improved care, if it did, was as important, if not more important, than understanding that it did. Because that will help us to understand how we need to potentially modify or adapt the intervention either in the same place or in future places to be able to achieve the same success. đ Who will benefit from listening to this episode?Digital health companies : Founders, data scientists, AI/ML scientists, PMs, designers, clinical evaluation/study teams, cliniciansFunders : donors, investorsGlobal and public health professionalsImplementersResearchersRegulatorsShay explains the use of Program Theory and Logic models to visually connect your intervention, all the intermediate steps (not just technology, but people and change) to outputs, outcomes and long term impact. Making space to observe other levers in the system you didn't initially anticipate. The work that Shay and her team have done with The Malaria Consortium and LSTM's Centre for Evaluation looking at a digital health tool for community health workers in villages and facilities showed this perfectly. Because they were intentional about observing the whole system, they discovered other factors contributing to impact and could redirect efforts accordingly.In this age of tightening budgets, and pressure to show clean shiny KPIs, how do you make room to observe these things? We also discussed decolonizing evaluation: capturing what's locally valuable, not just paradigms of success developed in comfortable offices and ivory towers.So many learnings here for understanding good co-design, implementation, and how to measure what matters.Chapters00:00 Intro04:15 Mozambique digital health tool for CHWs : context14:19 Tsking a system lens approach16:25 Measuring What Matters in Health Interventions23:02 Resourcefulness in Low-Resource Settings27:25 Challenges and Successes in Digital Interventions29:34 Proactive Measurement and Adaptation31:20 Scaling Up Successful Interventions32:59 Revolutionizing Evaluation Practices39:38 Incorporating Indigenous Knowledge in Evaluations44:57 Key Recommendations for Measuring What Matters đĄ Keep learning!If you found this episode helpful listen to:Episode 15 : Implementation 101 and how to Fail well with Caroline Perrin Episode 12 : Health First, Innovation second, with Smisha AggarwalEpisode 5 : What is the right approach for regulation and evaluation of digital health technologies?I have also written a Substack article summarising what I learned from this episode. Check it out for easy reference to the concepts outlined. Found it useful? Know someone who is struggling with this very thing? Share and keep raising the profile of people impacting underserved community globally. References and links to things we spoke aboutinSCALE papers:https://journals.plos.org/digitalhealth/article?id=10.1371/journal.pdig.0000235https://journals.plos.org/digitalhealth/article?id=10.1371/journal.pdig.0000217 Link for upSCALE (roll-out of project):https://www.malariaconsortium.org/projects/upscalehttps://www.malariaconsortium.org/resources/integrating-upscale-into-the-ministry-of-health Study protocol for Uganda and Mozambique:https://trialsjournal.biomedcentral.com/articles/10.1186/s13063-015-0657-6 Links to Chilisa Bagele â one of the most foremost voices on decolonial evaluation methodology so a good start for anyone interested (talk at LSHTM):https://www.lshtm.ac.uk/newsevents/events/decoloniality-and-indigenous-methods-global-health-evaluation-professor-bageleCentre for Evaluation Website https://www.lshtm.ac.uk/research/centres/centre-evaluation About Dr Shay SoremekunDr Shay Soremekun is an epidemiologist and co-deputy Director of the Centre for Evaluation at the London School of Hygiene and Tropical Medicine.Her research in child and adolescent health and development covers trials of low-cost disease prevention programmes, and identification of risk factors and mitigation strategies for poor developmental and economic outcomes in this group. She is a member of the UK Government Evaluation and Trial Advice Panel (ETAP), and sits on the steering committee for the John Snow Society. She lectures at postgraduate level on topics of evaluation, epidemiology and public health, and has developed and organises an MSc module in Study Design. LSHTM page: https://www.lshtm.ac.uk/aboutus/people/soremekun.shay
đ¨ Nick Martin, social impact titan, one of the biggest voices out there in this space with a huge following and community, helping connect people to jobs, funding and much more. Founder and CEO of TechChange, working with funders, government and tech, and giving people the skills they need to create real impact. Weâve talked a lot about the impacts of the USAID funding cuts on this podcast. Nick, who has worked shares his view on the impact on grantees, humans and more. We cover:How do we deal with this new world order, and this new scarcity?What have we learned about the things that went well with USAID, and what could be better in the future?How do people affected think about their careers?How do organizations adapt their vision, strategy and tactics in a world of scarcity, and be sustainable through this perma-uncertainty. We also talked about the upcoming Global Digital Health Forum, a gathering that has been going strong for years, and why this year feels particularly important to bring people in global and digital health together. The next one at the time of recording is 3-5 December 2025, online and in Nairobi in person. If you're into Global Digital Helath and underserved communities, this is a great gathering to attend.If you are working in this space, dealing with the uncertainty, either as an organization or an individual, then this podcast is for you. Nick shares some of the wisdom and insights he has gained over the years, and why community across borders is such an integral part of how we shape what global digital health looks like in the next 10 years. Check it out - share with others working in the space. Leave us a comment or review - it really helps us reach more people. 00:00 Intro : Nick's story03:58 USAID cuts. Before, during, after : reflections from the development sector21:13 Career transition advice24:14 Funders need to step up more, but it's not that simple26:34 TechChange origins28:22 The Global Digital Health Forum : bringing people together34:30 Fail festivals and creating the space to talk about failure38:56 Wisdom corner : Nick's top tipsLiked this episode? What to listen to next:- Episode 15: Implementation 101 and how to fail well- Episode 16 : How the WHO is evolvingWhat to read next:Check out my Substack : Can Global Health and Venture Capital Get Along?About NickNick Martin is the Founder and CEO of TechChange, a social enterprise that has become a leading provider of digital health training and convening solutions worldwide. Under his leadership, TechChange has trained thousands of Ministry of Health officials in more than 90 countries through flagship programs such as Digital Health: Planning National Systems, developed in partnership with USAID, WHO, and Digital Square. He also leads the Global Digital Health Forum, the premier annual gathering for policymakers, donors, researchers, and implementers working at the intersection of technology and health. With two decades of experience in global development and public health, Nick has built partnerships with organizations including USAID, UNICEF, WHO, and the Gates Foundation. His work has been featured in outlets such as The Economist, Forbes, The New York Times, and Fast Company, highlighting his role in advancing digital health and social impact through technology.
As underserved communities go, refugees are up there as the most vulnerable. We already have systemic and deep challenges for physical health services not being designed for displaced people, and has we rush to bolt on digital and LLMs to everything, are we stopping and thinking about the key needs of these communities?And a word on innovation in humanitarian settings. This is not a problem that investors go for with attractive returns, or something you can meet the market where it is at. Rather, its down to governments and philanthropy to help invest in this. Aral recounts the massive challenge HERA have had to overcome after USAID funding was cut and how he has navigated and needed to prioiritize after the stop work order. Aral is the founder of HERA Digital health. HERA have built a tool that serves Syrian refugees in Turkey - targeted initially at mothers to be and mothers with young children, who, in the mess of trying to rebuild any semblance of routine in refugee camps would miss key antenatal or child development appointments. Aral saw the problems the community were having, and as a doctor in the ER in Turkey saw the downstream effects of this. Some key takeaways for me:In humanitarian settings, context is not static and always changing. HERA hired developers who were in the refugee camps - short feedback loops and people immersed in the context. This is a great example of 'Build with, not for"The communities he is serving do not get to quit, why should heDefining what success is in this context - it is not about HERA surviving, or Aral being successful, but the work, and the impact to those people. So he is optimizing decisions for that - and this means he can be open to other options. This episode is not a shiny rainbow story, but one about real struggle. Aral is still on the path of struggle and working to navigate through. If you care about what he is doing, seriously, get in touch with him.Chapters00:00 Introduction to Digital Health Innovations05:05 The Journey of Aral Surmeli: From Medicine to Tech07:47 Understanding Refugee Health Needs10:13 Building HERA: The Digital Health Tool18:23 Cultural Sensitivity in Health Solutions28:01 Navigating Challenges in Humanitarian Innovation30:50 Funding Dilemmas in Humanitarian Contexts36:01 Adapting Strategies for Sustainability44:03 The Role of Digital Identity in Healthcare46:51 Finding Hope Amidst Challenges48:50 Advice for Aspiring Humanitarian InnovatorsAbout Aral:https://heradigitalhealth.org/Dr. Aral Surmeli is the Founder and CEO of HERA Digital Health, a nonprofit organization that helps refugee women and children access local healthcare through AI-powered digital tools. With a strong foundation in humanitarian health and digital innovation, Dr. Surmeli has led HERA to serve over 300,000 users, providing services like immunization reminders, prenatal care tracking, and digital health recordsâaccessible even in low-connectivity settings via WhatsApp and offline apps.HERA began its work in Turkey with Syrian refugees and is now expanding across the Middle East and Africa. The platform is open source, co-designed with local healthcare workers and refugee communities, and integrates with national health systems and NGO services.Dr. Surmeli holds an MPH from Harvard University and is currently pursuing a DrPH at Johns Hopkins University. He is a former Innovation Fellow at Harvard Humanitarian Initiative and has been supported by organizations such as Google.org, Grand Challenges Canada, MIT Solve, and Harvard iLab. HERA was recently a finalist at the AI for Good Summit's Impact Awards, and Dr. Surmeli is an active voice on the intersection of AI, digital health, and humanitarian response
đ I speak with Rubayat Khan, a systems thinker, entrepreneur, and now investor at the Endless Foundation, where he is helping reimagine how global health innovation is funded and delivered.We unpack what it means to move beyond vertical health solutions, how to prioritize innovation in an era of shrinking aid budgets, and how large language models (LLMs) might be the missing piece in unlocking integrated, people-centered care especially in low-resource settings. We also talk about the challenges, risks, pitfalls and how to think about the right counterfactuals for the context when we evaluate LLMs in healthcare.Rubayat brings the rare perspective of someone who has been a patient, builder, and funder, and who now advocates for rethinking everything from clinic hours to global incentive structures.đ In this episode:Why governments are not always best placed to design people-centered care : an example of this was the fact that government run clinical in villages in rural Bangladesh were only open 10am till 2pm. And so nobody would access them as people are working on the fields or caring for their family. Four priority areas Endless Foundation is focusing onHow LLMs could shift access, quality, and cost curve if implemented thoughtfullyWhy self-care is the blind spot we can no longer afford to ignoreThe dangers of "digital colonialism" and what needs to change in global AI governanceWhat funders can do differently to support impact beyond rhetoricđŁď¸ Key QuotesâMost of what we call healthcare happens outside clinics. If we ignore that, we miss the biggest opportunity for real impact.âTo paraphrase Rubayat:"A consultation with a doctor in Bangladesh averages 48 seconds. There should be little surprise that people (including Rubayat's parents!) find value and better quality information with a LLM than they would in their own contexts. So much valuable insight for people who are building, or wrestling how to invest or fund the right interventions for last mile impact in LMICs. đ§ Listen if youâre:A global health funder or innovator rethinking priorities post development cutsA builder working on AI in healthcareA policymaker exploring people-centered designAn innovator aiming to solve real problems in underserved settingsShare, subscribe, comment, leave us a 5 star review. It really helps us reach people who would find this useful. About RubayatReubayat's Substack: https://rubayatkhan.substack.com/Rubayat Khan is a health entrepreneur and technologist from Bangladesh and currently Director of Health Programs at Endless Network, a US family foundation. Prior to Endless, Rubayat co-founded mPower Social Enterprises and Jeeon, which have both pioneered innovative models for delivering healthcare and other essential services to last-mile populations across 15 countries, currently reaching over 120 million people. Rubayat is a passionate advocate for bottom-up and user-centered thinking in global health, and has written extensively in leading global publications like the Guardian, SSIR and Frontiers in Public Health. He is an Acumen and Aspen New Voices Fellow, and is currently based in Baltimore with his wife and two children. (00:00) - Introduction (08:33) - Navigating the Challenges of Healthcare in Bangladesh (10:46) - Transitioning from Ground-Level Solutions to System-Level Thinking (18:07) - Identifying Key Challenges in Global Health Systems (23:20) - Seizing Opportunities Amidst Crisis in Global Health (29:30) - Leveraging AI and Technology for Healthcare Transformation (37:49) - AI vs Human Decision-Making in Healthcare (41:26) - Evaluating AI in Healthcare Contexts (44:07) - AI's Potential in Low-Resource Settings (46:47) - Concerns About Digital Colonialism and Data Ownership (51:05) - The Need for Coordinated Leadership in Healthcare (54:26) - Finding the Right Problems to Solve
Your Metrics Donât Matter If No One Gets the Message : Health Tech Comms with James Somauroo of SomX. Listen on Apple, Spotify. Watch on Youtube. We cover one of the most overlooked drivers of success in health innovation: communication. Our guest, James Somauroo, has hosted over 400 podcasts and built one of the most influential media and communications agencies in health tech. His work spans startups, Big Tech, life sciences, and everything in between, helping organizations craft messages that resonate, build trust, and drive impact.We explore why so many well-intentioned digital health projects struggle to scale. Not because the tech or evidence isnât good enough, but because the story isnât being told in a way people understand or care about.đ§ What Youâll Learn:Why comms is a massive unlock all the way from policy, implementation to on the ground innovation. How to go beyond metrics to communicate real value and outcomes. Thinking about 2nd and 3rd order effects the way journalists do. What makes a pitch deck land with donors or investorsThe power of giving people value in content to stand out. Don't just try to beat the algorithm!đ§ Who Should Listen:Anyone trying to change people's mindsHealth tech founders, especially those working in low and middle-income countriesGlobal health implementers, policymakers, and fundersAnyone trying to drive adoption, trust, or policy buy-in for digital health solutionsđŁÂ Subscribe, Connect & Share:If this episode resonated with you, share it with someone trying to make their work land more powerfully â whether with partners, funders, or frontline teams. And let us know your biggest takeaway on LinkedInđđž About ShubsDr Shubs Upadhyay, the podcast host is a Primary care physician who has worked across policy, AI product leadership, and evidence comms. Shubs brings clinical leadership to help founders and investors focus on and communicate real value in healthcare. Get in touch at shubs.me and hello@shubs.meAbout JamesJames is the cofounder and CEO of SomX, a communications and creative agency for healthcare companies. He hosts The Healthtech Podcast and is the Editor-In-Chief of Healthtech Pigeon. He is an anaesthetics and ICU doctor by training, has held roles in leadership, management and innovation at NHS England, Health Education England and the British Medical Journal and previously directed two healthtech startup accelerators. He has degrees in medicine, biomedical sciences and education and is a guest lecturer on healthtech innovation and entrepreneurship at academic institutions around the world. (00:00) - Introduction to James and SomX (07:51) - The Evolution of Health Communications (13:12) - State of Comms in Digital Health (18:20) - Navigating the Attention Economy (21:00) - The Importance of Authenticity in Content (23:41) - Psychology in Marketing and Communication (31:11) - The Role of Emotion in Decision Making (32:48) - Transforming Scientific Communication through Storytelling (37:09) - Crafting Compelling Narratives for Impact (43:59) - Communicating Evidence Effectively (48:28) - The Importance of Sharing Failures (52:10) - James' quick fire tips (00:00) - Chapter 13
A conversation on the future of the World Health Organization, rethinking how we approach digital implementation and funding in LMICs, and what it really means to decolonize global health.âCountries need to be in the driverâs seat of their own development agenda⌠The earmarking of assistance has to stop.â - Dr. Alain Labrique, Director, Digital and Innovation, World Health OrganizationAbsolutely chuffed to have had the chance to speak with WHO Director of Digital Health and Innovation, Dr Alain Labrique.  That heâs also a listener of the podcast? Even better.Alain is an engaging storyteller, shaped by a childhood in the streets of Dhaka and decades spent advancing implementation science to reach underserved communities.We spoke at length about the unfolding crisis in healthcare delivery in many settings that have relied heavily on development funding. While the impact on clinical services has received attention, Alain highlighted a deeper, less visible collapse: the quiet failure of digital infrastructure: servers, IT systems, and backend platforms now left unfunded and unsupported.At the same time, WHO itself is navigating similar pressures: budget cuts, shifting priorities, and an urgent need to re-focus. Alain offers a refreshingly honest take on what comes next for WHO and how the organization can stay relevant in a rapidly changing world.Crucially, it's inspiring to hear a leader from such an influential institution speak with clarity and conviction about shifting power, building local ownership, and rethinking how global health is funded and framed.On our episode on AI ethics with Jess Morley, she spoke about how rhetoric shapes collective mindsets and changes behavior and markets. This conversation with Alain is, in my view, the kind of rhetoric we need more of.00:00 Introduction and Background of Alain Labrique12:05 Development funding cuts impact on digital infrastructure24:01 What future funding mechanisms need to consider27:26 The Role of Private Sector in Health Infrastructure31:14 Responsible Partnerships in Health Innovation33:00 The Evolving Role of WHO in Digital health now38:21 Building Capacity and Governance in Health Systems40:41 Navigating AI in Health Care46:22 Learning from Failures in Health Initiatives50:27 Advice to Founders building in Underserved Communities
Implementation. What does it mean? Why is it hard? How do we get it right? Failure. Are we, in Global Digital Health, thinking and acting in a way that doesn't help us move forward?Caroline Perrin, Director of the Geneva Digital Health Hub joins us on the Global Perspectives on Digital Health podcast to share her views on this. This episode is for the people doing the work to ensure digital health tools actually work for people.It's about the messy realities - people, process, managing change and looking to constantly improve as our assumptions get updated. How to work through the hard, day to day challenges towards real outcomes for people experiencing and delivering care.We hear about: How the Geneva Digital Health Hub's Implementome maps out implementations across the world, especially the Global south and brings together a Community of people wrestling similar problems for knowledge exchange and connectionThe key challenges in getting digital health tools adopted, and how to use good implementation principles to get through the messy, day to day realitiesChallenges in Monitoring and Evaluation, and how to get the balance right between impact on what matters and what is in your locus of control. How our industry and ecosystem needs to rethink how we talk about failure, and how we're opening up this conversation at the Geneva Digital Health day on 22 May 2025Plus some great examples and ways you can share challenges to contribute to the ecosystem. Links:Building and Implementing? Join Implementome: https://gdhub.unige.ch/implementome/mainDigital Health Atlas: https://gdhub.unige.ch/implementome/projectsGeneva Digital Health Day: https://gdhub.org/25gdhd/Implementation Reports: https://medinform.jmir.org/themes/145-implementation-reportiCHECK-DH Guidelines: https://www.jmir.org/2023/1/e46694Chapters (with quick links)00:00Introduction to Digital Health and Geneva Digital Health Hub07:40Understanding Implementation in Digital Health12:22A community of implementors20:09Approaching Implementation as Founders and Developers26:19Monitoring and Evaluation Challenges32:45Future of Digital Health in Underserved Communities35:31Navigating Uncertainty in Digital Health Projects38:45Rethinking Success and Failure in Health Evaluations45:46The Need for a Paradigm Shift in Global Health49:53Creating Safe Spaces for Sharing Failures54:04Building Communities of Practice for Learning57:30Finding Hope in the Face of ChallengesAbout CarolineCaroline Perrin is the Executive Director of the Geneva Digital Health Hub (gdhub) and has over 15 years of experience in digital innovation, programme management and multi-stakeholder collaboration. She specialises in using technology to improve the efficiency, sustainability and impact of health care. With a PhD in Global Health, a Master's in Information Systems Management, and a Bachelor's in Business Administration, she has led large-scale digital health initiatives and worked with governments, international organisations, and the private sector to drive innovation and policy alignment. Previously, she was a project manager for the RAFT telemedicine network and the Geneva University Hospitals eHealth and Telemedicine Service, focusing on the development and evaluation of digital health solutions to strengthen health systems. Caroline is passionate about bridging technology, strategy and policy to create scalable, high-impact solutions.Help GPODH growSubscribe, leave us a comment on Youtube or your podcast player if you found this useful. It really does help us grow and reach the right people who would benefit from this podcast. Who would benefit from this in your network? Please share the love đFind out more about me and read my Substack: https://shubstack.substack.com/
A healthtech investor's view on healthcare in Africa in the next 10 years. If youâre building or investing for impact with underserved communities in Africa, this episode is for you. Rowena Luk has walked the talk. She is the âOGâ of creating visibility on digital health in Africa, and has built and scaled for impact in over 40 countries on the continent, and has now scaled her own impact to support founders through her VC firm Africa Health Ventures. This episode covers so much, especially the challenges that Africa faces through development aid funding cuts. Rowena is bullish about where healthcare in Africa can get to, and makes a measured and compelling case for why this is a real moment to turn crisis into opportunity, and how the right mindset from VCs can help create resilient businesses that can scale and realize real health outcomes. Iâve often been vocal and skeptical about how hyper-fixated our industry is on financial aspects of value, and we discussed this at length. Rowena definitely altered my perspective.My highlights:This is Africaâs moment. Rowenaâs case for investing and the hard metrics business case is pretty compelling. It is not always the case that VCs look at the short term only. In fact, philanthropy and development funding cycles themselves are short, and good VCs have a 10+ year view and look at broader value.You have to meet the market where itâs at. My takeaway: Africa is not homogenous. Do the work to know what is needed right now, and build a path to where you want to get. If youâre a US or Europe based investor looking to invest in Africa, check your biases and leave your mental models at the door. Things are different here, in a good way. Useful advice for people building and investing in AfricaThe importance of investing in distribution and infrastructure. My read: You might have the perfect tool, but itâs hard to get hospitals and institutions to even know about them, let alone maintain, find payment models and have a path to adoption.Chapters00:00 Introduction to Digital Health in Africa04:36 Rowena's Journey in Digital Health06:43 The Investing landscape in Africa13:19 The case for VC investment over philanthropic funding20:47 Reimagining healthcare in Africa : building resilient health systems26:00 Do VCs see value beyond financial returns?29:08 Myths and Realities of Investing in Africa36:14 Understanding Investment Biases in Africa38:39 Identifying Market Needs and Product Diversity40:44 Evaluating Healthcare Solutions and Market Readiness42:22 Spotlight on Innovative Startups46:26 Learning from Past Experiences51:56 Key Tips for Building in Underserved Communities54:30 Getting Started with Impact Investing54:54 The Future of Healthcare in AfricaAdditional ResourcesIf you're a healthcare founder, investor, or donor in Africa, check out the Africa Health Ventures monthly newsletter. It includes grant and investment opportunities, events, deals, notable startups, and industry news. Subscribe at AfricaHealthVentures.com/Newsletter.About RowenaRowena Luk is the Managing Partner of Africa Health Ventures, a pan-African seed fund investing in healthcare innovations that will dramatically improve access and quality of healthcare on the continent and around the world. She is a healthtech founder, entrepreneur, and software engineer. She has deep experience scaling healthtech in 40 countries across Africa, including products adopted at national and international scale. Previously, Rowena was a Director at Madiro, an impact fund investing in healthcare in Africa; CSO at Dimagi, a social enterprise deploying digital health to 130 countries, and founder of a digital health nonprofit in West Africa. She hosts the Africa Health Ventures Podcast and lives with her family in beautiful South Africa. Help GPODH growSubscribe, leave us a comment on Youtube or your podcast player if you found this useful. It really does help us grow and reach the right people who would benefit from this podcast. Who would benefit from this in your network? Please share the love đ
Practical breadth and depth on the global state of regulation from someone at the cutting edge of regulatory policy. Shubs welcomes Hugh Harvey, founder of Hardian Health and a regulatory expert in digital health. Shubs and Hugh discuss the complex landscape of medical device regulation worldwide, with emphasis on how these frameworks impact digital health innovation in low and middle-income countries (LMICs). Guest BackgroundHugh Harvey is a former radiologist who transitioned to the digital health industry. After working at Babylon Health and serving as Clinical Director at Kheiron Medical (where they secured Europe's first CE mark for a deep learning-based breast cancer detection device), Hugh founded Hardian Health to help companies navigate regulatory pathways for AI and digital health solutions.Key Discussion PointsThe Global State of Medical Device RegulationRegulatory Variation: Hugh explains the significant differences in regulatory approaches between regions like the EU (stricter) and the US (increasingly deregulatory)(Some) LMIC Contexts: Only about 40% of African countries have actual medical device regulations, with only South Africa having comprehensive frameworksCybersecurity Risks: Medical health data sells for more than financial data on black markets, making robust regulation essential for protecting patientsChallenges for Digital Health InnovatorsFounder Misconceptions: Many startups allocate insufficient resources for regulatory compliance. Regulatory compliance should represent 5-10% of a company's overall budget.Regulatory Debt: Postponing regulatory considerations creates compounding challenges that become increasingly difficult to address later down the lineLarge Language Models: Hugh's skepticism about LLMs in healthcare, noting they're "massively overrated" for medical reasoning tasks and face significant regulatory hurdles, and yet Hugh sees some promise over the hill. Practical Guidance for Digital Health CompaniesGetting the mindset right: Thinking about it in the same way you might approach a driving test. Universal Standards: Quality management systems (ISO 13485) are increasingly harmonized globallyApproach in LMICs: Thinking about deploying in an place where there is little to no regulatory structure? You might want to consider securing certification in countries with established regulatory pathways prior, and then work with the local government.What Regulators Could Do BetterProactivity: Regulators should be more proactive in providing guidance on novel technologiesTransparency: More open sharing of regulatory decisions would help the entire industry move forwardCapacity: Governments should increase funding for regulatory bodies to reduce backlogs and improve efficiencyChapters00:00 Introduction and Background04:22 The State of Regulation in Healthcare06:36 Why so regulated?09:24 Global Perspectives on Regulatory Approaches16:53 Harmonization of Global Standards19:34 Recommendations for Regulators31:53 Regulatory strategy for founders: The Driving Analogy42:14 What if there is little or no regulatory enforcement where I operate?43:59 The Five Stages of Regulatory Grief47:44 Hugh's spicy takes đśď¸Key Quotes"Regulations are written in blood... the reason regulations/FDA are as they are today is because of events like thalidomide."What's the cost of not being compliant? Well, it's everything, it's your entire business model.""The attack surface vector, especially under these generative AI models, is huge, vast, and frankly, completely unknown."Hugh provides a compelling case for embracing regulatory requirements as not just necessary obligations but strategic advantages. While acknowledging the challenges of navigating complex regulatory landscapes, particularly for innovators working in regions with limited frameworks, he offers practical guidance on approaching compliance as a foundation for long-term success in digital health. You can find Hugh and his work at hardianhealth.comLiked this episode? You might also like these:Episode 3: Bilal MateenEpisode 5: Prof Stephen Gilbert đ If you found this podcast episode helpful, please leave a comment, give us a 5 star review and share with your colleagues and people who would find this useful. It helps us reach the people who are implementing and could find this useful in creating an impact with digital health in LMICs. Check out the Substack here
Dr. Smisha Agarwal (Johns Hopkins University Global Center for Digital Health Innovation) joins me to unpack some hard truths on the Global Perspectives on Digital Health podcast.  đ Why global health systems remain fragile đ How digital tools often scale without legitimacy đ Whatâs wrong with our fixation on "did it work?" and significance values đ And how global health is stuck in systems that havenât evolvedđŹ âWeâre brought into a system of aid and global development that has stopped questioning how things were done. And the world has progressed, but our field hasnât.âđ But Smisha also offers a way through:- Thinking differently about evaluating system level impact- The work of the Oxford Open Health Journal to increase visibility and representation of people's research in LMICsđ If you work in digital health, research, policy, or global development this is for you.About Smisha:Dr.Smisha Agarwal, PhD, MPH, MBA, BDS is the Director of the Center for Global Digital Health Innovation and Associate Professor in the Department of International Health at the Johns Hopkins Bloomberg School of Public Health. She brings expertise in advancing primary healthcare through strengthening community health systems and leveraging innovative technological solutions including digital devices. A part of her research has focused on using predictive analytics and machine learning algorithms based on routine monitoring data to enhance our understanding of quality of care, create safety nets to care for high-risk populations and improve effectiveness of reproductive health services.Over the last two decades, her research has been leveraged by normative agencies like WHO to develop guidelines on national digitaltransformation, donors to guide investments in primary health care, and governments to develop their national digital health strategies. She is the Editor-in-Chief of the Oxford Open Digital Health Journal. (00:00) - Introduction, Smisha's background (05:01) - Impact of Global Health Aid Cuts (10:03) - Challenges in Monitoring and Evaluation (15:35) - Health First: How to think about outcomes, measuring the right things (20:54) - Supporting healthcare workers as a goal (26:37) - Impact: Measuring what matters (28:39) - Donor decision making dynamics (30:09) - Rethinking system level success: from yes/no to more nuance (33:50) - Innovations in Academic Publishing (37:37) - Challenging the Status Quo in Global Health (42:20) - Upcoming events in Global Digital Health
Bringing research to implementation. Shubs is joined by Lucy Cesnakova from DiMe in this quick dive into an important piece of research bringing insights to advance digital solutions that serve mental health, DiMe's work here was great here because Lucy and the team intentionally sought out input from different countries, access levels and other demographics across the ecosystem. It's great that the Wellcome Trust funded this work. And more of this type of research would clearly be, ahem, welcome. Why should I listen? This will be useful if you are building solutions with underserved communities in mental health and applies both to high income and LMIC settings. If you are researching digital endpoints and biomarkers in mental health or involved in health system transformation for mental health this is well worth a listen. We cover the big commonalities across all geographies, plus what we can learn about local nuances as well as some key a-ha moments Lucy and the team gained. Check out my substack article for a written reflection on this episode. Chapters00:00 Introduction to DiMe and Lucy's work03:57 Current healthcare context06:01 Methodology of Research and Data Collection08:27 Key Findings: Universal Commonalities in Mental Health Technologies16:04 Unique local insightsHere's the full report: https://datacc.dimesociety.org/mental-health/DiMe society:https://dimesociety.org/About Lucy:Lucy Cesnakova, MS, is a Program Lead at Digital Medicine Society (DiME).At DiME Lucy has led several projects in the space of digital measurements and technologies for health: a flagship pre-competitive collaboration project to advance the digital measurement of nocturnal scratch; an initiative that explores path forward for sensor-based digital health technologies for mental health; or recent work on use of patient-generated health data in development of medical products and health technologies. At DiMe she works with industry, patient organizations, regulators, clinicians and payers to create resources that will improve adoption of digital technologies in research and care. In the past, Lucy has led technical development of digital endpoints or other software solutions as a product lead.Â
Special bonus episode. This is a shortened version of the episode with Jess Morley, focusing on ethical approaches and takeaways relevant for digital health companies. How do you move from having ethical approaches as 'nice to haves' to actually getting prioritised. It can be tough when good intentions rub up against business realities. Jess shows how you can avoid getting your initiative or feature getting kicked down the roadmap. People who will find this useful:Product managers and product leadersDesign/UX folkSoftware engineersClinical safety officersClinicians working at digital health companiesQuality assurance and regulatory professionalsFoundersOperations leads
Making Ethics Actionable. What can the UKâs AI Action plan reveal about the state of ethics in the digital health industry?If you are a health system leader or government entity, how do you elevate ethical approaches in your ecosystem? What levers are available?And for the builders: product leaders, founders, clinicians at health tech companies, software engineers, designers, QA folk: how to negotiate and advocate for ethical approaches against business realities in the current climate?We cover all of this, as well as Jessâ super hot takes on the UK govt AI action plan, Ethics 101 for vendors, researchers and policy folk, AND what leaders in LMIC settings can take away from all of this.Jess Morley of the Digital Ethics Center at Yale also previously worked with NHSX and also on the Goldacre review in 2022. She has deep expertise on ethics and policy and some really unique insights. Links to stuff we discussedUK Govt AI opportunities action planJess' linkedin article with commentary on the action planGoldacre reviewJess' Google ScholarYale Digital Ethics Center
đď¸ New Episode: Connecting Global & Local Perspectives in Digital Health ImplementationIn this special episode, we close out the year with two fantastic guests: Rigveda Kadam, Digital and AI Lead at FIND, and Andrew Muhire, Chief Digital Officer at Rwanda's Ministry of Health.Building on the innovator insights in Rwanda from our previous episode with Dr. Jana Alagarajah, this discussion dives into the critical connection between product development partners and ministries of health to create conditions for successful digital health implementation.From Rwandaâs patient-centered approach to FINDâs global perspective across LMICs, we explore:âď¸ How Rwanda balances top-down mandates with frontline adoption to foster trust.âď¸ Tackling antimicrobial resistance with clinician decision support as a case study.âď¸ Using systems thinking to measure success in evolving interventionsâAndrewâs "contribution lines" approach is a must-hear!âď¸ Rigvedaâs wishlist for the ecosystem: from WHO priority areas to better alignment on patient journeys.Tune in for an inspiring conversation on infrastructure, policy, and trustâkey elements for making impactful digital health innovations stick.FIND: https://www.finddx.org/ Some resources mentioned: Rwanda MoH 7 priorities: https://newmoh.staging.risa.rw/1/strategic-plans-prioritiesThe value of diagnostic imaging for enhancing primary care in low- and middle-income countries: https://www.thelancet.com/journals/eclinm/article/PIIS2589-5370(24)00478-4/fulltextASSESSMENT OF DIAGNOSTIC GAPS AND RELEVANT DIGITAL HEALTH SOLUTIONS SUMMARY FINDINGS FROM PERU, INDIA, NIGERIA AND UGANDA: https://www.finddx.org/wp-content/uploads/2022/12/20201515_rep_assessment_dx_digital_health_FV_EN.pdfPrinciples for Digital Development: https://digitalprinciples.org/Overview of the validation platform: AI validation platformAbout the guests: Riveda Kadam: Experienced global health strategist with expertise in digital health, AI, and public health programs, driving impactful innovations and policy alignment.Andrew Muhire : Chief Digital Officer at the Rwanda Ministry of Health
Implementing a digital mental health solution in Rwanda. What does real, meaningful co-design look like?What does it mean to truly engage with a community to develop a solution they actually use and that addresses their problems? What incentives and mindsets allow us to, instead of 'driving' a certain technology into a context, stop and listen, and go in with no pre-conceived notion of what would be built?How do we get procurement and policy to really value and elevate equitable solutions?These are some of the great questions we covered in the latest podcast episode with Dr Jana Alagarajah. His wide experience, and work implementing a digital mental health tool with young people in Rwanda gives us some great talking points. Jana shares what he learned working with people and community leaders, as well as carers, and how they approached co-design and evaluation. We also talked about his insights from working in partnership with UNICEF, USAID, the King's Fund and Health Foundation. Dr Jana Alagarajah (MD MPH): Digital mental health specialist, UK-trained public health doctor, and psychiatrist co-designing equitable and impactful digital health innovation in Africa with young people as Technical Lead at YLabs. Partnering with UNICEF, USAID, and Gates Foundation, he leverages digital tools to strengthen health systems.Linkedin: https://www.linkedin.com/in/janaganalagarajah/YLabs: https://www.ylabsglobal.org/Grand Challenges Canada: https://www.grandchallenges.ca/Key Resources: Design:Co-design: YLabsâ Youth-driven approach to digital health focussing on co-designing with youth. Designing for diversity: Importance of co-creating culturally adapted tech to address health inequalities in diverse populations - report from NHS Race and Health Observatory. Regulation: Co-developed Africaâs first evidence-based digital mental health regulation (âHealthTech Hub Africa Digital Health Policy Blueprint 1.0â) with the Rwandan Ministry of Heath, Africa CDC, Jhpiego, Novartis Foundation covering key design elements such as interoperability, data privacy and UX approaches to meet the needs of diverse populations. Implementation:Digital stigma reduction tool: âPrepare for a Better Tomorrowâ (Rwanda, 2021): Rwandaâs first youth-driven, holistic digital learning and peer support platform to increase mental health literacy and psychosocial support for Rwandan youth aged 10-19 years old. Funded by Grand Challenges Canada. Digital tools for mental healthcare workers: âUSAID Kijana Nahodhaâ (Tanzania, 2023): digital mental health education and referral tool for community health workers in Tanzania as part of a $5.4M USAID grant, impacting 140,000 youth and 250 community health workers. Evaluation: Evidence generation: A systematic literature review evaluating the efficacy of digital mental health technologies for youth in low and middle-income countries: Alagarajah J, Ceccolini D, Butler S. Digital mental health interventions for treating mental disorders in young people based in low-and middle-income countries: A systematic review of the literature. Cambridge Prisms: Global Mental Health. 2024;11:e74. doi:10.1017/gmh.2024.71Innovative approaches to digital evaluation: At YLabs, we have utilized traditional evaluation methodologies such as cluster RCTs to evaluate our digital interventions. However, given the challenges in evaluating digital health solutions, novel methodologies, such as cyclical evaluation, can be used to maximize usability, and support integration into health systems.Â
OverviewWe are joined in this episode by Ruchit Nagar, the CEO and Co-Founder of Khushi Baby. For his efforts to deliver scalable public health impact, Ruchit has been recognized as a Forbes 30 under 30 leader in Health Care, a World Innovation Summit in Health Young Innovator, and a Distinguished Young Alumnus by by the Yale School of Public HealthThis episode covers into the role of India's Community health workers, or Accredited Social Health Activists (ASHAs). We hear from Ruchit's experience implementing with Khushi baby the challenges of integrating technology to support health workers. Ruchit also gives us a lowdown on India's digital public health approach and shares his key learnings and insights for people looking to implement tech in rural areas. We get into the challenges in policy, the data infrastructure layer and the importance of funding for sustainable health initiatives. Ruchit also emphasizes the need for better support and resources for ASHAs, the impact of technology on healthcare delivery, and the necessity of aligning incentives to improve health outcomes.Chapters00:00 Intro03:53 Khushi baby origins05:10 Challenges in Community Health Delivery07:18 The Role of ASHAs in Healthcare12:28 The Need for Integrated Solutions14:14 India public health digital ecosystem 10119:51 When "too much digital" gets in the way of good care22:28 Pitfalls in funding and investment approaches23:20 The 3 I's that drive implementation success31:56 Leveraging Data for Health System Strengthening35:53 Challenges in Health System Integration41:35 Measuring Impact and Effectiveness47:44 Aligning Incentives with Ground Realities53:56 Navigating Quality, Evaluation and Regulatory Challenges56:48 Future Directions for Khushi Baby01:02:02 Ruchit's top takeaway for developersReferencesKhushi baby websiteBook reference: Poor Economics by Abhijit V. Banerjee and Esther DufloTB free India 2025Like what you hear? Follow us. I'd love your feedback on what you found valuable, and what you want to see. Comment on youtube, and follow the page on linkedin. Want to share your story? Get in touch.
Episode 6: What can we learn about creating real impact for underserved communities from The Kingdom of Eswatini, a tiny country enveloped by South Africa?Join us on the latest episode of the Global Perspectives on Digital Health podcast where we hear from Executive Director of The Luke Commission, Echo Vanderwal about the way that set up healthcare services for those who needed it most and added digital in a way that enabled healthcare workers to deliver compassionate, high quality care. If you want to see how to create real impact for patients who have real transportation barriers rurally, then hear about what the Luke commission have done with :Telehealth hubs out of shipping containers with access to doctors by video, and secure vending machine for simple non controlled medications, all enabled via Starlink Drones piloted by local staff that deliver critical but high value medications like antivenom rapidly and effectively - and the impact this has had on the people who can now survive Black mamba neurotoxic envenomationOr if you want to learn how they:Enabled their community health workers by creating their own bespoke EHR and admin tool that was clinician centred and based on the fastest feedback loops possible (see the clip!)Created a culture of treating patients with compassion to overcome and the values that helped them make decisions and get through some challenging timesUsed the end goal of what they wanted great healthcare to look like, and made decisions on how digital could help them get thereProactively worked with the countryâs Civil Aviation Authority to work together on how to get in place what was necessary to be safe and deliver a high quality and reliable drone service (lesson here for regulators and developers on working closely together in unchartered territory with the end goal in mind - Stephen talked in episode 5 about how the regulator could and should be a facilitator. Thereâs so much insight packed into this episode - it was energising to record it to say the least and Iâm really inspired by the work and impact they have had. If you are a healthcare leader, developer or working to create impact for the people rurally, you have got to give this one a listen and share it with others who might benefit from Echoâs insights. Links: https://www.lukecommission.org/https://www.luvelo.org/
Episode 5: What is the right approach for regulation and evaluation of digital health technologies?In this conversation, Shubs Upadhyay interviews Stephen Gilbert, a professor of medical device regulatory science, about the challenges and successes of digital health regulation and implementation. They discuss the need for flexibility in regulation, the importance of feedback from clinicians and patients, and the evaluation of digital health technologies. They also explore the concept of suites or groupings of digital devices and the need for regulatory approaches that acknowledge their flexibility. Gilbert emphasizes the need for regulation to adapt to the changing landscape of digital health and to ensure that it is fit for purpose. Stephen also talks about what health system leaders, policy makers and developers can learn from the challenges of the DiGA fast track reimbursement framework in Germany, the PECAN framework in France and what is coming in the UK. As well as calling out the stark differences in approach from the FDA and the EU, Stephen helps understand the deeper reasons for the different approaches.On evaluation of effectivenss, Stephen emphasizes the importance of integrating different technologies into a cohesive system rather than treating them as isolated tools. The discussion also highlights the role of regulation in facilitating interoperability and promoting the use of digital technologies in healthcare. A great section on the need for long-term thinking on exactly how we want to transform healthcare delivery, setting clear goals, and continuous feedback loops is emphasized, along with the recognition that digital transformation in healthcare requires investment, embedding and time to get back the ROI. TakeawaysFlexibility is crucial in digital health regulation to adapt to the rapidly changing landscape of technology and healthcare.Feedback from clinicians and patients should be actively encouraged and integrated into the regulatory process.The evaluation of digital health technologies should consider their unique characteristics and the need for holistic assessment.Regulatory approaches should acknowledge the flexibility and groupings of digital devices, such as suites, to ensure they are fit for purpose.Digital healthcare involves the integration of various technologies, such as teleconsultation and remote monitoring, into a cohesive system.Regulation plays a crucial role in promoting interoperability and facilitating the use of digital technologies in healthcare.Long-term thinking and clear goals are necessary for effective digital transformation in healthcare.Continuous feedback loops and monitoring are essential for evaluating the effectiveness of digital health tools.Investment is required for digital transformation in healthcare, and initial costs may be higher before yielding dividends.Links to papers mentioned:Paper on regulation, reimbursement approach: flexible groupingsPaper on regulation, reimbursement approach: flexible suites of technologiesA/B testing framework mentioned hereStephen Gilbert, Professor at the Faculty of Medicine, Dresden University of Technology (Else KrĂśner Fresenius Center for Digital Health), leads a multidisciplinary team specializing in regulatory science for medical devices and in vitro diagnostic devices. With over 15 years of expertise in clinical research, computational biology, and regulatory science, he is committed to advancing digital health innovation and governance.
Episode 4: Creating real impact with AI in isolated communitiesIn this episode, Dino, a pioneer in the digital health landscape, shares his insights on how digital solutions and AI are revolutionizing the healthcare industry. He discusses the critical challenges such as overcrowded healthcare facilities and the urgent need for support for healthcare professionals. Dinoâs extensive work at Audere focuses on utilizing digital tools to improve healthcare delivery and outcomes, particularly in underserved communities.Key Topics:Digital Solutions for Overcrowded Healthcare Facilities: Dino explores how digital health can alleviate pressure on healthcare systems.AI Tools in Healthcare: The role of AI and language models in enhancing diagnostic and conversational capabilities in healthcare settings.Building Trust and Reliability: The importance of trust in technology, especially large language models, and how it's achieved through rigorous data handling and prompt engineering.Partnerships and Local Collaboration: How forming strong local partnerships contributes to the successful implementation of technology solutions in healthcare.Regulatory Support and Technology Verification: The necessity of regulatory backing in ensuring the safety and efficacy of new technologies.Impactful Insights:Improving access to rapid diagnostic testing through digital tools and AI.Enhancing patient-provider conversations with advanced language models to make healthcare more accessible and efficient.The significance of trust, reliability, and local partnerships in implementing successful digital health solutions.Utilization of specific data sets and prompt engineering to enhance the accuracy and relevance of responses from large language models.The role of specialization and niche focus in addressing particular healthcare and technology challenges.LinksAudere
Episode 3: How do we meaningfully bridge policy and real impact at the last mile of healthcare. Shubs Upadhyay interviews Bilal Mateen, Executive Director of Digital Square at PATH about digital health challenges and successes in underserved communities. They explore the importance of safety, regulatory considerations, and the need for inclusive data sets and data infrastructure. The discussion also covers the role of community health workers and the impact of AI and large language models in healthcare.Addendum : [As of October 2024 Bilal is now Chief AI Officer at PATH]Key Topics:Digital Public Goods: Exploring how digital solutions can be accessible public goods.Health Data Poverty: Discussing how this issue affects global health equity.The importance of strong data infrastructureRegulatory challenges : Bilal explores some of the work that still needs to be doneAI in Healthcare: Insights on the use of AI and large language models to improve healthcare outcomes.Chapters:00:00 Introduction and Background05:06 Digital Public Goods07:04 How Health Data Poverty Plays Out08:59 Reaching the Last Mile of Healthcare15:47 AI and Large Language Models in Healthcare29:00 Investing in Data Science Ecosystems and Regulatory Frameworks32:59 More Global Representation in Regulation37:03 Considering Local Nuances in AI Deployment39:12 Divergent Approaches to Regulating LLMs45:28 Regulation of LLMs as Medical Devices48:40 Recommendations to Innovators about Healthcare RegulationLinks:AMIE paper by Alan Karthikesalingam et alDigitalsquare.orgAudere WebsiteViamo Canada Call CentreSiontis et al Paper on Diagnostic test comparisons