The Performance Audit Report

Maura Hayes-Chaffe is the Deputy Comptroller for Audit in New York City.Maura joined us to discuss the Bureau of Audit, including:the challenges with recruiting and attracting performance auditorsnew approaches to conducting PAsthe changing environment (e.g., economics)equity in PA, citizen powered audits, and a recent housing related auditthe use of data, including open data, by the audit team.LinksNYC Office of the ComptrollerThis episode's web pageAbout this podcast The podcast for performance auditors. Hosted by Conor McGarrity and Yusuf Moolla. Produced by PA Reports - Performance Audit Research.  Get updates New episodes, and a fortnightly newsletter with a selection of PA reports and practices from audit offices. https://pareports.com/#cta-subscribe This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit performanceaudit.substack.com

Our guest is Cathy O'Neil, author of Weapons of Math Destruction: how big data increases inequality and threatens democracy.Cathy joined us to discuss critical aspects for auditors to consider, including:The human element: why understanding 'for whom an algorithm might fail' is more important than the detailed technical design of an algorithm.Algorithm testing: the importance of post-deployment testing to ensure there are no unintended adverse outcomes. Setting standards: the need for industry-wide standards so that regulators can ensure there is no bias or discrimination in how algorithms operate. Laggers and leaders: the four big industries/domains using algorithms that need to step up audit in their space and how law-makers and regulators are catching up.Emerging AI, e.g. language models/chatbots - why trust by users is key, and the pros and cons of using these models.LinksCathy's company, ORCAA Weapons of Math Destruction (book)  This episode's web page: https://pareports.com/podcast/24 About this podcast The podcast for performance auditors.Hosted by Conor McGarrity and Yusuf Moolla.Produced by PA Reports - Performance Audit Research. Get updates New episodes, and a fortnightly newsletter with a selection of PA reports and practices from audit offices. https://pareports.com/#cta-subscribe This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit performanceaudit.substack.com

This episode featured guests from the North Dakota State Audit Office:Allison Bader, Performance Audit Manager and Claudia Kuball, Performance Auditor.In this episode we discuss:Team diversity – the value of recruiting performance auditors from different backgrounds, incl. how specialist expertise can help focus on major risksImpact – their communications team is instrumental in engaging citizens and maximising audit impactSignificant audit – their audit of Storage and Handling of COVID Vaccines including its design, execution and findingsCitizen engagement – how North Dakotans can get involved, and even speak to the State Auditor directly on radio every Wed morningLinks:Office of the State Auditor, North Dakota: https://www.nd.gov/auditor Submit ideas for audit topics here What’s On Your Mind – the radio program that Josh Gallion (State Auditor) guest appears on from 10-11 CST every Wednesday: https://kfyr.iheart.com/ Sign up for the office's quarterly e-newsletter here The office on Facebook, LinkedIn and YouTube This episode's webpage: https://pareports.com/podcast/23About this podcast The podcast for performance auditors.Hosted by Conor McGarrity and Yusuf Moolla.Produced by PA Reports - Performance Audit Research.Get updates New episodes, and a fortnightly newsletter with a selection of PA reports and practices from audit offices. https://pareports.com/#cta-subscribe This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit performanceaudit.substack.com

In this episode, we discuss how audit offices are engaging citizens in performance audits. The show notes here and the full article here contain further details and examples.We delve into how audit offices are engaging citizens:Forward work plans – consulting broadly for focus areasRefining the broad focus areas for specific citizen concerns Audit planning – citizen input on audit scope Audit execution – seeking agency staff views during fieldwork Reporting – tailoring reports for citizens and civil society organisations Links: This episode's web page with further details and examples - https://pareports.com/podcast/22 The full accompanying article - https://riskinsights.com.au/engaging-citizens-in-pa/About this podcast The podcast for performance auditors. Hosted by Conor McGarrity and Yusuf Moolla. Produced by PA Reports - Performance Audit Research.Get updates New episodes, and a fortnightly newsletter with a selection of PA reports and practices from audit offices. https://pareports.com/#cta-subscribe This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit performanceaudit.substack.com

This originally aired as episode 50 of The Assurance Show.In this episode, we discuss the benefits of adopting a data focused approach during audit planning.Link: This episode's web page - https://pareports.com/podcast/21About this podcast The podcast for performance auditors.Hosted by Conor McGarrity and Yusuf Moolla.Produced by PA Reports - Performance Audit Research.Get updates New episodes, and a fortnightly newsletter with a selection of PA reports and practices from audit offices. https://pareports.com/#cta-subscribe This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit performanceaudit.substack.com

This originally aired as episode 49 of The Assurance Show.In this episode, we discuss the importance of balance in your work program - covering each of the four objectives:EffectivenessEfficiencyEconomyComplianceLink: This episode's web page - https://pareports.com/podcast/20About this podcast The podcast for performance auditors.Hosted by Conor McGarrity and Yusuf Moolla.Produced by PA Reports - Performance Audit Research.Get updates New episodes, and a fortnightly newsletter with a selection of PA reports and practices from audit offices. https://pareports.com/#cta-subscribe This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit performanceaudit.substack.com

This originally aired as episode 48 of The Assurance Show.In this episode, we discuss strategic audit planning (a.k.a. forward work planning).Understanding your community / public expectationsUnderstanding your elected body / parliamentariansUnderstanding the entities within your audit office's jurisdictionLink: This episode's web page - https://pareports.com/podcast/19About this podcast The podcast for performance auditors.Hosted by Conor McGarrity and Yusuf Moolla.Produced by PA Reports - Performance Audit Research.Get updates New episodes, and a fortnightly newsletter with a selection of PA reports and practices from audit offices. https://pareports.com/#cta-subscribe This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit performanceaudit.substack.com

Michael Pickup is the Auditor General of British Columbia in Canada.In this episode we discuss:the changing role of performance auditrecruitment and retentionworkplace flexibilitypeople and product focused innovationthe use of video and other media to increase reachLinks:Michael Pickup on LinkedIn - https://www.linkedin.com/in/michael-pickup-fcpa-fca-a1ab91168/ Auditor General of British Columbia - https://www.bcauditor.com/ This episode's web page - https://pareports.com/podcast/18  About this podcast The podcast for performance auditors.Hosted by Conor McGarrity and Yusuf Moolla.Produced by PA Reports - Performance Audit Research.Get updates New episodes, and a fortnightly newsletter with a selection of PA reports and practices from audit offices. https://pareports.com/#cta-subscribe This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit performanceaudit.substack.com

This originally aired as episode 39 of The Assurance Show.In using data for audits, quality assurance of our work is important.In this episode, we discuss: 1.  The four main QA stepsTechnical QA Functional QA Business QA (validation) Audit QA 2. Why we need a QA PlanLink: This episode's web page - https://pareports.com/podcast/17About this podcast The podcast for performance auditors.Hosted by Conor McGarrity and Yusuf Moolla.Produced by PA Reports - Performance Audit Research.Get updates New episodes, and a fortnightly newsletter with a selection of PA reports and practices from audit offices. https://pareports.com/#cta-subscribe This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit performanceaudit.substack.com

This is an edited version of episode 13 of The Assurance Show.In this episode we discuss the use of Open Data.Links The Open Data Handbook - https://opendatahandbook.org/ Google dataset search - https://datasetsearch.research.google.com/ This episode's web page - https://pareports.com/podcast/16About this podcast The podcast for performance auditors.Hosted by Conor McGarrity and Yusuf Moolla.Produced by PA Reports - Performance Audit Research.Get updates New episodes, and a fortnightly newsletter with a selection of PA reports and practices from audit offices. https://pareports.com/#cta-subscribe This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit performanceaudit.substack.com

Patrick Barr is Director, Performance Audit at the Northern Ireland Audit Office.In this episode we discuss:Recruitment challenges and staff attraction strategiesThe audit office's approach to topic selection and responsive auditingUpskilling performance auditors in the use of dataRecent significant reports and their impactOutreach to other offices to share experiences and collaborate.Links:Patrick Barr on LinkedIn - https://www.linkedin.com/in/patrick-barr-22a5233 Northern Ireland Audit Office - https://www.niauditoffice.gov.uk/ This episode's web page - https://pareports.com/podcast/15 About this podcast The podcast for performance auditors.Hosted by Conor McGarrity and Yusuf Moolla.Produced by PA Reports - Performance Audit Research.Get updates New episodes, and a fortnightly newsletter with a selection of PA reports and practices from audit offices. https://pareports.com/#cta-subscribe This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit performanceaudit.substack.com

This originally aired as Episode 32 of The Assurance Show.Alli is an information design consultant and host of the Data Viz Today podcast.We talk about a range of data visualization matters, including 3 key mistakes that people make when starting out in dataviz (and solutions to those). You can reach out to Alli via her website - https://www.allitorban.comLinksFinancial Times Visual VocabularyStephanie Evergreen's What, So What, Now WhatTemplate for all those charts done in excel Alli's brainstorming worksheet for developing visual metaphors Alli's brainstorming template for focusing the story in your data viz Alli's Tessellated Unit Chart for Viz for Social Good Viz for Social Good Reuter's "Drowning in Plastic" This episode's web page - https://pareports.com/podcast/14About this podcast The podcast for performance auditors.Hosted by Conor McGarrity and Yusuf Moolla.Produced by PA Reports - Performance Audit Research.Get updates New episodes, and a fortnightly newsletter with a selection of PA reports and practices from audit offices. https://pareports.com/#cta-subscribe This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit performanceaudit.substack.com

In this episode, we talk about how audit offices monitor (and report on) the status of implementation of performance audit recommendations.We monitor the status of our PA recommendations for several reasons:We want to ensure that our work has impactWe may have a legislative requirement to follow-upStakeholders have asked us to provide updates on progress against agreed actionsWe want to check if further work is required in areas that we've previously audited.How we do this ranges in nature and scale, from ongoing monitoring that relies on auditee submissions to full-scale follow-up audits with significant levels of testing.Links:The associated article - https://riskinsights.com.au/monitoring-the-status-of-performance-audit-recommendations This episode's webpage - https://pareports.com/podcast/13About this podcast The podcast for performance auditors.Hosted by Conor McGarrity and Yusuf Moolla.Produced by PA Reports - Performance Audit Research.Get updates New episodes, and a fortnightly newsletter with a selection of PA reports and practices from audit offices. https://pareports.com/#cta-subscribe This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit performanceaudit.substack.com

This originally aired as Episode 37 of The Assurance Show.Morag (Senior Manager) and Gemma (Director) are Performance Auditors with Audit Scotland.In this episode we discuss how they are using data in their audits, their experience in developing Audit Scotland's data capabilities, collaboration with other audit offices and what the future roadmap looks like.We also discuss their focus on ensuring their reports meet their audience’s needs, and how data visualisations and charts are being used to communicate key messages.Link: This episode's web page - https://pareports.com/podcast/12About this podcast The podcast for performance auditors.Hosted by Conor McGarrity and Yusuf Moolla.Produced by PA Reports - Performance Audit Research.Get updates New episodes, and a fortnightly newsletter with a selection of PA reports and practices from audit offices. https://pareports.com/#cta-subscribe This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit performanceaudit.substack.com

Kelsey Thompson is an Assistant City Auditor with the Office of the City Auditor in Austin, Texas.In this episode we discuss:What Kelsey enjoys about performance audit in local governmentThe role of the Office of the City AuditorWhy performance auditors need a data ‘mindset’ Topical reports on issues such as climate change, homelessness and policingWe also talk about Audit ATX - the Office's podcast that Kelsey hosts - and its many benefits, including:how auditees are listening to it as a ‘starting point’ to understand the PA processits use as a recruitment tool to inform potential new hireshow it maximises the reach and impact of their audits for community membershow podcast feedback can inform future audit topic selection and focus Audit ATX is available on Apple Podcasts and Spotify.In the episode, you will hear Kelsey saying that she is a supervising senior auditor - she was at the time of recording, and was promoted to Assistant City Auditor in the short window between recording this episode and publishing it. Congrats Kelsey.Links:Kelsey Thompson on LinkedIn - https://www.linkedin.com/in/kelslau/ Office of the City Auditor at the City of Austin - https://www.austintexas.gov/department/auditor Audit Reports - https://www.austintexas.gov/page/audit-reports Audit ATX - https://podcasts.apple.com/us/podcast/audit-atx/id1541056396 This episode's webpage - https://pareports.com/podcast/11/ About this podcast The podcast for performance auditors.Hosted by Conor McGarrity and Yusuf Moolla.Produced by PA Reports - Performance Audit Research.Get updates New episodes, and a fortnightly newsletter with a selection of PA reports and practices from audit offices. https://pareports.com/#cta-subscribe This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit performanceaudit.substack.com

This originally aired as Episode 30 of The Assurance Show.John talks to us about the role of the CTO in an audit office.He shares his views on a range of issues including: How he supports the audit officeCyber and data governance frameworksEvolving IT practices that auditors need to keep a close eye onLinksJohn Moore on LinkedIn (www.linkedin.com/in/john-moore-01986a7a)Where John works (Queensland Audit Office) (www.qao.qld.gov.au)Where John studied (Loughborough University) (www.lboro.ac.uk)The Essential Eight (strategies to mitigate cyber security incidents) (www.cyber.gov.au/acsc/view-all-content/essential-eight/essential-eight-explained)ISO27001 (www.iso.org/isoiec-27001-information-security.html)Microsoft password policy recommendations (docs.microsoft.com/en-us/microsoft-365/admin/misc/password-policy-recommendations)About this podcast The podcast for performance auditors.Hosted by Conor McGarrity and Yusuf Moolla.Produced by PA Reports - Performance Audit Research.Get updates New episodes, and a fortnightly newsletter with a selection of PA reports and practices from audit offices. https://pareports.com/#cta-subscribe This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit performanceaudit.substack.com

Performance audits are usually conducted as reasonable assurance engagements. Many audit offices also provide other types of assurance or other forms of reporting. These are often produced by performance audit functions, or are closely related to PA.Sometimes these alternate forms of delivery can be faster and cheaper, without diminishing the intended impact.In this episode, we briefly introduce 15 types of alternate reporting/assurance. The accompanying blog article has further detail and examples. Links:This episode's webpage - https://pareports.com/podcast/9/The accompanying blog article - https://riskinsights.com.au/alternate-forms-of-pa-assurance-and-reporting About this podcast The podcast for performance auditors.Hosted by Conor McGarrity and Yusuf Moolla.Produced by PA Reports - Performance Audit Research.Get updates New episodes, and a fortnightly newsletter with a selection of PA reports and practices from audit offices. https://pareports.com/#cta-subscribe This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit performanceaudit.substack.com

This originally aired as Episode 35 of The Assurance Show.Stephen McAteer leads the data science team within Performance Audit at VAGO, the Victorian Auditor General's Office.  In this episode, we discuss:What VAGO doesStephen's approach to helping auditors use dataWhy it's important for all performance auditors to have data skillsHow VAGO has used open data to improve transparencyYou can reach out to Stephen via LinkedIn (www.linkedin.com/in/smcateer).LinksVAGO Audit Report – Measuring  and reporting on service delivery with interactive dashboard for exploring Victorian department performance (www.audit.vic.gov.au/report/measuring-and-reporting-service-delivery)VAGO Audit Report – Accessibility  of Tram Services with interactive map on the accessibility of Melbourne’s tram network (www.audit.vic.gov.au/report/accessibility-tram-services)VAGO Audit Report – Council Libraries which uses Data Envelopment Analysis (DEA) to evaluate efficiency (www.audit.vic.gov.au/report/council-libraries)About this podcast The podcast for performance auditors.Hosted by Conor McGarrity and Yusuf Moolla.Produced by PA Reports - Performance Audit Research.Get updates New episodes, and a fortnightly newsletter with a selection of PA reports and practices from audit offices. https://pareports.com/#cta-subscribe This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit performanceaudit.substack.com

Mike Scott is Lead Auditor and Manager Assurance and Audit at the New Zealand Department of Internal Affairs. He has over 30 years of audit experience, including 15 years in performance audit with the New Zealand Office of the Auditor-General and 17 years with the UK National Audit Office.In this episode, we discuss:How the discipline of performance audit/assessment is similar, whether you're auditing performance from outside the organization or improving it from the insideMaximising the impact of PAWhat has, and what hasn't, changed within the world of PASome ideas on recruitment and retention of audit professionalsLinks:Mike Scott on LinkedInThis episode's webpage - https://pareports.com/podcast/7/ About this podcast The podcast for performance auditors.Hosted by Conor McGarrity and Yusuf Moolla.Produced by PA Reports - Performance Audit Research.Get updates New episodes, and a fortnightly newsletter with a selection of PA reports and practices from audit offices. https://pareports.com/#cta-subscribe This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit performanceaudit.substack.com

This originally aired as Episode 34 of The Assurance Show.Taka Ariga is the inaugural Chief Data Scientist at the US Government Accountability Office.  He also leads the GAO's Innovation Lab.  In this episode, we discussed:What the GAO doesHow data science is helping GAOInnovation and experimentation at GAOMachine learning assuranceYou can reach out to Taka via the GAO website.About this podcast The podcast for performance auditors.Hosted by Conor McGarrity and Yusuf Moolla.Produced by PA Reports - Performance Audit Research.Get updates New episodes, and a fortnightly newsletter with a selection of PA reports and practices from audit offices. https://pareports.com/#cta-subscribe This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit performanceaudit.substack.com