DiscoverTest & Code : Python Testing
Test & Code :  Python Testing
Claim Ownership

Test & Code : Python Testing

Author: Brian Okken

Subscribed: 3,336Played: 49,135


Test & Code is a weekly podcast hosted by Brian Okken.
The show covers a wide array of topics including software engineering, development, testing, Python programming, and many related topics.
When we get into the implementation specifics, that's usually Python, such as Python packaging, tox, pytest, and unittest. However, well over half of the topics are language agnostic, such as data science, DevOps, TDD, public speaking, mentoring, feature testing, NoSQL databases, end to end testing, automation, continuous integration, development methods, Selenium, the testing pyramid, and DevOps.
149 Episodes
Corey Quinn is the Chief Cloud Economist at The Duckbill Group. He's also a podcaster and writes a newsletter. And he also automates things with Python. But he doesn't write tests. Let's find out why. Reason for the interview. Rough summary of a twitter conversation: Corey: What podcasts should I try to get an invite onto? ToonArmyCaptain: Python Bytes, Test & Code, Talk Python Corey: But... I don't test my code, "crappy Python" is all I write, and I'd feel like a giant imposter. So yeah, I'd be game. link ( So here we are. This diagram is referenced in the show, the Last Week In AWS Newsletter Production Pipeline ( Special Guest: Corey Quinn.
How do you test installed packages using Also, a couple followups from last week's episode on using coverage for single file applications.
Have you ever written a single file Python application or script? Have you written tests for it? Do you check code coverage? This is the topic of this weeks episode, spurred on by a listener question. The questions: * For single file scripts, I'd like to have the test code included right there in the file. Can I do that with pytest? * If I can, can I use code coverage on it? The example code discussed in the episode: ``` def foo(): return 5 def main(): x = foo() print(x) if name == 'main': # pragma: no cover main() test code To test: pip install pytest pytest To test with coverage: put this file ( in a directory by itself, say foo then from the parent directory of foo: pip install pytest-cov pytest --cov=foo foo/ To show missing lines pytest --cov=foo --cov-report=term-missing foo/ def test_foo(): assert foo() == 5 def test_main(capsys): main() captured = capsys.readouterr() assert captured.out == "5\n" ``` Suggestion by @cfbolz ( if you need to import pytest: if __name__ == '__main__': # pragma: no cover main() else: import pytest
Building any software, including web apps and APIs requires testing. There's automated testing, and there's manual testing. In between that is exploratory testing aided by automation tools. Michael Kennedy joins the show this week to share some of the tools he uses during development and maintenance. We talk about tools used for semi-automated exploratory testing. We also talk about some of the other tools and techniques he uses to keep Talk Python Training, Talk Python, and Python Bytes all up and running smoothly. We talk about: Postman ngrok sitemap link testing scripts for manual processes using failover servers during maintenance, redeployments, etc gitHub webhooks and scripts to between fail over servers and production during deployments automatically floating IP addresses services to monitor your site: StatusCake, BetterUptime the affect of monitoring on analytics crash reporting: Rollbar, Sentry response times load testing: Locus Special Guest: Michael Kennedy.
A discussion about mocking in Python with the original contributor of unittest.mock, Michael Foord. Of course we discuss mocking and unittest.mock. We also discuss: * testing philosophy * unit testing and what a unit is * TDD * where Michael's towel is, and what color Micheal was instrumental in the building of testing tools for Python, and continues to be a pragmatic source of honest testing philosopy in a field that has a lot of contradictory information. Special Guest: Michael Foord.
Test Driven Development, TDD, is not easy to incorporate in your daily development. Martin and Brian discuss TDD and testing and Martin's experience with testing, TDD, and using it for code involved with scientific research. We discuss lots of topics around this, including: What is TDD? Should research software be tested in order to be trusted? Time pressure and the struggle to get code done quickly. How do you make time for tests also? Is testing worth it for code that will not be reused? Sometimes it's hard to know how to test something. Maybe people should learn to test alongside learning how to code. A desire for a resource of testing concepts for non-CS people. Are the testing needs and testing information needs different in different disciplines? Biology, Physics, Astrophysics, etc. Do they have different testing needs? Do we need a "how to test" resource for each? Special Guest: Martin Héroux.
Completely nerding out about pytest markers with Anthony Sottile. Some of what we talk about: Running a subset of tests with markers. Using marker expressions with and, or, not, and parentheses. Keyword expressions also can use and, or, not, and parentheses. Markers and pytest functionality that use mark, such as parametrize, skipif, etc. Accessing markers with itermarkers and get_closest_marker through item. Passing values, metadata through markers to fixtures or hook functions. Special Guest: Anthony Sottile.
MongoDB is possibly the most recognizable NoSQL document database. Mark Smith, a developer advocate for MongoDB, answers my many questions about MongoDB. We cover some basics, but also discuss some advanced features that I never knew about before this conversation. Special Guest: Mark Smith.
Visual Testing has come a long way from the early days of x,y mouse clicks and pixel comparisons. Angie Jones joins the show to discuss how modern visual testing tools work and how to incorporate visual testing into a complete testing strategy. Some of the discussion: Classes of visual testing: problems with pixel to pixel testing DOM comparisons, css, html, etc. AI driven picture level testing, where failures look into the DOM to help describe the problem. Where visual testing fits into a test strategy. Combining "does this look right" visual testing with other test workflows. "A picture is worth a thousand assertions" - functional assertions built into visual testing. Baselining pictures in the test workflow. Also discussed: - automation engineer - Test Automation University Special Guest: Angie Jones.
Scientists learn programming as they need it. Some of them learn it in college, but even if they do, that's not their focus. It's not surprising that sharing the software used for scientific research and papers is spotty, at best. And what about testing? We'd hope that the software behind scientific research is tested. But why would we expect that? We're lucky if CS students get a class or two that even mentions automated tests. Why would we expect other scientists to just know how to test their code? Martin works in research and this discussion is about software and testing in scientific research and academia. Special Guest: Martin Héroux.
Talking with Nalin Parbhu about the software evolution towards more test automation and the creation of Infuse and useMango. We talk a software development and "shift left" where automated tests and quality checks have moved earlier into the software lifecycle. Software approaches and where quality fits in Shift left Test automation Roles of software developers, SDETs (software development engineer in test), testers, QA, etc. Developers doing testing and devops Automated testing vs manual testing Regression testing, UI testing, black bock testing Unit testing, white box, API, end to end testing User acceptance testing (UAT) Mullet Methodology (Agile at the front, Waterfall at the back) Waterwheel Methodology (Requirements -> iterative development -> QA) What's an agile team? Developer resistance to testing Manifesto for agile software development Iterative development Adapting to change Agility: being able to change course quickly Special Guests: Nalin Parbhu and Ola Omiyale.
Your test suite tells you about the quality of your code under test. Mutation testing is a way to tell you about the quality of your test suite. Anders Hovmöller wrote mutmut ( for mutation testing in Python, and can be used with pytest, unittest, and others. In this episode, Anders explains mutation testing, how mutation testing with mutmut works, and good workflows. Special Guest: Anders Hovmöller.
Matt Harrison, author of many Python books, is putting together a course, Effective Book Authoring, to help other people write and publish books. As part of this course, he's including interviews with people who have already written books, including me. This is that interview. We discuss: * Why I wrote "Python Testing with pytest" * Self publishing vs working with a publisher * The writing, editing, and publishing process * Writing format * Book promotion * Advice to other writers Special Guest: Matt Harrison.
Wearable technology is not just smart consumer devices like watches and activity trackers. Wearable tech also includes one off projects by designers, makers, and hackers and there are more and more people producing tutorials on how to get started. Wearable tech is also a great way to get both kids and adults excited about coding, electronics, and in general, engineering skills. Sophy Wong is a designer who makes really cool stuff using code, technology, costuming, soldering, and even jewelry techniques to get tech onto the human body. Sophy joins the show to answer my many questions about getting started safely with wearable tech. Some of the questions and topics: Can I wash my clothing if I've added tech to it? Is there any danger in wearing technology or building wearable tech? Are there actual wires and cables conductive thread in the fabric and textiles of some wearable tech projects? What's a good starter project? Especially if I want to do a wearable tech project with my kids? Dealing with stretch with clothing and non-bendy electronics. Some questions around the Sophy Wong and HackSpace "Wearable Tech Projects" book. How did you get into wearable tech? Do you have a favorite project? Can I get into wearable tech if I don't know how to code or solder? Are these projects accessible to people with limited budgets? Making projects so you can reuse the expensive bits on multiple projects. Special Guest: Sophy Wong.
All test suites start fast. But as you grow your set of tests, each test adds a little bit of time to the suite. What can you do about it to keep test suites fast? Some things, like parallelization, are applicable to many domains. What about, for instance, Django applications? Well, Adam Johnson has thought about it a lot, and is here to tell us how we can speed up our Django test suites. Topics include: parallelizing tests moving from disk to memory using fake data and factory functions targeted mocking Special Guest: Adam Johnson.
Within software projects, there are lots of metrics we could measure. But which ones really matter. Instead of a list, Benjamin Harding shares with us a way of thinking about business outcomes that can help us with every day decision making. We talk about: * Business outcomes vs vanity metrics * As a developer, how do you keep business outcomes in mind * Thinking about customer value all the time * Communicating decisions and options in terms of costs and impact on business outcomes * Company culture and it's role in reinforcing a business outcome mindset * And even the role of team lead as impact multiplier I really enjoyed this conversation. But I admit that at first, I didn't realize how important this is on all software development. Metrics are front and center in a web app. But what about a service, or an embedded system with no telemetry. It still matters, maybe even more so. Little and big decisions developers face every day that have impact on costs and benefits with respect to customer value and business outcome, even if it's difficult to measure. Special Guest: Benjamin Harding.
Hackathons have been spreading around the world; many at university campuses. Major League Hacking, MLH, has been encouraging and helping hackathons. Hacking can be thought of as tinkering. Taking things apart and putting them back together as an interesting experience. There's always been some of this as part of software culture. The people at Major League Hacking have taken this to a whole new level, bringing together Tech creators who enjoy playing around with and crating new technology, on campuses, and now in virtual spaces, all over the world. Jonathon Gottfried, one of the cofounders of Major League Hacking, joins the show to talk about: hacker meetups and events hackathons what it's like to go to a hackathon how to help out with hackathons as an experienced engineer, even virtually as a mentor hackathons continuing virtually during the pandemic internships and fellowships on open source projects to help students gain experience, even during the pandemic MLH approach to internships, giving interns a support group, including peers, mentors, and project maintainers and MLH itself Special Guest: Jon Gottfried.
Using mock objects during testing in Python. Anna-Lena joins the podcast to teach us about mocks and using unittest.mock objects during testing. We discuss: - the different styles of using mocks - pros and cons of mocks - dependency injection - adapter pattern - mock hell - magical universe - and much more Special Guest: Anna-Lena Popkes.
Some people avoid writing tests. Some drudge through it painfully. There is a better way. In this episode, I'm going to share some advice from Luke Plant on how to "Test Smarter, Not Harder" (
virtualenv supports six shells: bash, csh, fish, xonsh, cmd, posh. Each handles prompts slightly differently. Although the virtualenv custom prompt behavior should be the same across shells, Brian Skinn noticed inconsistencies. He set out to fix those inconsistencies. That was the start of an adventure in open source collaboration, shell prompt internals, difficult test problems, and continuous integration quirks. Brian Skinn initially noticed that on Windows cmd, a space was added between a prefix defined by --prompt and the rest of the prompt, whereas on bash no space was added. For reference, there were/are three nominal virtualenv prompt modification behaviors, all of which apply to the prompt changes that are made at the time of virtualenv activation: If the environment variable VIRTUAL_ENV_DISABLE_PROMPT is defined and non-empty at activation time, do not modify the prompt at all. Otherwise: If the --prompt argument was supplied at creation time, use that argument as the prefix to apply to the prompt; or, If the --prompt argument was not supplied at creation time, use the default prefix of "({{ envname }}) " as the prefix (the environment folder name surrounded by parentheses, and with a trailing space after the last paren. Special Guest: Brian Skinn.
Comments (6)

Max Ong Zong Bao

I would be interested to invite you as keynote for PyCon Singapore in June and I would love to know more on your PyTest online courses when it releases.

Jan 3rd

Александр Михеев

Such a great episode! I've even listened to it twice

Dec 9th

Eduardo Costa

I enjoyed this episode. Hope more episodes on this subject.

Mar 15th

Leora Juster

react tables

Jan 12th


another great episode

Dec 16th

Antonio Andrade

Thanks for sharing these good tips

Dec 9th
Download from Google Play
Download from App Store