Python Bytes

Sponsored by Datadog: pythonbytes.fm/datadog Michael #1: Data driven journalism via cjworkbench via Michael Paholski The data journalism platform with built in training Think spreadsheet + ETL automation Designed around modular tools for data processing -- table in, table out -- with no code required Features include: Modules to scrape, clean, analyze and visualize data An integrated data journalism training program Connect to Google Drive, Twitter, and API endpoints. Every action is recorded, so all workflows are repeatable and transparent All data is live and versioned, and you can monitor for changes. Write custom modules in Python and add them to the module library Brian #2: remi: A Platform-independent Python GUI library for your applications. Python REMote Interface library. “Remi is a GUI library for Python applications which transpiles an application's interface into HTML to be rendered in a web browser. This removes platform-specific dependencies and lets you easily develop cross-platform applications in Python!” No dependencies. pip install git+https://github.com/dddomodossola/remi.git doesn’t install anything else. Yes. Another GUI in a web page, but for quick and dirty internal tools, this will be very usable. Basic app: import remi.gui as gui from remi import start, App class MyApp(App): def __init__(self, *args): super(MyApp, self).__init__(*args) def main(self): container = gui.VBox(width=120, height=100) self.lbl = gui.Label('Hello world!') self.bt = gui.Button('Press me!') self.bt.onclick.do(self.on_button_pressed) container.append(self.lbl) container.append(self.bt) return container def on_button_pressed(self, widget): self.lbl.set_text('Button pressed!') self.bt.set_text('Hi!') start(MyApp) Michael #3: Typer Build great CLIs. Easy to code. Based on Python type hints. Typer is FastAPI's little sibling. And it's intended to be the FastAPI of CLIs. Just declare once the types of parameters (arguments and options) as function parameters. You do that with standard modern Python types. You don't have to learn a new syntax, the methods or classes of a specific library, etc. Based on Click Example (min version) import typer def main(name: str): typer.echo(f"Hello {name}") if __name__ == "__main__": typer.run(main) Brian #4: Effectively using Matplotlib Chris Moffitt “… I think I was a little premature in dismissing matplotlib. To be honest, I did not quite understand it and how to use it effectively in my workflow.” That very much sums up my relationship with matplotlib. But I’m ready to take another serious look at it. one reason for complexity is 2 interfaces MATLAB like state-based interface object based interface (use this) recommendations: Learn the basic matplotlib terminology, specifically what is a Figure and an Axes . Always use the object-oriented interface. Get in the habit of using it from the start of your analysis. Start your visualizations with basic pandas plotting. Use seaborn for the more complex statistical visualizations. Use matplotlib to customize the pandas or seaborn visualization. Runs through an example Describes figures and plots Includes a handy reference for customizing a plot. Related: StackOverflow answer that shows how to generate and embed a matplotlib image into a flask app without saving it to a file. Style it with pylustrator.readthedocs.io :) Michael #5: Django Simple Task django-simple-task runs background tasks in Django 3 without requiring other services and workers. It runs them in the same event loop as your ASGI application. Here’s a simple overview of how it works: On application start, a queue is created and a number of workers starts to listen to the queue When defer is called, a task(function or coroutine function) is added to the queue When a worker gets a task, it runs it or delegates it to a threadpool On application shutdown, it waits for tasks to finish before exiting ASGI server It is required to run Django with ASGI server. Example from django_simple_task import defer def task1(): time.sleep(1) print("task1 done") async def task2(): await asyncio.sleep(1) print("task2 done") def view(requests): defer(task1) defer(task2) return HttpResponse(b"My View") Brian #6: PyPI Stats at pypistats.org Simple interface. Pop in a package name and get the download stats. Example use: Why is my open source project now getting PRs and issues? I’ve got a few packages on PyPI, not updated much. cards and submark are mostly for demo purposes for teaching testing. pytest-check is a pytest plugin that allows multiple failures per test. I only hear about issues and PRs on one of these. So let’s look at traffic. cards: downloads day: 2 week: 24 month: 339 submark: day: 5 week: 9 month: 61 pytest-check: day: 976 week: 4,524 month: 19,636 That totally explains why I need to start actually supporting pytest-check. Cool. Note: it’s still small. Top 20 packages are all downloaded over 1.3 million times per day. Extras: Comment from January Python PDX West meetup “Please remember to have one beginner friendly talk per meetup.” Good point. Even if you can’t present here in Portland / Hillsboro, or don’t want to, I’d love to hear feedback of good beginner friendly topics that are good for meetups. PyCascades 2020 discount code listeners-at-pycascades for 10% off FireFox 72 is out with anti-fingerprinting and PIP - Ars Technica Joke: Language essays comic

Sponsored by us! Support us by visiting pythonbytes.fm/biz [courses] and pythonbytes.fm/pytest [book], or becoming a patron at patreon.com/pythonbytes Brian #1: Meditations on the Zen of Python Moshe Zadka The Zen of Python is not "the rules of Python" or "guidelines of Python". It is full of contradiction and allusion. It is not intended to be followed: it is intended to be meditated upon. Moshe give some of his thoughts on the different lines of the Zen of Python. Full Zen of Python can be found here or in a REPL with import this A few Beautiful is better than ugly Consistency helps. So black, flake8, pylint are useful. “But even more important, only humans can judge what humans find beautiful. Code reviews and a collaborative approach to writing code are the only realistic way to build beautiful code. Listening to other people is an important skill in software development.” Complex is better than complicated. “When solving a hard problem, it is often the case that no simple solution will do. In that case, the most Pythonic strategy is to go "bottom-up." Build simple tools and combine them to solve the problem.” Readability counts “In the face of immense pressure to throw readability to the side and just "solve the problem," the Zen of Python reminds us: readability counts. Writing the code so it can be read is a form of compassion for yourself and others.” Michael #2: nginx raided by Russian police Russian police have raided today the Moscow offices of NGINX, Inc., a subsidiary of F5 Networks and the company behind the internet's most popular web server technology. Russian search engine Rambler.ru claims full ownership of NGINX code. Rambler claims that Igor Sysoev developed NGINX while he was working as a system administrator for the company, hence they are the rightful owner of the project. Sysoev never denied creating NGINX while working at Rambler. In a 2012 interview, Sysoev claimed he developed NGINX in his free time and that Rambler wasn't even aware of it for years. Update Promptly following the event we took measures to ensure the security of our master software builds for NGINX, NGINX Plus, NGINX WAF and NGINX Unit—all of which are stored on servers outside of Russia. No other products are developed within Russia. F5 remains committed to innovating with NGINX, NGINX Plus, NGINX WAF and NGINX Unit, and we will continue to provide the best-in-class support you’ve come to expect. Brian #3: I'm not feeling the async pressure Armin Ronacher “Async is all the rage.” But before you go there, make sure you understand flow control and back pressure. “…back pressure is resistance that opposes the flow of data through a system. Back pressure sounds quite negative … but it's here to save your day.” If parts of your system are async, you have to make sure the entire flow throw the system doesn’t have overflow points. An example shown with reader/writer that is way hairier than you’d think it should be. “New Footguns: async/await is great but it encourages writing stuff that will behave catastrophically when overloaded.” “So for you developers of async libraries here is a new year's resolution for you: give back pressure and flow control the importance they deserve in documentation and API.” Michael #4: codetiming from Real Python via Doug Farrell A flexible, customizable timer for your Python code For a complete tutorial on how codetiming works, see Python Timer Functions: Three Ways to Monitor Your Code on Real Python. Time your code via A timer class A decorator A context manager Brian #5: Making Python Programs Blazingly Fast Martin Heinz Seemed like a good followup to the last topic Profiling with command line time python something.py python -m cProfile -s time something.py timing functions with wrapper Misses timeit, but see that also, https://docs.python.org/3.8/library/timeit.html How to make things faster: use built in types over custom types caching/memoization with lru_cache use local variables and local aliases when looping use functions… (kinda duh, but sure). don’t repeatedly access attributes in loops use f-strings over other formatting use generators. or at least experiment with them. the memory savings could result in speedup Michael #6: LocalStack via Graham Williamson and Jan 'oglop' Gazda A fully functional local AWS cloud stack. Develop and test your cloud & Serverless apps offline! LocalStack spins up the following core Cloud APIs on your local machine: S3, DynamoDB, Lambda, Elasticsearch see many more services paid one has more LocalStack builds on existing best-of-breed mocking/testing tools, most notably kinesalite/dynalite and moto. While these tools are awesome (!), they lack functionality for certain use cases. LocalStack combines the tools, makes them interoperable, and adds important missing functionality on top of them Has lots of config and knobs, but runs in docker so that helps Extras: Python Job Board Michael: Guido interviewed for JavaScript language! Microsoft: We're creating a new Rust-based programming language for secure coding New webcast: Python for the .NET developer Ace Python Interviews free course Joke: Types of software jobs.

Sponsored by DataDog: pythonbytes.fm/datadog Special guest: Aly Aly #1: Andrew Godwin - Just Add Await: Retrofitting Async into Django — DjangoCon 2019 Andrew is leading the implementation of asynchronous support for Django Overview of Async Landscape How synchronous and asynchronous code interact Async functions are different than sync functions which makes it hard to design APIs Difficulties in adding Async support to Django Django is a project that a lot of people are familiar with; it’s new async implementation also needs to feel familiar Plan was Implement async capabilities in three phases Phase 1: ASGI Support (Django 3.0) This phase lays the groundwork for future changes ORM is async-aware: using it from async code raises a SynchronousOnlyOperation exception Phase 2: Async Views, Async Handlers, and Async Middleware (Django 3.1) Add async capabilities for the core part of the request path There is a branch where things are mostly working, just need to fix a couple of tests Phase 3: Async ORM (Django 3.2 / 4.0) Largest, most difficult and most unbounded part of the project ORM queries can result in lots of database lookups; have to be careful here Async Project Wiki - project status, find out how to contribute Brian #2: gamesbyexample Al Sweigart “PythonStdioGames : A collection of games (with source code) to use for example programming lessons. Written in Python 3. Click on the src folder to view all of the programs.” I first learned programming by modifying games written by others and seeing what the different parts do when I change them. For me it was Lunar Lander on a TRS-80, and it took forever to type in the listing from the back of a magazine. But now, you can just clone a repo and play with existing files. Cool features: They're short, with a limit of 256 lines of code. They fit into a single source code file and have no installer. They only use the Python standard library. They only use stdio text; print() and input() in Python. They're well commented. They use as few programming concepts as possible. If classes, list comprehensions, recursion, aren't necessary for the program, then they are't used. Elegant and efficient code is worthless next to code that is easy to understand and readable. These programs are for education, not production. Standard best practices, like not using global variables, can be ignored to make it easier to understand. They do input validation and are bug free. All functions have docstrings. There’s also a todo list if people want to help out. Aly #3: Bulwark Open-source library that allows users to property test pandas DataFrames Goal is to make it easy for data analysts and data scientists to write tests Tests around data are different; they are not deterministic, they requires us to think about testing in a different way With property tests, we can check an object has a certain property Property tests for DataFrames includes validating the shape of the DataFrame, checking that a column is within a certain range, verifying a DataFrame has no NaNs, etc Bulwark allows you to implement property tests as checks. Each check Takes a DataFrame and optional arguments The check will make an assertion about a DataFrame property If the assertion passes, the check will return the original, unaltered DataFrame If the check fails, an AssertionError is raised and you have context around why it failed Bulwark also allows you to implement property checks as decorators This is useful if you design data pipelines as functions Each function take in input data, performs an action, returns output Add decorators validate properties of input DataFrame to pipeline functions Lots of builtin checks and decorators; easy to add your own Slides with example usage and tips: Property Testing with Pandas with Bulwark Brian #4: Poetry 1.0.0 Sebastien Eustace caution: not backwards compatible full change log Highlights: Poetry is getting serious. more ways to manage environments switch between python versions in a project with poetry env use /path/to/python or poetry env use python3.7 Imroved support for private indices (instead of just pypi) can specify index per dependency can specify a secondary index can specify a non-pypi index as default, avoiding pypi Env variable support to more easily work with poetry in a CI environment Improved add command to allow for constraints, paths, directories, etc for a dependency publishing allows api tokens marker specifiers on dependencies. Aly #5: Kubernetes for Full-Stack Developers With the rise of containers, Kubenetes has become the defacto platform for running and coordinating containerized applications across multiple machines With the rise of containers, Kubenetes is the defacto platform for running and coordinating applications across multiple machines This guide follows steps new users would take when learning how to deploy applications to Kubernetes: Learn Kubernetes core concepts Build modern 12 Factor web applications Get applications working inside of containers Deploy applications to Kubernetes Manage cluster operations New to containers? Check out my Introduction to Docker talk Brian #6: testmon: selects tests affected by changed files and methods On a previous episode (159) we mentioned pytest-picked and I incorrectly assumed it would run tests related to code that has changed, ‘cause it says “Run the tests related to the unstaged files or the current branch (according to Git)”. I was wrong, Michael was right. It runs the tests that are in modified test files. What I was thinking of is “testmon” which does what I was hoping for. “pytest-testmon is a pytest plugin which selects and executes only tests you need to run. It does this by collecting dependencies between tests and all executed code (internally using Coverage.py) and comparing the dependencies against changes. testmon updates its database on each test execution, so it works independently of version control.” If you had tried testmon before, like me, be aware that there have been significant changes in 1.0.0 Very cool to see continued effort on this project. Extras: Aly: Finding local Python User Groups PyCon.org Events Calendar Meetup.com search for Python PyTennessee 2019 on March 7 - 8. Tickets on sale now! I will be giving a talk on the Facade Design Pattern Brian: Next episode is planned to be a live recording during the Jan 7 Python PDX West meetup. There will also be 1-2 other talks. Joke: From Tyler Matteson Two coroutines walk into a bar. RuntimeError: 'bar' was never awaited. From Ben Sandofsky Q: How many developers on a message board does it take to screw in a light bulb? A: “Why are you trying to do that?”

Sponsored by DigitalOcean: pythonbytes.fm/digitalocean Special guest: Anthony Herbert Anthony #1: Larry Hastings - Solve Your Problem With Sloppy Python - PyCon 2018 Michael’s personal automation things that I do all the time stripe to sheets automation urlify tons of reporting wakeup - to get 100 on Lighthouse deploy (on my servers) creating import data for video courses measuring duration of audio files Michael #2: Introduction to ASGI: Emergence of an Async Python Web Ecosystem by Florimond Manca Python growth is not just data science Python web development is back with an async spin, and it's exciting. One of the main drivers of this endeavour is ASGI , the Asynchronous Standard Gateway Interface. A guided tour about what ASGI is and what it means for modern Python web development. Since 3.5 was released, the community has been literally async-ifying all the things. If you're curious, a lot of the resulting projects are now listed in aio-libs and awesome-asyncio . An overview of ASGI Why should I care? Interoperability is a strong selling point, there are many more advantages to using ASGI-based components for building Python web apps. Speed: the async nature of ASGI apps and servers make them really fast (for Python, at least) — we're talking about 60k-70k req/s (consider that Flask and Django only achieve 10-20k in a similar situation). Features: ASGI servers and frameworks gives you access to inherently concurrent features (WebSocket, Server-Sent Events, HTTP/2) that are impossible to implement using sync/WSGI. Stability: ASGI as a spec has been around for about 3 years now, and version 3.0 is considered very stable. Foundational parts of the ecosystem are stabilizing as a result. To get your hands dirty, try out any of the following projects: uvicorn: ASGI server. Starlette: ASGI framework. TypeSystem: data validation and form rendering Databases: async database library. orm: asynchronous ORM. HTTPX: async HTTP client w/ support for calling ASGI apps (useful as a test client). Anthony #3: Python Insights Michael #4: Assembly via Luiz Honda Assembly is a Pythonic Object-Oriented Web Framework built on Flask, that groups your routes by class Assembly is a pythonic object-oriented, mid stack, batteries included framework built on Flask, that adds structure to your Flask application, and group your routes by class. Assembly allows you to build web applications in much the same way you would build any other object-oriented Python program. Assembly helps you create small to enterprise level applications easily. Decisions made for you + features: github.com/mardix/assembly#decisions-made-for-you--features Examples, root URLs: # Extends to Assembly makes it a route automatically # By default, Index will be the root url class Index(Assembly): # index is the entry route # -> / def index(self): return "welcome to my site" # method name becomes the route # -> /hello/ def hello(self): return "I am a string" # undescore method name will be dasherize # -> /about-us/ def about_us(self): return "I am a string" Example of /blog. # The class name is part of the url prefix # This will become -> /blog class Blog(Assembly): # index will be the root # -> /blog/ def index(self): return [ { "title": "title 1", "content": "content" }, ... ] # with params. The order will be respected # -> /comments/1234/ # 1234 will be passed to the id def comments(self, id): return [ { comments... } ] Anthony #5: Building a Standalone GPS Logger with CircuitPython using @Adafruit and particle hardware Michael #6: 10 reasons python is good to learn Python is popular and good to learn because, in Michael’s words, it’s a full spectrum language. And the reasons are: Python Is Free and Open-Source Python Is Popular, Loved, and Wanted Python Has a Friendly and Devoted Community Python Has Elegant and Concise Syntax Python Is Multi-Platform Python Supports Multiple Programming Paradigms Python Offers Useful Built-In Libraries Python Has Many Third-Party Packages Python Is a General-Purpose Programming Language Python Plays Nice with Others Extras: Michael: I was just on .NET Rocks podcast talking about Python for the .NET Developer New Python for the .NET Developer 9-hour course New Python for Decision Makers course, 2.5 hours of exploring Python for your org. Hidden files in Finder: use shortcut cmd+shift+. Anthony: Pretty Printed YouTube channel Joke: The failed pickup line A girl is hanging out at a bar with her friends. Some guy comes up to her an says: “You are the ; to my line of code.” She responds, “Get outta here creep, I code in Python.”

Sponsored by DigitalOcean: pythonbytes.fm/digitalocean Brian #1: Type Hints for Busy Python Programmers Al Sweigart, @AlSweigart We’ve (Michael and myself, of course) convinced you that type hints might be a good thing to help reduce bugs or confusion or whatever. Now what? Al’s got your back with this no nonsense guide to get you going very quickly. Written as a conversation between a programmer and an type hint expert. Super short. Super helpful. typing and mypy are the modules you need. There are other tools, but let’s start there. Doesn’t affect run time, so you gotta run the tool. Gradually add, don’t have to do everything in one go. Covers the basics And then the “just after basics” stuff you’ll run into right away when you start, like: Allowing a type and None: Union[int, NoneType] Optional parameters Shout out to Callable, Sequence, Mapping, Iterable, available in the documentation when you are ready for them later Just really a great get started today guide. Michael #2: auto-py-to-exe A .py to .exe converter using a simple graphical interface built using Eel and PyInstaller in Python. Using the Application Select your script location (paste in or use a file explorer) Outline will become blue when file exists Select other options and add things like an icon or other files Click the big blue button at the bottom to convert Find your converted files in /output when complete Short 3 min video. Brian #3: How to document Python code with Sphinx Moshe Zadka, @moshezadka I’m intimidated by sphinx. Not sure why. But what I’ve really just wanted to do is to use it for this use of generating documentation of code based on the code and the docstrings. Many of the tutorials I’ve looked at before got me stuck somewhere along the way and I’ve given up. But this looks promising. Example module with docstring shown. Simple docs/index.rst, no previous knowledge of restructured text necessary. Specifically what extensions do I need: autodoc, napolean, and viewcode example docs/conf.py that’s really short setting up tox to generate the docs and the magic command like incantation necessary: sphinx-build -W -b html -d {envtmpdir}/doctrees . {envtmpdir}/html That’s it. (well, you may want to host the output somewhere, but I can figure that out. ) Super simple. Awesome Michael #4: Snek is a cross-platform PowerShell module for integrating with Python via Chad Miars Snek is a cross-platform PowerShell module for integrating with Python. It uses the Python for .NET library to load the Python runtime directly into PowerShell. Using the dynamic language runtime, it can then invoke Python scripts and modules and return the result directly to PowerShell as managed .NET objects. Kind of funky syntax, but that’s PowerShell for you ;) Even allows for external packages installed via pip Brian #5:How to use Pandas to access databases Irina Truong, @irinatruong You can use pandas and sqlalchemy to easily slurp tables right out of your db into memory. But don’t. pandas isn’t lazy and reads everything, even the stuff you don’t need. This article has tips on how to do it right. Recommendation to use the CLI for exploring, then shift to pandas and sqlalchemy. Tips (with examples, not shown here): limit the fields to just those you care about limit the number of records with limit or by selecting only rows where a particular field is a specific value, or something. Let the database do joins, even though you can do it in pandas Estimate memory usage with small queries and .memory_usage().sum(). Tips on reading chunks and converting small int types into small pandas types instead of 64 bit types. Michael #6: ijson — Iterative JSON parser with a standard Python iterator interface Iterative JSON parser with a standard Python iterator interface Most common usage is having ijson yield native Python objects out of a JSON stream located under a prefix. Here’s how to process all European cities: // from: { "earth": { "europe": [ ... ] } } stream each entry in europe as item: objects = ijson.items(f, 'earth.europe.item') cities = (o for o in objects if o['type'] == 'city') for city in cities: do_something_with(city) Extras: Michael: Python decision makers webcast on January 14th, 9:30am US Pacific Guido steps down from Steering Council via Vincent POULAILLEAU GitHub Archive Program, Preserving open source software for future generations, video Python 2.7 will be removed from Homebrew, via Allan Hansen Django 3.0 released Joke: Question: "What is the best prefix for global variables?" Answer: # via shinjitsu A web developer walks into a restaurant. He immediately leaves in disgust as the restaurant was laid out in tables. via shinjitsu

Sponsored by DigitalOcean: pythonbytes.fm/digitalocean Michael #1: Final type PEP 591 -- Adding a final qualifier to typing This PEP proposes a "final" qualifier to be added to the typing module---in the form of a final decorator and a Final type annotation---to serve three related purposes: Declaring that a method should not be overridden Declaring that a class should not be subclassed Declaring that a variable or attribute should not be reassigned Some situations where a final class or method may be useful include: A class wasn’t designed to be subclassed or a method wasn't designed to be overridden. Perhaps it would not work as expected, or be error-prone. Subclassing or overriding would make code harder to understand or maintain. For example, you may want to prevent unnecessarily tight coupling between base classes and subclasses. You want to retain the freedom to arbitrarily change the class implementation in the future, and these changes might break subclasses. # Example for a class: from typing import final @final class Base: ... class Derived(Base): # Error: Cannot inherit from final class "Base" ... And for a method: class Base: @final def foo(self) -> None: ... class Derived(Base): def foo(self) -> None: # Error: Cannot override final attribute "foo" # (previously declared in base class "Base") ... It seems to also mean const RATE: Final = 3000 class Base: DEFAULT_ID: Final = 0 RATE = 300 # Error: can't assign to final attribute Base.DEFAULT_ID = 1 # Error: can't override a final attribute Brian #2: flit 2 Michael #3: Pint via Andrew Simon Physical units and builtin unit conversion to everyday python numbers like floats. Receive inputs in different unit systems it can make life difficult to account for that in software. Pint handles the unit conversion automatically in a wide array of contexts – Can add 2 meters and 5 inches and get the correct result without any additional work. The integration with numpy and pandas are seamless, and it’s made my life so much simpler overall. Units and types of measurements Think you need this? How about the Mars Climate Orbiter The MCO MIB has determined that the root cause for the loss of the MCO spacecraft was the failure to use metric units in the coding of a ground software file, “Small Forces,” used in trajectory models. Specifically, thruster performance data in English units instead of metric units was used in the software application code titled SM_FORCES (small forces). Brian #4: 8 great pytest plugins Jeff Triplett Michael #5: 11 new web frameworks via LuisCarlos Contreras Sanic [flask like] - a web server and web framework that’s written to go fast. It allows the usage of the async / await syntax added in Python 3.5 Starlette [flask like] - A lightweight ASGI framework which is ideal for building high performance asyncio services, designed to be used either as a complete framework, or as an ASGI toolkit. Masonite - A developer centric Python web framework that strives for an actual batteries included developer tool with a lot of out of the box functionality. Craft CLI is the edge here. FastAPI - A modern, high-performance, web framework for building APIs with Python 3.6+ based on standard Python type hints. Responder - Based on Starlette, Responder’s primary concept is to bring the niceties that are brought forth from both Flask and Falcon and unify them into a single framework. Molten - A minimal, extensible, fast and productive framework for building HTTP APIs with Python. Molten can automatically validate requests according to predefined schemas. Japronto - A screaming-fast, scalable, asynchronous Python 3.5+ HTTP toolkit integrated with pipelining HTTP server based on uvloop and picohttpparser. Klein [flask like] - A micro-framework for developing production-ready web services with Python. It is ‘micro’ in that it has an incredibly small API similar to Bottle and Flask. Quart [flask like]- A Python ASGI web microframework. It is intended to provide the easiest way to use asyncio functionality in a web context, especially with existing Flask apps. BlackSheep - An asynchronous web framework to build event based, non-blocking Python web applications. It is inspired by Flask and ASP.NET Core. BlackSheep supports automatic binding of values for request handlers, by type annotation or by conventions. Cyclone - A web server framework that implements the Tornado API as a Twisted protocol. The idea is to bridge Tornado’s elegant and straightforward API to Twisted’s Event-Loop, enabling a vast number of supported protocols. Brian #6: Raise Better Exceptions in Python Extras Michael: Naming venvs --prompt Another new course coming soon: Python for decision makers and business leaders Some random interview over at Real Python: Python Community Interview With Brian Okken Joke via Daniel Pope What's a tractor's least favorite programming language? Rust.

This episode is sponsored by DigitalOcean - pythonbytes.fm/digitalocean Brian #1: Python already replaced Excel in banking “If you wanted to prove your mettle as an entry-level banker or trader it used to be the case that you had to know all about financial modeling in Excel. Not any more. These days it's all about Python, especially on the trading floor. "Python already replaced Excel," said Matthew Hampson, deputy chief digital officer at Nomura, speaking at last Friday's Quant Conference in London. "You can already walk across the trading floor and see people writing Python code...it will become much more common in the next three to four years." Michael #2: GitHub launches 'Security Lab' to help secure open source ecosystem At the GitHub Universe developer conference, GitHub announced the launch of a new community program called Security Lab GitHub says Security Lab founding members have found, reported, and helped fix more than 100 security flaws already. Other organizations, as well as individual security researchers, can also join. A bug bounty program with rewards of up to $3,000 is also available, to compensate bug hunters for the time they put into searching for vulnerabilities in open source projects. Bug reports must contain a CodeQL query. CodeQL is a new open source tool that GitHub released today; a semantic code analysis engine that was designed to find different versions of the same vulnerability across vasts swaths of code. Starting today automated security updates are generally available and have been rolled out to every active repository with security alerts enabled. Once a security flaw is fixed, the project owner can publish the security, and GitHub will warn all upstream project owners who are using vulnerable versions of the original maintainer's code. But before publishing a security advisory, project owners can also request and receive a CVE number for their project's vulnerability directly from GitHub. And last, but not least, GitHub also updated Token Scanning, its in-house service that can scan users' projects for API keys and tokens that have been accidentally left inside their source code. Brian #3: pybit.es now has some test challenges Uses pytest, coverage.py, and mutpy (for mutation testing) Most other challenges have tests that validate the code you write. New challenges (3 so far) have you write the tests for existing code. Tests are evaluated with both coverage.py and mutpy another mutation testing tool is mutmut, written about earlier this year by Ned Badtchelder. Michael #4: pyhttptest - a command-line tool for HTTP tests over RESTful APIs via Florian Dahlitz A command-line tool for HTTP tests over RESTful APIs Tired of writing test scripts against your RESTFul APIs anytime? Describe an HTTP Requests test cases in a simplest and widely used format JSON within a file. Run one command and gain a summary report. Example { "name": "TEST: List all users", "verb": "GET", "endpoint": "users", "host": "https://github.com", "headers": { "Accept-Language": "en-US" }, "query_string": { "limit": 5 } } Brian #5: xarray suggested by Guido Imperiale xarray is a mature library that builds on top of numpy, pandas and dask to offer arrays that are n-dimensional (numpy and dask do it, but pandas doesn't) self-described and indexed (pandas does it, but numpy and dask don't) out-of-memory, multi-threaded, and cloud-distributed (dask does it, but numpy and pandas don't). Additionally, xarray can semi-transparently swap numpy with other backends, such as sparse , while retaining the same API. Michael #6: Animated SVG Terminals Florian Dahlitz termtosvg is a Unix terminal recorder written in Python that renders your command line sessions as standalone SVG animations. Features: Produce lightweight and clean looking animations or still frames embeddable on a project page Custom color themes, terminal UI and animation controls via user-defined SVG templates Rendering of recordings in asciicast format made with asciinema Examples: nbedos.github.io/termtosvg/pages/examples.html Extras pytest 5.3.0 released, please read changelog if you use pytest, especially if you use it with CI and depend on --junitxml, as they have changed the format to a newer version. Michael: PyCon registration is open (via Jacqueline Wilson) Facebook: Microsoft's Visual Studio Code is now our default development platform Black friday at Talk Python Training! New course coming soon: Python for the .NET developer Jokes What do you get when you put root beer in a square glass? Beer. Q: What do you call optimistic front-end developers? A: Stack half-full developers. Also, I was going to tell a version control joke, but they are only funny if you git them.

This episode is sponsored by DigitalOcean: pythonbytes.fm/digitalocean Michael #1: pydantic via Colin Sullivan Data validation and settings management using python type annotations. (We covered Cerberus, this is similar) pydantic enforces type hints at runtime, and provides user friendly errors when data is invalid. class User(pydantic.BaseModel): id: int name = 'John Doe' signup_ts: datetime = None friends: List[int] = [] external_data = { 'id': '123', 'signup_ts': '2019-06-01 12:22', 'friends': [1, 2, '3'] } user = User(**external_data) id is of type int; the annotation-only declaration tells pydantic that this field is required. Strings, bytes or floats will be coerced to ints if possible; otherwise an exception will be raised. name is inferred as a string from the provided default; because it has a default, it is not required. signup_ts is a datetime field which is not required (and takes the value None if it's not supplied). Why use it? There's no new schema definition micro-language to learn. In benchmarks pydantic is faster than all other tested libraries. Use of recursive pydantic models, typing's standard types (e.g. List, Tuple, Dict etc.) and validators allow complex data schemas to be clearly and easily defined, validated, and parsed. As well as BaseModel, pydantic provides a [dataclass](https://pydantic-docs.helpmanual.io/usage/dataclasses/) decorator which creates (almost) vanilla python dataclasses with input data parsing and validation. Brian #2: Coverage.py 5.0 beta 1 adds context support Please try out the beta, even without trying contexts, as it helps Ned Batchelder to make sure it’s as backwards compatible as possible while still adding this super cool functionality. Coverage 5.0 beta 1 announcement The changes. Measurement contexts in depth. Trying out contexts with pytest and pytest-cov: (venv) $ pip install coverage==5.0b1 (venv) $ pip install pytest-cov (venv) $ pytest --cov=foo --cov-context=test test_foo.py (venv) $ coverage html --show-contexts (venv) $ open htmlcov/index.html results in coverage report that has little dropdowns on the right for lines that are covered, and what context they were covered. For the example above, with pytest-cov, it shows what test caused each line to be hit. Contexts can do way more than this. One example, split up different levels of tests, to see which lines are only hit by unit tests, indicating missing higher level tests, or the opposite. The stored db could also possibly be mined to see how much overlap there is between tests, and maybe help with higher level tools to predict the harm or benefit from removing some tests. I’m excited about the future, with contexts in place. Even if you ignore contexts, please go try out the beta ASAP to make sure your old use model still works. Michael #3: PSF is seeking developers for paid contract improving pip via Brian Rutledge The Python Software Foundation Packaging Working Group is receiving funding to work on the design, implementation, and rollout of pip's next-generation dependency resolver. This project aims to complete the design, implementation, and rollout of pip's next-generation dependency resolver. Lower the barriers to installing Python software, empowering users to get a version of a package that works. It will also lower the barriers to distributing Python software, empowering developers to make their work available in an easily reusable form. Because of the size of the project, funding has been allocated to secure two contractors, a senior developer and an intermediate developer, to work on development, testing and building test infrastructure, code review, bug triage, and assisting in the rollout of necessary features. Total pay: Stage 1: $116,375, Stage 2: $103,700 Brian #4: dovpanda - Directions OVer PANDAs Dean Langsam “Directions are hints and tips for using pandas in an analysis environment. dovpanda is an overlay for working with pandas in an analysis environment. "If you think your task is common enough, it probably is, and Pandas probably has a built-in solution. dovpanda is an overlay module that tries to understand what you are trying to do with your data, and help you find easier ways to write your code.” “The main usage of dovpanda is its hints mechanism, which is very easy and works out-of-the-box. Just import it after you import pandas, whether inside a notebook or in a console.” It’s like training wheels for pandas to help you get the most out of pandas and learn while you are doing your work. Very cool. Michael #5: removestar via PyCoders newsletter Tool to automatically replace 'import *' in Python files with explicit imports Report only mode and modify in place mode. Brian #6: pytest-quarantine : Save the list of failing tests, so that they can be automatically marked as expected failures on future test runs. Brian Rutlage Really nice email from Brian: >"Hi Brian! We've met a couple times at PyCon in Cleveland. Thanks for your podcasts, and your book. I've gone from being a complete pytest newbie, to helping my company adopt it, to writing a plugin. The plugin was something I developed at work, and they let me open-source it. I wanted to share it with you as a way of saying "thank you", and because you seem to be a bit of connoisseur of pytest plugins. ;)" Here it is: https://github.com/EnergySage/pytest-quarantine/” pytest has a cool feature called xfail, to allow you to mark tests you know fail. pytest-quarantine allows you to run your suite and generate a file of all failures, then use that to mark the xfails. Then you or your team can chip away at these failures until you get rid of them. But in the meantime, your suite can still be useful for finding new failures. And, the use of an external file to mark failures makes it so you don’t have to edit your test files to mark the tests that are xfail. Extras: MK: Our infrastructure is fully carbon neutral! Joke: A cop pulls Dr. Heisenberg over for speeding. The officer asks, "Do you know how fast you were going?" Heisenberg pauses for a moment, then answers, "No, but I know where I am.” [1] See Uncertainty principle, also called Heisenberg uncertainty principle or indeterminacy principle, statement, articulated (1927) by the German physicist Werner Heisenberg, that the position and the velocity of an object cannot both be measured exactly, at the same time, even in theory.

Sponsored by DigitalOcean: pythonbytes.fm/digitalocean Special guests: Dan Bader Cecil Philip Dan #1: Why You Should Use python -m pip https://snarky.ca/why-you-should-use-python-m-pip/ Cecil #2: Visual Studio Online: Web-Based IDE & Collaborative Code Editor https://visualstudio.microsoft.com/services/visual-studio-online/ Michael #3: Python Adopts a 12-month Release Cycle The long discussion on changing the Python project's release cadence has come to a conclusion: the project will now be releasing new versions on an annual basis. Described in PEP 602 The steering council thinks that having a consistent schedule every year when we hit beta, RC, and final it will help the community: Know when to start testing the beta to provide feedback Known when the expect the RC so the community can prepare their projects for the final release Know when the final release will occur to coordinate their own releases (if necessary) when the final release of Python occurs Allow core developers to more easily plan their work to make sure work lands in the release they are targeting Make sure that core developers and the community have a shorter amount of time to wait for new features to be released Dan #4: Black 19.10b0 Released — stable release coming soon https://twitter.com/llanga/status/1188968251918819329 “Black Friday” release date? https://twitter.com/llanga/status/1189145837991014402 Playground: https://black.now.sh/ Cecil 5: Navigating code on GitHub Example: https://github.com/talkpython/100daysofcode-with-python-course/blob/master/days/10-12-pytest/guess/guess.py Michael #6: lolcommits: selfies for software developers. lolcommits takes a snapshot with your webcam every time you git commit code, and archives a lolcat style image with it. git blame has never been so much fun. Infinite uses: Animate your progress through a project and watch as you age. See what you looked like when you broke the build. Keep a joint lolrepository for your entire company. Plugins: Lolcommits allows a growing list of plugins to perform additional work on your lolcommit image after capturing. Animate: Configure lolcommits to generate an animated GIF with each commit for extra lulz! Extras: Dan: Article & Course on Python 3.8 https://realpython.com/python38-new-features/ https://realpython.com/courses/cool-new-features-python-38/ Cecil: Twitch learning Python channel Michael: New Anvil course, free one - https://talkpython.fm/anvil PSF yearly survey is out: https://twitter.com/thepsf/status/1190004772704784385 Joke: LOLCODE

Sponsored by Datadog: pythonbytes.fm/datadog Michael #1: Guido retires Guido van Rossum has left DropBox and retired (post) Let’s all take a moment to say thank you. I wonder what will come next in terms of creative projects Some comments from community members (see Twitter thread) Brian #2: SeleniumBase Automated UI Testing with Selenium WebDriver and pytest. Very expressive and intuitive automation library built on top of Selenium WebDriver. method overview very readable (this is a workflow test, but still, quite readable): from seleniumbase import BaseCase class MyTestClass(BaseCase): def test_basic(self): self.open("https://xkcd.com/353/") self.assert_title("xkcd: Python") self.assert_element('img[alt="Python"]') self.click('a[rel="license"]') self.assert_text("free to copy and reuse") self.go_back() self.click("link=About") self.assert_text("xkcd.com", "h2") self.open("https://store.xkcd.com/collections/everything") self.update_text("input.search-input", "xkcd book\n") self.assert_exact_text("xkcd: volume 0", "h3") includes plugins for including screenshots in test results. supports major CI systems some cool features that I didn’t expect user onboarding demos assisted QA (partially automated with manual questions) support for selenium grid logs of command line options, including headless Michael #3: Reimplementing a Solaris command in Python gained 17x performance improvement from C Postmortem by Darren Moffat Is Python slow? A result of fixing a memory allocation issue in the /usr/bin/listusers command Decided to investigate if this ancient C code could be improved by conversion to Python. The C code was largely untouched since 1988 and was around 800 lines long, it was written in an era when the number of users was fairly small and probably existed in the local files /etc/passwd or a smallish NIS server. It turns out that the algorithm to implement the listusers is basically some simple set manipulation. Rewrite of listusers in Python 3 turned out to be roughly a 10th of the number of lines of code But Python would be slower right ? Turns out it isn't and in fact for some of my datasets (that had over 100,000 users in them) it was 17 times faster. A few of the comments asked about the availability of the Python version. The listusers command in Oracle Solaris 11.4 SRU 9 and higher. Since we ship the /usr/bin/listusers file as the Python code you can see it by just viewing the file in an editor. Note though that is not open source and is covered by the Oracle Solaris licenses. Brian #4: 20 useful Python tips and tricks you should know I admit it, I’m capable of getting link-baited by the occasional listicle. Some great stuff, especially for people coming from other languages. Chained assignment: x = y = z = 2 Chained comparison: 2 < x <= 8 2 < x > 4 0 < x < 4 < y < 16 Multiple assignment: x, y, z = 2, 4, 8 More Advanced Multiple Assignment: x, *y, z = 2, 4, 8, 16 I’ve been using the * for unpacking a lot, especially with *_ Merge dictionaries: z = {**x, **y} Join strings: '_'.join(['u', 'v', 'w']) using list(set(something)) to remove duplicates. aggregate elements. using zip to element-wise combine two or more iterables. >>> x = [1, 2, 3] >>> y = ['a', 'b', 'c'] >>> zip(x, y) [(1, 'a'), (2, 'b'), (3, 'c')] and then some other weird stuff that I don’t find that useful. Michael #5: Complexity Waterfall via Ahrem Ahreff Heavy use of wemake-python-styleguide Code smells! Use your refactoring tools and write tests. Automation enable an opportunity of “Continuous Refactoring” and “Architecture on Demand” development styles. Brian #6: Plynth Plynth is a GUI framework for building cross-platform desktop applications with HTML, CSS and Python. Plynth has integrated the standard CPython implementation with Chromium's rendering engine. You can run your python scripts seamlessly with HTML/CSS instead of using Javascript with modules from pip Plynth uses Chromium/Electron for its rendering. With Plynth, every Javascript library turns into a Python module. Not open source. But free for individuals, including commercial use and education. A bunch of tutorial videos that are not difficult to follow, and not long, but… not really obvious code either. Python 3.6 and 3.7 development kits available Extras: Michael: Google Is Uncovering Hundreds Of Race Conditions Within The Linux Kernel Joke: Q: What's a web developer's favorite tea? A: URL gray via Aideen Barry

Sponsored by Datadog: pythonbytes.fm/datadog Special guest: Bob Belderbos Brian #1: Lesser Known Coding Fonts Interesting examination of some coding fonts. Link to a great talk called Cracking the Code, by Jonathan David Ross, about coding fonts and Input. I’m trying out Input Mono right now, and quite like it. Fira code: https://github.com/tonsky/FiraCode Bob #2: Django Admin Handbook As a Django developer knowing the admin is pretty important. Free ebook of 40 or so pages, you can consume it in one evening. There are a lot of good tricks, 3 I liked: How to optimize queries in Django admin (override get_queryset) How to export CSV from Django admin (useful for data analysis in Jupyter for example) How to override save behaviour for Django admin (used this to notify users upon publishing a new exercise on our platform) Some more cool ebooks on that site, e.g. Tweetable #Python. Michael #3: Your Guide to the CPython Source Code Let’s talk about exploring the CPython code You’ll want to get the code: git clone https://github.com/python/cpython Compile the code (Anthony gives lots of steps for macOS, Windows, and Linux) Structure: cpython/ │ ├── Doc ← Source for the documentation ├── Grammar ← The computer-readable language definition ├── Include ← The C header files ├── Lib ← Standard library modules written in Python ├── Mac ← macOS support files ├── Misc ← Miscellaneous files ├── Modules ← Standard Library Modules written in C ├── Objects ← Core types and the object model ├── Parser ← The Python parser source code ├── PC ← Windows build support files ├── PCbuild ← Windows build support files for older Windows versions ├── Programs ← Source code for the python executable and other binaries ├── Python ← The CPython interpreter source code └── Tools ← Standalone tools useful for building or extending Python Some cool “hidden” goodies. For example, check out Lib/concurrent/futures/process.py, it comes with a cool ascii diagram of the process. Lots more covered, that we don’t have time for The Python Interpreter Process The CPython Compiler and Execution Loop Objects in CPython The CPython Standard Library Installing a custom version Brian #4: Six Django template tags not often used in tutorials Here’s a few: {% empty %}, for use in for loops when the array is empty {% lorem \[count\] [method] [random] %} for automatically filling with Lorem Ipsum text. {% verbatim %} … {% endverbatim %}, stop the rendering engine from trying to parse it and replace stuff. https://hipsum.co/ Bob #5: Beautiful code snippets with Carbon Beautiful images, great for teaching Python / programming. Used by a lot of developer, nice example I spotted today. Supports typing and drag and drop, just generated this link by dropping a test module onto the canvas! Great to expand Twitter char limit (we use it to generate Python Tip images). Follow the project here, seems they now integrate with Github. Michael #6: Researchers find bug in Python script may have affected hundreds of studies More info via Mike Driscoll at Thousands of Scientific Papers May be Invalid Due to Misunderstanding Python In a paper published October 8, researchers at the University of Hawaii found that a programming error in a set of Python scripts commonly used for computational analysis of chemistry data returned varying results based on which operating system they were run on. Scientists did not understand that Python’s glob.glob() does not return sorted results Throwing doubt on the results of more than 150 published chemistry studies. the researcher were trying to analyze results from an experiment involving cyanobacteria discovered significant variations in results run against the same nuclear magnetic resonance spectroscopy (NMR) data. The scripts, called the "Willoughby-Hoye" scripts after their creators, were found to return correct results on macOS Mavericks and Windows 10. But on macOS Mojave and Ubuntu, the results were off by nearly a full percent. The module depends on the operating system for the order in which the files are returned. And the results of the scripts' calculations are affected by the order in which the files are processed. The fix: A simple list.sort()! Williams said he hopes the paper will get scientists to pay more attention to the computational side of experiments in the future. Extras: Nov 5 is the next Python PDX West Using Big Tech Tools Working on: PyBites platform: added flake8/ black code formatting, UI enhancements. Michael: Bezos DDoS'd: Amazon Web Services' DNS systems knackered by hours-long cyber-attack PyPI Just Crossed the 200,000 Packages Threshold! (via RP) XKCD Date — via André Jaenisch, Enter https://explainxkcd.com/wiki/index.php/1425:_Tasks and learn, that it was published on 24th Sep 2014. Joke: Q: What did the Network Administrator say when they caught a nasty virus? A: It hurts when IP

Sponsored by DigitalOcean: pythonbytes.fm/digitalocean Michael #1: Building a Python C Extension Module Tutorial, learn to use the Python API to write Python C extension modules. And Invoke C functions from within Python Pass arguments from Python to C and parse them accordingly Raise exceptions from C code and create custom Python exceptions in C Define global constants in C and make them accessible in Python Test, package, and distribute your Python C extension module Extending Your Python Program there may be other lesser-used system calls that are only accessible through C Steps: Writing a Python Interface in C Figure out the arguments (e.g. int fputs(const char *, FILE *) ) Implement in C: #include Python.h static PyObject *method_fputs(PyObject *self, PyObject *args) { char *str, *filename = NULL; int bytes_copied = -1; /* Parse arguments */ if(!PyArg_ParseTuple(args, "ss", &str, &filename)) { return NULL; } FILE *fp = fopen(filename, "w"); bytes_copied = fputs(str, fp); fclose(fp); return PyLong_FromLong(bytes_copied); } In line 2, you declare the argument types you wish to receive from your Python code line 6, then you’ll see that PyArg_ParseTuple() copies into the char*’s Write regular C code (fopen, fputs) Return: PyLong_FromLong() creates a PyLongObject, which represents an integer object in Python. a few extra functions that are necessary write definitions of your module and the methods it contains Before you can import your new module, you first need to build it. You can do this by using the Python package distutils. Brian #2: What’s New in Python 3.8 - docs.python.org We’ve already talked about the big hitters: assignment expressions, (the walrus operator) positional only parameters, (the / in the param list) f-strings support = for self-documenting expressions and debugging There are a few more goodies I wanted to quickly mention: More async: python -m asyncio launches a native async REPL More helpful warnings and messages when using is and is not to compare strings and integers and other types intended to be compared with == and != Missing the comma in stuff like [(1,2) (3,4)]. Happens all the time with parametrized testing you can do iterable unpacking in a yield or return statement x = (1, 2, 3) a, *b = x return a, *b <- this used to be a syntax error you had to do return (a, *b) New module importlib.metadata lets you access things like version numbers or dependent library required version numbers, and cool stuff like that. quite a few more goodies. I run through all my favorites on testandcode.com/91 Michael #3: UK National Cyber Security Centre (NCSC) is warning developers of the risks of sticking with Python 2.7, particularly for library writers NCSC likens companies continuing to use Python 2 past its EOL to tempting another WannaCry or Equifax incident. Equifax details: a vulnerability, dubbed CVE-2017-5638, was discovered in Apache Struts, an open source development framework for creating enterprise Java applications that Equifax, along with thousands of other websites, uses… Quote: "If you're still using 2.x, it's time to port your code to Python 3," the NCSC said. "If you continue to use unsupported modules, you are risking the security of your organisation and data, as vulnerabilities will sooner or later appear which nobody is fixing." Moreover: "If you maintain a library that other developers depend on, you may be preventing them from updating to 3," the agency added. "By holding other developers back, you are indirectly and likely unintentionally increasing the security risks of others.” "If migrating your code base to Python 3 is not possible, another option is to pay a commercial company to support Python 2 for you," the NCSC said. NCSC: If you don't migrate, you should expect security incidents Python's popularity makes updating code imperative: The reason the NCSC is warning companies about Python 2's impending EOL is because of the language's success. Brian #4: Pythonic News Sebastian A. Steins “A Hacker News lookalike written in Python/Django” “ powering https://news.python.sc" Cool that it’s open source, and on github Was submitted to us by Sebastian, and a few others too, so there is excitement. It’s like 6 days old and has 153 stars on github, 4 contributors, 18 forks. Fun. Michael #5: Deep Learning Workstations, Servers, Laptops, and GPU Cloud GPU-accelerated with TensorFlow, PyTorch, Keras, and more pre-installed. Just plug in and start training. Save up to 90% by moving off your current cloud and choosing Lambda. They offer: TensorBook: GPU Laptop for $2,934 Lambda Quad: 4x GPU Workstation for $21,108 (yikes!) All in: Lambda Hyperplane: 8x Tesla V100 Server, starting at $114,274 But compare to: AWS EC2: p3.8xlarge @ $12.24 per Hour => $8,935 / month Brian #6: Auto formatters for Python A comparison of autopep8, yapf, and black Auto formatters are super helpful for teams. They shut down the unproductive arguments over style and make code reviews way more pleasant. People can focus on content, not being the style police. We love black. But it might be a bit over the top for some people. Here are a couple of other alternatives. autopep8 - mostly focuses on PEP8 “autopep8 automatically formats Python code to conform to the PEP 8 style guide. It uses the pycodestyle utility to determine what parts of the code needs to be formatted. autopep8 is capable of fixing most of the formatting issues that can be reported by pycodestyle.” black - does more doesn’t have many options, but you can alter line length, can turn of string quote normalization, and you can limit or focus the files it sees. does a cool “check that the reformatted code still produces a valid AST that is equivalent to the original.” but you can turn that off with --fast yapf - way more customizable. Great if you want to auto format to a custom style guide. “The ultimate goal is that the code YAPF produces is as good as the code that a programmer would write if they were following the style guide. It takes away some of the drudgery of maintaining your code.” Article is cool in that it shows some sample code and how it’s changed by the different formatters. Extras: Michael: New courses coming Financial Aid Launches for PyCon US 2020! Joke: American Py Song From Eric Nelson: Math joke. “i is as complex as it gets. jk.”

Sponsored by DigitalOcean: pythonbytes.fm/digitalocean Michael #1: JPMorgan’s Athena Has 35 Million Lines of Python 2 Code, and Won’t Be Updated to Python 3 in Time With 35 million lines of Python code, the Athena trading platform is at the core of JPMorgan's business operations. A late start to migrating to Python 3 could create a security risk. Athena platform is used internally at JPMorgan for pricing, trading, risk management, and analytics, with tools for data science and machine learning. This extensive feature set utilizes over 150,000 Python modules, over 500 open source packages, and 35 million lines of Python code contributed by over 1,500 developers, according to data presented by Misha Tselman, executive director at J.P. Morgan Chase in a talk at PyData 2017. And JPMorgan is going to miss the deadline Roadmap puts "most strategic components" compatible with Python 3 by the end of Q1 2020 JPMorgan uses Continuous Delivery, with 10,000 to 15,000 production changes per week "If you maintain a library that other developers depend on," the post states, "you may be preventing them from updating to 3. By holding other developers back, you are indirectly and likely unintentionally increasing the security risks of others," adding that developers who do not publish code publicly should "consider your colleagues who may also be using your code internally." Brian #2: organize suggested by Ariel Barkan a Python based file management automation tool configuration is via a yml file command line tool to organize your file system examples: move all of your screenshots off of your desktop into a screenshots folder move old incomplete downloads into trash remove empty files from certain folders organize receipts and invoices into date based folders Michael #3: PEP 589 – TypedDict: Type Hints for Dictionaries With a Fixed Set of Keys Author: Jukka Lehtosalo Sponsor: Guido van Rossum Status: Accepted Version: 3.8 PEP 484 defines the type Dict[K, V] for uniform dictionaries, where each value has the same type, and arbitrary key values are supported. It doesn't properly support the common pattern where the type of a dictionary value depends on the string value of the key. Core idea: Consider creating a type to validate an arbitrary JSON document with a fixed schema Proposed syntax: from typing import TypedDict class Movie(TypedDict): name: str year: int movie: Movie = {'name': 'Blade Runner', 'year': 1982} Operations on movie can be checked by a static type checker movie['director'] = 'Ridley Scott' # Error: invalid key 'director' movie['year'] = '1982' # Error: invalid value type ("int" expected) Brian #4: gazpacho gazpacho is a web scraping library “It replaces requests and BeautifulSoup for most projects. “ “gazpacho is small, simple, fast, and consistent.” example of using gazpacho to scrape hockey data for fantasy sports. simple interface, short scripts, really beginner friendly retrieve with get, parse with Soup. I don’t think it will completely replace the other tools, but for simple get/parse/find operations, it may make for slimmer code. Note, I needed to update certificates to get this to work. see this. Michael #5: How pip install Works via PyDist What happens when you run pip install [somepackage]? First pip needs to decide which distribution of the package to install. This is more complex for Python than many other languages There are 7 different kinds of distributions, but the most common these days are source distributions and binary wheels. A binary wheel is a more complex archive format, which can contain compiled C extension code. Compiling, say, numpy from source takes a long time (~4 minutes on my desktop), and it is hard for package authors to ensure that their source code will compile on other people's machines. Most packages with C extensions will build multiple wheel distributions, and pip needs to decide which if any are suitable for your computer. To find the distributions available, pip requests https://pypi.org/simple/[somepackage], which is a simple HTML page full of links, where the text of the link is the filename of the distribution. To select a distribution, pip first determines which distributions are compatible with your system and implementation of python. binary wheels, it parses the filenames according to PEP 425, extracting the python implementation, application binary interface, and platform. All source distributions are assumed to be compatible, at least at this step in the process Once pip has a list of compatible distributions, it sorts them by version, chooses the most recent version, and then chooses the "best" distribution for that version It prefers binary wheels if there are any Determining the dependencies for this distribution is not simple either. For binary wheels, the dependencies are listed in a file called METADATA. But for source distributions the dependencies are effectively whatever gets installed when you execute their setup.py script with the install command. What happens though if one of the distributions pip finds violates the requirements of another? It ignores the requirement and installs idna anyway! Next pip has to actually build and install the package. it needs to determine which library directory to install the package in—the system's, the user's, or a virtualenvs? Controlled by sys.prefix, which in turn is controlled by pip's executable path and the PYTHONPATH and PYTHONHOME environment variables. Finally, it moves the wheel files into the appropriate library directory, and compiles the python source files into bytecode for faster execution. Now your package is installed! Brian #6: daily pandas tricks Kevin Markham is sending out one pandas tip or trick per day via twitter. It’s been fun to watch and learn new bits. The link is a sampling of a bunch of them. Here’s just one example: Need to rename all of your columns in the same way? Use a string method: Replace spaces with _: df.columns = df.columns.str.replace(' ', '_') Make lowercase & remove trailing whitespace: df.columns = df.columns.str.lower().str.rstrip() Extras Michael: Switched to Adobe Audition Azure Databricks drops Python 2 Better Jupyter in VS Code macOS Catalina (so far so good) Jokes: via Sarcastic Pharmacist Hard to distinguish hard from easy in programming

Sponsored by DigitalOcean: pythonbytes.fm/digitalocean Brian #1: Python alternative to Docker Matt Layman Using Shiv, from LinkedIn Mentioned briefly in episode 114 Shiv uses zipapp, PEP 441. Execute code directly from a zip file. App code and dependencies can be bundled together. “Having one artifact eliminates the possibility of a bad interaction getting to your production system.” article includes an example of all the steps for packaging a Django app with Gunicorn. includes talking about deployment. Matt includes shoutouts to: Platform as a Service providers Manual steps to do it all. Docker Compares the process against Docker and discusses when to choose one over the other. Also an interesting read: Docker is in deep trouble Michael #2: How to support open-source software and stay sane via Jason Thomas written by Anna Nowogrodzki Releasing lab-built open-source software often involves a mountain of unforeseen work for the developers. Article opens: “On 10 April, astrophysicists announced that they had captured the first ever image of a black hole. This was exhilarating news, but none of the giddy headlines mentioned that the image would have been impossible without open-source software.” The image was created using Matplotlib, a Python library for graphing data, as well as other components of the open-source Python ecosystem. Just five days later, the US National Science Foundation (NSF) rejected a grant proposal to support that ecosystem, saying that the software lacked sufficient impact. Open-source software is widely acknowledged as crucially important in science, yet it is funded non-sustainably. “It’s sort of the difference between having insurance and having a GoFundMe when their grandma goes to the hospital,” says Anne Carpenter Challenges Scientists writing open-source software often lack formal training in software engineering. Yet poorly maintained software can waste time and effort, and hinder reproducibility. If your research group is planning to release open-source software, you can prepare for the support work Obsolescence isn’t bad, she adds: knowing when to stop supporting software is an important skill. However long your software will be used for, good software-engineering practices and documentation are essential. These include continuous integration systems (such as TravisCI), version control (Git) and unit testing. To facilitate maintenance, Varoquaux recommends focusing on code readability over peak performance. Brian #3: The Hippocratic License Coraline Ada Ehmke Interesting idea to derive from MIT, but add restrictions. This license adds these restrictions: “The software may not be used by individuals, corporations, governments, or other groups for systems or activities that actively and knowingly endanger, harm, or otherwise threaten the physical, mental, economic, or general well-being of individuals or groups in violation of the United Nations Universal Declaration of Human Rights” I could see others with different restrictions, or this but more. Michael #4: MATLAB vs Python: Why and How to Make the Switch MATLAB® is widely known as a high-quality environment for any work that involves arrays, matrices, or linear algebra. I personally used it for wavelet-decomposition of real time eye measurements during cognitively intensive human workloads… That toolbox costs $2000 per user. Difference in philosophy: Closed, paid vs. open source. Since Python is available at no cost, a much broader audience can use the code you develop Also, there is GNU Octave is a free and open-source clone of MATLAB apparently Brian #5: PyperCard - Easy GUIs for All Nicholas Tollervey Came up on episode 143 Also, episode 89 of Test & Code Really easy to quickly set up a GUI specified by a list of “Card” objects. (different from cards project) Simple examples are choose your own adventure type applications, where one button takes you to another card, and another button, a different card. However, the “next card” could be a Python function that can do anything, as long as it returns a string with the name of the next card. Lots of potential here, especially with input boxes, images, sound, and more. Super fun, but also might have business use. Michae #6: pynode Article: Bridging Node.js and Python with PyNode to Predict Home Prices Call python code from node.js Define a Python method In node: require pynode: const pynode = require('@fridgerator/pynode') Start an interpreter: pynode.startInterpreter() Call the function pynode.call('add', 1, 2, (err, result) => { if (err) return console.log('error : ', err) result === 3 // true }) Jokes The "Works on My Machine" Certification Program, get certified!

Sponsored by Datadog: pythonbytes.fm/datadog Michael #1: How to Stand Out in a Python Coding Interview Real Python, by James Timmins Are tech interviews broken? Well at least we can try to succeed at them anyway You’ve made it past the phone call with the recruiter, and now it’s time to show that you know how to solve problems with actual code… Interviews aren’t just about solving problems: they’re also about showing that you can write clean production code. This means that you have a deep knowledge of Python’s built-in functionality and libraries. Things to learn Use enumerate() to iterate over both indices and values Debug problematic code with breakpoint() Format strings effectively with f-strings Sort lists with custom arguments Use generators instead of list comprehensions to conserve memory Define default values when looking up dictionary keys Count hashable objects with the collections.Counter class Use the standard library to get lists of permutations and combinations Brian #2: The Python Software Foundation has updated its Code of Conduct There’s now one code of conduct for PSF and PyCon US and other spaces sponsored by the PSF This includes some regional conferences, such as PyCascades, and some meetup groups, (ears perk up) The docs Code of Conduct Enforcement Guidelines Reporting Guidelines Do we need to care? all of us, yes. If there weren’t problems, we wouldn’t need these. attendees, yes. Know before you go. organizers, yes. Better to think about it ahead of time and have a plan than have to make up a strategy during an event if something happens. me, in particular, and Michael. Ugh. yes. our first meetup is next month. I’d like to be in line with the rest of Python. So, yep, we are going to have to talk about this and put something in place. Michael #3: The Interview Study Guide For Software Engineers A checklist on my last round of interviews that covers many of the popular topics. Warm Up With The Classics Fizz Buzz 560. Subarray Sum Equals K Arrays: Left Rotation Strings: Making Anagrams Nth Fibonacci Many many videos on interview topics and ideas Data Structures Algorithms Big O Notation Dynamic Programming String Manipulation System Design Operating Systems Threads Object Oriented Design Patterns SQL Fun conversation in the comments Brian #4: re-assert : “show where your regex match assertion failed” Anthony Sotille “re-assert provides a helper class to make assertions of regexes simpler.” The Matches objects allows for useful pytest assertion messages In order to get my head around it, I looked at the test code: https://raw.githubusercontent.com/asottile/re-assert/master/tests/re_assert_test.py and modified it to remove all of the with pytest.raises(AssertionError)… to actually get to see the errors and how to use it. def test_match_old(): > assert re.match('foo', 'fob') E AssertionError: assert None E + where None = [HTML_REMOVED]('foo', 'fob') E + where [HTML_REMOVED] = re.match test_re.py:8: AssertionError ____________ test_match_new ___________________ def test_match_new(): > assert Matches('foo') == 'fob' E AssertionError: assert Matches('foo') ^ == 'fob' E -Matches('foo') E - # regex failed to match at: E - # E - #> fob E - # ^ E +'fob' Michael #5: awesome-python-typing Collection of awesome Python types, stubs, plugins, and tools to work with them. Taxonomy Static type checkers Stub packages Tools Integrations Articles Communities Related Static type checkers: mypy - Optional static typing for Python 3 and 2 (PEP 484). Stub packages: Typeshed - Collection of library stubs for Python, with static types. Tools (super category): pytest-mypy - Mypy static type checker plugin for Pytest. Articles: Typechecking Django and DRF - Full tutorial about type-checking django. Brian #6: Developer Advocacy: Frequently Asked Questions Dustin Ingram I know a handful of people who have this job title. What is it? disclaimer: Dustin is a DA at Google. Other companies might be different What is it? “I help represent the Python community at [company]" “part of my job is to be deeply involved in the Python community.” working on projects that help Python, PyPI, packaging, etc. speaking at conferences talking to people. customers and non-customers talking to product teams being “user zero” for new products and features paying attention to places users might raise issues about products working in open source creating content for Python devs being involved in the community as a company rep representing Python in the company coordinating with other DAs Work/life? Not all DAs travel all the time. that was my main question. Talk Python episode: War Stories of the Developer Evangelists Extras: https://www.meetup.com/Python-PDX-West/ Michael: requests moves to PSF Joke: via https://twitter.com/NotGbo/status/1173667028965777410 Web Dev Merit Badges

Sponsored by Datadog: pythonbytes.fm/datadog Brian #1: Dropbox: Our journey to type checking 4 million lines of Python Continuing saga, but this is a cool write up. Benefits “Experience tells us that understanding code becomes the key to maintaining developer productivity. Without type annotations, basic reasoning such as figuring out the valid arguments to a function, or the possible return value types, becomes a hard problem. Here are typical questions that are often tricky to answer without type annotations: Can this function return None? What is this items argument supposed to be? What is the type of the id attribute: is it int, str, or perhaps some custom type? Does this argument need to be a list, or can I give a tuple or a set?” Type checker will find many subtle bugs. Refactoring is easier. Running type checking is faster than running large suites of unit tests, so feedback can be faster. Typing helps IDEs with better completion, static error checking, and more. Long story, but really cool learnings of how and why to tackle adding type hints to a large project with many developers. Conclusion. mypy is great now, because DropBox needed it to be. Michael #2: Setting Up a Flask Application in Visual Studio Code Video, but also as a post Follow on to the same in PyCharm: video and post Steps outside VS Code Clone repo Create a virtual env (via venv) Install requirements (via requirements.txt) Setup flask app ENV variable flask deploy ← custom command for DB VS Code Open the folder where the repo and venv live Open any Python file to trigger the Python subsystem Ensure the correct VENV is selected (bottom left) Open the debugger tab, add config, pick Flask, choose your app.py file Debug menu, start without debugging (or with) Adding tests via VS Code Open command pallet (CMD SHIFT P), Python: Discover Tests, select framework, select directory of tests, file pattern, new tests bottle on the right bar Brian #3: Multiprocessing vs. Threading in Python: What Every Data Scientist Needs to Know How data scientists can go about choosing between the multiprocessing and threading and which factors should be kept in mind while doing so. Does not consider async, but still some great info. Overview of both concepts in general and some of the pitfalls of parallel computing. The specifics in Python, with the GIL Use threads for waiting on IO or waiting on users. Use multiprocessing for CPU intensive work. The surprising bit for me was the benchmarks Using something speeds up the code. That’s obvious. The difference between the two isn’t as great as I would have expected. A discussion of merits and benefits of both. And from the perspective of data science. A few more examples, with code, included. Michael #4: ORM - async ORM And https://github.com/encode/databases The orm package is an async ORM for Python, with support for Postgres, MySQL, and SQLite. SQLAlchemy core for query building. databases for cross-database async support. typesystem for data validation. Because ORM is built on SQLAlchemy core, you can use Alembic to provide database migrations. Need to be pretty async savy Brian #5: Getting Started with APIs dataquest.io post Conceptual introduction of web APIs Discussion of GET status codes, including a nice list with descriptions. examples: 301: The server is redirecting you to a different endpoint. This can happen when a company switches domain names, or an endpoint name is changed. 400: The server thinks you made a bad request. This can happen when you don’t send along the right data, among other things. endpoints endpoints that take query parameters JSON data Examples in Python for using: requests to query endpoints. json to load and dump JSON data. Michael #6: Memory management in Python This article describes memory management in Python 3.6. Everything in Python is an object. Some objects can hold other objects, such as lists, tuples, dicts, classes, etc. such an approach requires a lot of small memory allocations To speed-up memory operations and reduce fragmentation Python uses a special manager on top of the general-purpose allocator, called PyMalloc. Layered managers RAM OS VMM C-malloc PyMem Python Object allocator Object memory Three levels of organization To reduce overhead for small objects (less than 512 bytes) Python sub-allocates big blocks of memory. Larger objects are routed to standard C allocator. three levels of abstraction — arena, pool, and block. Block is a chunk of memory of a certain size. Each block can keep only one Python object of a fixed size. The size of the block can vary from 8 to 512 bytes and must be a multiple of eight A collection of blocks of the same size is called a pool. Normally, the size of the pool is equal to the size of a memory page, i.e., 4Kb. The arena is a chunk of 256kB memory allocated on the heap, which provides memory for 64 pools. Python's small object manager rarely returns memory back to the Operating System. An arena gets fully released If and only if all the pools in it are empty. Extras Brian: Tuesday, Oct 6, Python PDX West, Thursday, Sept 26, I’ll be speaking at PDX Python, downtown. Both events, mostly, I’ll be working on new programming jokes unless I come up with something better. :) Michael: Gitbook Call for Proposals for PyCon 2020 Is Open Jokes: A few I liked from the dad joke list. What do you call a 3.14 foot long snake? A π-thon What if it’s 3.14 inches, instead of feet? A μ-π-thon Why doesn't Hollywood make more Big Data movies? NoSQL. Why didn't the div get invited to the dinner party? Because it had no class.

Sponsored by DigitalOcean: pythonbytes.fm/digitalocean Brian #1: Annual Release Cycle for Python - PEP 602 Under discussion Annual release cadence Seventeen months to develop a major version 5 months unversioned, 7 months alpha releases when new features and fixes come in, 4 months of betas with no new features, 1 month final RCs. One year of full support, four more years of security fixes. Rationale/Goals smaller releases features and fixes to users sooner more gradual upgrade path predictable calendar releases that line up will with sprints and PyConUS explicit alpha phase decrease pressure and rush to get features into beta 1 Risks Increase concurrent supported versions from 4 to 5. Test matrix increase for integrators and distributions. PEP includes rejected ideas like: 9 month cadence keep 18 month cadence Michael #2: awesome-asgi A curated list of awesome ASGI servers, frameworks, apps, libraries, and other resources ASGI is a standard interface positioned as a spiritual successor to WSGI. It enables communication and interoperability across the whole Python async web stack: servers, applications, middleware, and individual components. Born in 2016 to power the Django Channels project, ASGI and its ecosystem have been expanding ever since, boosted by the arrival of projects such as Starlette and Uvicorn in 2018. Frameworks for building ASGI web applications. Bocadillo - Fast, scalable and real-time capable web APIs for everyone. Powered by Starlette. Supports HTTP (incl. SSE) and WebSockets. Channels - Asynchronous support for Django, and the original driving force behind the ASGI project. Supports HTTP and WebSockets with Django integration, and any protocol with ASGI-native code. FastAPI - A modern, high-performance web framework for building APIs with Python 3.6+ based on standard Python type hints. Powered by Starlette and Pydantic. Supports HTTP and WebSockets. Quart - A Python ASGI web microframework whose API is a superset of the Flask API. Supports HTTP (incl. SSE and HTTP/2 server push) and WebSockets. Responder - A familiar HTTP Service Framework for Python, powered by Starlette. (ASGI 2.0 only, ed.) Starlette - Starlette is a lightweight ASGI framework/toolkit, which is ideal for building high performance asyncio services. Supports HTTP and WebSockets. Brian #3: Jupyter meets the Earth Lindsey Heagy & Fernando Pérez “We are thrilled to announce that the NSF is funding our EarthCube proposal “Jupyter meets the Earth: Enabling discovery in geoscience through interactive computing at scale” “ “This project provides our team with $2 Million in funding over 3 years as a part of the NSF EarthCube program. It also represents the first time federal funding is being allocated for the development of core Jupyter infrastructure.” “Our project team includes members from the Jupyter and Pangeo communities, with representation across the geosciences including climate modeling, water resource applications, and geophysics. Three active research projects, one in each domain, will motivate developments in the Jupyter and Pangeo ecosystems. Each of these research applications demonstrates aspects of a research workflow which requires scalable, interactive computational tools.” “The adoption of open languages such as Python and the coalescence of communities of practice around open-source tools, is visible in nearly every domain of science. This is a fundamental shift in how science is conducted and shared.” Geoscience use cases climate data analysis hydrologic modeling geophysical inversions User-Centered Development data discovery scientific discovery through interactive computing established tools and data visualization using and managing shared computational infrastructure Michael #4: Asynchronous Django via Jose Nario Python compatibility Django 3.0 supports Python 3.6, 3.7, and 3.8. We highly recommend and only officially support the latest release of each series The Django 2.2.x series is the last to support Python 3.5. Other items but Big news: ASGI support Django 3.0 begins our journey to making Django fully async-capable by providing support for running as an ASGI application. This is in addition to our existing WSGI support. Django intends to support both for the foreseeable future. Note that as a side-effect of this change, Django is now aware of asynchronous event loops and will block you calling code marked as “async unsafe” - such as ORM operations - from an asynchronous context. Brian #5: The 1x Engineer Fun take on 10x. List actually looks like probably a 3-4x to me, maybe even 8x or more. How high does this scale go anyway? non-exhaustive list qualities, here’s a few. Has a life outside engineering. Writes code that others can read. Doesn't act surprised when someone doesn’t know something. Asks for help when they need it. Is able to say "I don't know." Asks questions. Constructively participates in code reviews. Can collaborate with others. Supports code, even if they did not write it. Can feel like an impostor at times. Shares knowledge. Never stops learning. [obviously listens to Python Bytes, Talk Python, and Test & Code] Is willing to leave their comfort zone. Contributes to the community. Has productive and unproductive days. Doesn't take themselves too seriously. Fails from time to time. Has a favorite editor, browser, and operating system, but realizes others do too. Michael #6: Sunsetting Python 2 January 1, 2020, will be the day that we sunset Python 2 Why are you doing this? We need to sunset Python 2 so we can help Python users. How long is it till the sunset date? pythonclock.org will tell you. What will happen if I do not upgrade by January 1st, 2020? If people find catastrophic security problems in Python 2, or in software written in Python 2, then most volunteers will not help fix them. I wrote code in Python 2. How should I port it to Python 3? Please read the official "Porting Python 2 Code to Python 3" guide. I didn't hear anything about this till just now. Where did you announce it? We talked about it at software conferences, on the Python announcement mailing list, on official Python blogs, in textbooks and technical articles, on social media, and to companies that sell Python support. Extras Brian: working on a Portland Westside Python Meetup, info will be at pythonpdx.com Hoping to get something ready for Oct. But… if not, hopefully by Nov. Michael: Humble Level Up Your Python Bundle

Sponsored by DigitalOcean: pythonbytes.fm/digitalocean Brian #1: rapidtables “rapidtables … converts lists of dictionaries to pre-formatted tables. And it does the job as fast as possible.” Also can do color formatting if used in conjunction with termcolor.colored, but I’m mostly excited about really easily generating tabular data with print. Can also format to markdown or reStructured text, and can do alignment, … Michael #2: httpx A next generation HTTP client for Python. 🦋 HTTPX builds on the well-established usability of requests, and gives you: A requests-compatible API. HTTP/2 and HTTP/1.1 support. Support for issuing HTTP requests in parallel. (Coming soon) Standard synchronous interface, but with [async](https://www.encode.io/httpx/async/)/[await](https://www.encode.io/httpx/async/) support if you need it. Ability to make requests directly to WSGI or ASGI applications. This is particularly useful for two main use-cases: Using httpx as a client, inside test cases. Mocking out external services, during tests or in dev/staging environments. Strict timeouts everywhere. Fully type annotated. 100% test coverage. Lovely support for “parallel requests” without full asyncio (at the API level). Also pairs with async / await with async client. Plus all the requests features Brian #3: Quick and dirty mock service with Starlette Matt Layman Mock out / fake a third party service in a testing environment. Starlette looks fun, but the process can be used with other API producing server packages. We tell people to do things like this all the time, but there are few examples showing how to. This example also introduces a delay because the service used in production takes over a minute and part of the testing is to make sure the system under test handles that delay gracefully. Very cool, easy to follow write up. (Should probably have Matt on a Test & Code episode to talk about this strategy.) Michael #4: Mocking out AWS APIs via Giuseppe Cunsolo A library that allows you to easily mock out tests based on AWS infrastructure. Lovely use of a decorator to mock out S3 Moto isn't just for Python code and it isn't just for S3. Look at the standalone server mode for more information about running Moto with other languages. Be sure to check out very important note. Brian #5: μMongo: sync/async ODM “μMongo is a Python MongoDB ODM. It inception comes from two needs: the lack of async ODM and the difficulty to do document (un)serialization with existing ODMs.” works with common mongo drivers such as PyMongo, TxMongo, motor_asyncio, and mongomock. (Hadn’t heard of mongomock before, I’ll have to try that out.) Note: We’ve discussed MongoEngine before. (I’m curious what Michael has to say about uMongo.) Michael #6: Single Responsibility Principle in Python via Tyler Matteson I’m a big fan of the SOLID principles They even come in demotivator style posters DI Liskov Substitution Principle Open/Closed Principle Single Responsibility Principle Interface Segregation Principle This article will guide you through a complex process of writing simple code. Extras Michael: Code Challenge Bite 220. Analyzing @pythonbytes RSS feed Jokes Q: What do you get when you cross a computer and a life guard? A: A screensaver! Q: What do you get when you cross a computer with an elephant? A: Lots of memory! via https://github.com/wesbos/dad-jokes Anti-joke (we ready for those yet?): A Python developer, a PHP developer, a C# developer, and a Go developer went to lunch together. They had a nice lunch and got along fine.

Sponsored by DigitalOcean: pythonbytes.fm/digitalocean Special guest: Trey Hunner Brian #1: Positional-only arguments in Python by Sanket Feature in 3.8 “To specify arguments as positional-only, a / marker should be added after all those arguments in the function definition. “ Great example of a pow(x, y, /, mod=None) function where the names x and y are meaningless and if given in the wrong order would be confusing. Trey #2: django-stubs Type checking for Django! It’s new and from my very quick testing on my Django site it definitely doesn’t work with everything yet, but it’s promising I don’t use type annotations in Django yet, but I very well might eventually Michael #3: CodeCombat Super fun game for learning to code Real code but incredibly easy coding Can subscribe or use the free tier Just got $6M VC funding Brian #4: Four Use Cases for When to Use Celery in a Flask Application or really any web framework by Nick Janetakis “Celery helps you run code asynchronously or on a periodic schedule which are very common things you'd want to do in most web projects.” examples: sending emails out connecting to 3rd party APIs. performing long running tasks. Like, really long. Running tasks on a schedule. Trey #5: pytest-steps Created by smarie Can use a generator syntax with yield statements to break a big test up into multiple named “steps” that’ll show up in your pytest output If one step fails, the rest of the steps will be skipped by default You can also customize it to make optional steps, which aren’t required for future steps to run, or steps which depend on other optional steps explicitly The documentation shows a lot of different ways to use it, but the generator approach looks by far the most readable to me (also state is shared between steps with this approach whereas the others require some fancy state-capturing object which looks confusing to me) I haven’t tried this, but my use case would be my end-to-end/functional tests, which would work great with steps because I’m often using Selenium to navigate between a number of different pages and forms, one click at a time. Michael #6: docassemble Created by Jonathan Pyle A free, open-source expert system for guided interviews and document assembly, based on Python, YAML, and Markdown. Features WYSIWYG: Compose your templates in .docx (with help of a Word Add-in) or .pdf files. Signatures: Gather touchscreen signatures and embed them in documents. Live chat: Assist users in real time with live chat, screen sharing, and remote screen control. AI: Use machine learning to process user input. SMS: Send text messages to your users E-mail: Send and receive e-mails in your interviews. OCR: Use optical character recognition to process images uploaded by the user. Multilingual: Offer interviews in multiple languages. Multiuser: Develop applications that involve more than one user, such as mediation or counseling interviews. Extensible: Use the power of Python to extend the capabilities of your interviews. Open: Package your interviews and use GitHub and PyPI to share your work with the docassemble user community. Background Tasks: Do things behind the scenes of the interview, even when the user is not logged in. Scalable: Deploy your interviews on multiple machines to handle high traffic. Secure: Protect user information with server-side encryption, two-factor authentication, document redaction, and other security features. APIs: Integrate with third-party applications using the API, or send your interviews input and extract output using code. Extras Michael: PyPI closes in on 200k NumPy 1.17.0 released Python 3.8.0b4 is out Joke via Avram Lubkin Knock! Knock! Who's there? Recursive function. Recursive function who? Knock! Knock! Nice. to get that joke, you’ll have to understand recursion. to understand recursion, either google “recursion”, and click on “did you mean “recursion”” learn it in small steps. step one, recursion text conversation: first person: “Hey, what’s your address?” second: gives IP address first: “No. Your local address” second: 127.0.0.1 first: “No. Your physical address” second: gives MAC address

Sponsored by Datadog: pythonbytes.fm/datadog Special guests Matt Harrison Anthony Sottile Michael #1: friendly-traceback via Jose Carlos Garcia (I think 🙂 ) Aimed at Python beginners: replacing standard traceback by something easier to understand Shows help for exception type Shows local variable values Shows code in a cleaner form with more context 3 ways to install As an exception hook Explicit explain When running an app Matt #2: Pandas Users Survey Most use it almost everyday but have less than 2 years experience Linux 61%, Windows 60%, Mac 42% 93% Python 3 Anthony #3: python3 “Y2K” problem (python3.10 / python4.0) with python3.8 close to release and python3.9 right around the corner, what comes after? both python3.10 and python4.0 present some problems sys.version[:3] which will suddenly report '``3.1``' in 3.10 a lot of code (including six.PY3!) uses sys.version_info[0] == 3 which will suddenly be false in python4.0 (and start running python2 code!) early-to-mid 2020 we should start seeing the next version in the wild as python3.9 reaches beta easy ways to start testing this early: python3.10 - a build of cpython for ubuntu with the version number changed flake8-2020 - a flake8 plugin which checks for these common issues- Michael #4: pypi research via Adam (Codependent Codr) Really interesting research paper on the current state of Pypi from a couple authors at the University of Michigan: "An Empirical Analysis of the Python Package Index" - https://arxiv.org/pdf/1907.11073.pdf Comprehensive empirical summary of the Python Package Repository, PyPI, including both package metadata and source code covering 178,592 packages, 1,745,744 releases, 76,997 contributors, and 156,816,750 import statements. We provide counts and trends for packages, releases, dependencies, category classifications, licenses, and package imports, as well as authors, maintainers, and organizations. Within PyPI, we find that the growth of the repository has been robust under all measures, with a compound annual growth rate of 47% for active packages, 39% for new authors, and 61% for new import statements over the last 15 years. In 2005, there were 96 active packages, 96! MIT is the most common license (Matt) I saw this and was surprised at most commonly used libraries. What do you think the most common 3rd party library is? Matt #5: DaPy “Pandas for humans” - Matt’s words Has portions of pandas, scikit-learn, yellowbrick, and numpy Designed for “data analysis, not for coders” Anthony #6: python-remote-pdb very small over-the-network remote debugger thin wrapper around pdb in a single file (easy to drop the file on PYTHONPATH if you can’t pip install) not as fully featured as other remote debuggers such as pudb / rpdb / pycharm’s debugger but very easy to drop in fully supports [breakpoint()](https://www.python.org/dev/peps/pep-0553/) (python3.7+ or via future-breakpoint) access pdb via telnet / nc / socat I’m using it to debug a text editor I’m writing to learn curses! Extras: Michael: Hacker Gets $12,000 In Parking Tickets After 'NULL' License Plate Trick Backfires PyCon 2020 site is up Matt http://bit.ly/psxgb - My new course on Machine Learning with XGBoost Anthony: https://github.com/DRMacIver/hecate “like selenium webdriver for the terminal” Jokes: Michael: Two mathematicians are sitting at a table in a pub having an argument about the level of math education among the general public. The one defending overall math knowledge gets up to go to the washroom. On the way back, he encounters their waitress and says, "I'll add an extra $10 to your tip, if you'll answer a question for me when I ask it. All you have to say is 'x-squared'." She agrees. A few minutes later the populist mathematician says to his buddy, "I'll bet you $20 that even our waitress can tell us the integral of 2x." The cynic agrees to the bet. So the schemer beckons the waitress to their table and asks the question, to which she replies "x-squared". As he begins to gloat and demand his winnings, the waitress continues, "Plus a constant." Anthony: I had a golang joke prepared, but then I panic()d