113 Episodes

Reverse

Sponsored by https://pythonbytes.fm/digitaloceanBrian #1: Advent of Code 2018 SolutionsMichael FoglemanEven if you didn’t have time or energy to do the 2018 AoC, you can learn from other peoples solutions. Here’s one set written up in a nice blog post.Michael #2: Python Lands on the Windows 10 App StorePython Software Foundation recently released Python 3.7 as an app on the official Windows 10 app store. Python 3.7 is now available to install from the Microsoft Store, meaning you no longer need to manually download and install the app from the official Python website.there is one limitation. “Because of restrictions on Microsoft Store apps, Python scripts may not have full write access to shared locations such as TEMP and the registry. Discussed with Steve Dower over on Talk Python 191Brian #3: How I Built A Python Web Framework And Became An Open Source MaintainerFlorimond MancaBocadillo - “A modern Python web framework filled with asynchronous salsa””maintaining an open source project is a marathon, not a sprint.”Tips at the end of the article include tips for the following topics, including recommendations and tool choices:Project definitionMarketing & CommunicationCommunityProject managementCode qualityDocumentationVersioning and releasingMichael #4: Python maintainability score via Wilyvia Anthony ShawA Python application for tracking, reporting on timing and complexity in testsEasiest way to calculate it is with wily https://github.com/tonybaloney/wily … the metrics are ‘maintainability.mi’ and ‘maintainability.rank’ for a numeric and the A-F scale.Build an index: wily build srcInspect report: wily report fileGraph: wily graph file metricBrian #5: A couple fun awesome listsAwesome Python Security resourcesToolsweb framework hardening, ex: secure.pymulti toolsstatic code analysis, ex: banditvulnerabilities and security advisoriescryptographyapp templatesEducationlots of resources for learningCompaniesAwesome Flake8 Extensionsclean codetesting, including flake8-pytest - Enforces to use pytest-style assertionsflake8-mock - Provides checking mock non-existent methodssecuritydocumentationenhancementscopyrightsMichael #6: fastloggingvia Robert YoungA faster replacement of the standard logging module with a mostly compatible API.For a single log file it is ~5x faster and for rotating log file ~13x faster.It comes with the following features:(colored, if colorama is installed) logging to consolelogging to file (maximum file size with rotating/history feature can be configured)old log files can be compressed (the compression algorithm can be configured)count same successive messages within a 30s time frame and log only once the message with the counted value.log domainslog to different fileswriting to log files is done in (per file) background threads, if configuredconfigure callback function for custom detection of same successive log messagesconfigure callback function for custom message formatterconfigure callback function for custom log writerExtras:Michael: My webcast on async, Jan 24, 11am PTMichael: Watch your YAML! Joke: >>> import antigravity

#113 Python Lands on the Windows 10 App Store

2019-01-1800:23:22

Sponsored by https://pythonbytes.fm/datadogBrian #1: nbgradernbgrader: A Tool for Creating and Grading Assignments in the Jupyter NotebookThe Journal of Open Source Education, paper accepted 6-Jan-2019nbgrader documentation, including a intro videoFrom the JOSE article:“nbgrader is a flexible tool for creating and grading assignments in the Jupyter Notebook (Kluyver et al., 2016). nbgrader allows instructors to create a single, master copy of an assignment, including tests and canonical solutions. From the master copy, a student version is generated without the solutions, thus obviating the need to maintain two separate versions. nbgrader also automatically grades submitted assignments by executing the notebooks and storing the results of the tests in a database. After auto-grading, instructors can manually grade free responses and provide partial credit using the formgrader Jupyter Notebook extension. Finally, instructors can use nbgrader to leave personalized feedback for each student’s submission, including comments as well as detailed error information.”CS teaching methods have come a long ways since I was turning in floppies and code printouts.Michael #2: profanity-checkA fast, robust Python library to check for offensive language in strings.profanity-check uses a linear SVM model trained on 200k human-labeled samples of clean and profane text strings.Making profanity-check both robust and extremely performantOther libraries like profanity-filter use more sophisticated methods that are much more accurate but at the cost of performance.profanity-filter runs in 13,000ms vs 24ms for profanity-check in a benchmarkTwo ways to use:predict(text) → 0 or 1 (1 = bad)predict_prob(text) → [0, 1] confidence interval (1 = bad)Brian #3: An Introduction to Python Packages for Absolute BeginnersEver tried to explain the difference between module and package? Between package-in-the-directory-with-init sense and package-you-can-distribute-and-install-with-pip sense? Here’s the article to read beforehand.Modules, packages, using packages, installing, importing, and more.And that’s not even getting into flit and poetry, etc. But it’s a good place to start for people new to Python.Michael #4: Python Dependencies and IoCvia Joscha GötzerOpen-closed principle is at work with these and is super valuable to testing (one of the SOLID principles): Software entities (classes, modules, functions, etc.) should be open for extension, but closed for modification.There is a huge debate around why Python doesn’t need DI or Inversion of Control (IoC), and a quick stackoverflow search yields multiple results along the lines of “python is a scripting language and dynamic enough so that DI/IoC makes no sense”. However, especially in large projects it might reduce the cognitive load and decoupling of individual componentsDependency Injector: I couldn’t get this one to work on windows, as it needs to compile some C libraries and some Visual Studio tooling was missing that I couldn’t really install properly. The library looks quite promising though, but sort of static with heavy usage of containers and not necessarily pythonic.Injector: The library that above mentioned article talks about, a little Java-esquepinject: Has been unmaintained for about 5 years, and only recently got new attention from some open source people who try to port it to python3. A product under Google copyright, and looks quite nice despite the lack of python3 bindings. Probably the most feature-rich of the listed libraries.python-inject: I discovered that one while writing this email, not really sure if it’s any good. Nice use of type annotations and testing featuresdi-py: Only works up to python 3.4, so I’ve also never tried it (I’m one of those legacy python haters, I’m sure you can relate 😄).Serum: This one is a little too explicit to my mind. It makes heavy use of context managers (literally with Context(...): everywhere 😉) and I’m not immediately sure how to work with it. In this way, it is quite powerful though. Interesting use of class decorators.And now on to my favorite and a repeated recommendation of mine around the internet→ Haps: This lesser-known, lightweight library is sort of the new kid on the block, and really simple to use. As some of the other libraries, it uses type annotations to determine the kind of object it is supposed to instantiate, and automatically discovers the required files in your project folder. Haps is very pythonic and fits into apps of any size, helping to ensure modularization as the only dependency of your modules will be one of the types provided by the library. Pretty good example here.Brian #5: A Gentle Introduction to PandasReally a gentle introduction to the Pandas data structures Series and DataFrame.Very gentle, with console examples.Create series objects:from an arrayfrom an array, and change the indexingfrom a dictionariesfrom a scalar, cool. didn’t know you could do thatAccessing elements in a seriesDataFramessorting, slicingselecting by label, positionstatistics on columnsimporting and exporting dataMichael #6: Don't use the greater than sign in programmingOne simple thing that comes up time and time again is the use of the greater than sign as part of a conditional while programming. Removing it cleans up code.Let's say that I want to check that something is between 5 and 10. There are many ways I can do this x > 5 and 10 > x 5 < x and 10 > x x > 5 and x < 10 10 < x and x < 5 x < 10 and x > 5 x < 10 and 5 < xSorry, one of those is incorrect. Go ahead and find out which oneIf you remove the use of the greater than sign then only 2 options remainx < 10 and 5 < x5 < x and x < 10The last is nice because x is literally between 5 and 10There is also a nice way of expressing that "x is outside the limits of 5 and 10”x < 5 or 10 < xAgain, this expresses it nicely because x is literally outside of 5 to 10.Interesting comment: What is cleaner or easier to read comes down to personal taste. But how to express "all numbers greater than 1" without '>'? ans: 1 < allNumbersExtrasMichaelTeaching Python podcast by Kelly Paredes & Sean TiborGithub private repos (now free)EuroPython 2019 announcedSouth African AWS Data Center coming (via William H.)Pandas is dropping legacy Python support any day nowJoke: Harry Potter Parser Tongue via Nick Spirit

#112 Don't use the greater than sign in programming

2019-01-1100:28:472

Sponsored by https://pythonbytes.fm/datadogBrian #1: loguru: Python logging made (stupidly) simpleFinally, a logging interface that is just slightly more syntax than print to do mostly the right thing, and all that fancy stuff like log rotation is easy to figure out.i.e. a logging API that fits in my brain.bonus: README is a nice tour of features with examples.Features:Ready to use out of the box without boilerplateNo Handler, no Formatter, no Filter: one function to rule them allEasier file logging with rotation / retention / compressionModern string formatting using braces styleExceptions catching within threads or mainPretty logging with colorsAsynchronous, Thread-safe, Multiprocess-safeFully descriptive exceptionsStructured logging as neededLazy evaluation of expensive functionsCustomizable levelsBetter datetime handlingSuitable for scripts and librariesEntirely compatible with standard loggingPersonalizable defaults through environment variablesConvenient parserExhaustive notifierMichael #2: Python gets a new governance modelby Brett CanonJuly 2018, Guido steps downPython progress has basically been on hold since thenended up with 7 governance proposalsVoting was open to all core developers as we couldn't come up with a reasonable criteria that we all agreed to as to what defined an "active" core devAnd the winner is ... In the end PEP 8016, the steering council proposal, won.it was a decisive win against second placePEP 8016 is heavily modeled on the Django project's organization (to the point that the PEP had stuff copy-and-pasted from the original Django governance proposal).What it establishes is a steering council of five people who are to determine how to run the Python project. Short of not being able to influence how the council itself is elected (which includes how the electorate is selected), the council has absolute power.result of the vote prevents us from ever having the Python project be leaderless again, it doesn't directly solve how to guide the language's design.What's next? The next step is we elect the council. It's looking like nominations will be from Monday, January 07 to Sunday, January 20 and voting from Monday, January 21 to Sunday, February 03A key point I hope people understand is that while we solved the issue of project management that stemmed from Guido's retirement, the council will need to be given some time to solve the other issue of how to manage the design of Python itself.Brian #3: Why you should be using pathlibTour of pathlib from Trey Hunnerpathlib combines most of the commonly used file and directory operations from os, os.path, and glob.uses objects instead of stringsas of Python 3.6, many parts of stdlib support pathlibsince pathlib.Path methods return Path objects, chaining is possibleconvert back to strings if you really need to for pre-3.6 codeExamples:make a directory: Path('src/__pypackages__').mkdir(parents=True, exist_ok=True)rename a file: Path('.editorconfig').rename('src/.editorconfig')find some files: top_level_csv_files = Path.cwd().glob('*.csv')recursively: all_csv_files = Path.cwd().rglob('*.csv')read a file: Path('some/file').read_text()write to a file: Path('.editorconfig').write_text('# config goes here')with open(path, mode) as x works with Path objects as of 3.6Follow up article by Trey: No really, pathlib is greatMichael #4: Altair and Altair Recipesvia Antonio Piccolboni (he wrote altair_recipes)Altair: Declarative statistical visualization library for PythonAltair is developed by Jake Vanderplas and Brian GrangerBy statistical visualization they mean:The data source is a DataFrame that consists of columns of different data types (quantitative, ordinal, nominal and date/time).The DataFrame is in a tidy format where the rows correspond to samples and the columns correspond to the observed variables.The data is mapped to the visual properties (position, color, size, shape, faceting, etc.) using the group-by data transformation.Nice example that I can get behind # cars = some Pandas data frame alt.Chart(cars).mark_point().encode( x='Horsepower', y='Miles_per_Gallon', color='Origin', )altair_recipesAltair allows generating a wide variety of statistical graphics in a concise language, but lacks, by design, pre-cooked and ready to eat statistical graphics, like the boxplot or the histogram. Examples: https://altair-recipes.readthedocs.io/en/latest/examples.html They take a few lines only in altair, but I think they deserve to be one-liners. altair_recipes provides that level on top of altair. The idea is not to provide a multitude of creative plots with fantasy names (the way seaborn does) but a solid collection of classics that everyone understands and cover most major use cases: the scatter plot, the boxplot, the histogram etc. Fully documented, highly consistent API (see next package), 90%+ test coverage, maintainability grade A, this is professional stuff if I may say so myself.Brian #5: A couple fun pytest pluginspytest-pickedUsing git status, this plugin allows you to:Run only tests from modified test filesRun tests from modified test files first, followed by all unmodified testsKinda hard to overstate the usefulness of this plugin to anyone developing or debugging a test. Very, very cool.pytest-clarityColorized left/right comparisonsEarly in development, but already helpful.I recommend running it with -qq if you don’t normally run with -v/--verbose since it overrides the verbosity currently.Michael #6: Secure 🔒 headers and cookies for Python web frameworksPython package called Secure, which sets security headers and cookies (as a start) for Python web frameworks.I was listening to the Talk Python To Me episode “Flask goes 1.0” with Flask maintainer David Lord. At the end of the interview he was asked about notable PyPI packages and spoke about Flask-Talisman, a third-party package to set security headers in Flask. As a security professional, it was surprising and encouraging to hear the maintainer of the most popular Python web framework speak passionately about a security package. Had been recently experimenting with emerging Python web frameworks and realized there was a gap in security packages. That inspired Caleb to (humbly) see if it were possible to make a package to correct that and I started with Responder and then expanded to support more frameworks. The outcome was Secure with functions to support aiohttp, Bottle, CherryPy, Falcon, hug, Pyramid, Quart, Responder, Sanic, Starlette and Tornado (most of these, if not all have been featured on Talk Python) and can also be utilized by frameworks not officially supported. The goal is to be minimalistic, lightweight and be implemented in a way that does not disrupt an individual framework’s design. I have had some great feedback and suggestions from the developer and OWASP community, including some awesome discussions with the OWASP Secure Project and the Sanic core team. Added support for Flask and Django too.Secure Cookies is nice in the mixExtras:Michael: SQLite bug impacts thousands of apps, including all Chromium-based browsersSee https://twitter.com/mborus/status/1080874700924964864Since this bug is triggered by an SQL command, general CPython usage should not be affected, and long as you don’t run arbitrary SQL-commands provided by the outside.Seems to NOT be a problem in CPython: https://twitter.com/mborus/status/1080883549308362753Michael: Follow up to our AI and healthcare conversationvia Bradley HintzeI found your discussion of deep learning in healthcare interesting, no doubt because that is my area. I am the data scientist for the National Oncology Program at the Veterans Health Administration. I work directly with clinicians and it is my strong opinion that AI cannot take the job from the MD. It will however make caring for patients much more efficient as AI takes care of the low hanging fruit, it you will.Healthcare, believe it or not, is a science and an art. This is why AI is never going to make doctors obsolete. It will, however, make doctors more efficient and demanded a more sophisticated doctor -- one that understands AI enough to not only trust it but, crucially, comprehend its limits.Michael: Upgrade to Python 3.7.2If you install via home brew, it’s time for brew update && brew upgradeMichael: New course! Introduction to Ansible

#111 loguru: Python logging made simple

2019-01-0500:34:162

Sponsored by DigitalOcean: pythonbytes.fm/digitaloceanThis episode originally aired on Talk Python at talkpython.fm/192.It's been a fantastic year for Python. Literally, every year is better than the last with so much growth and excitement in the Python space. That's why I've asked two of my knowledgeable Python friends, Dan Bader and Brian Okken, to help pick the top 10 stories from the Python community for 2018.GuestsBrian Okken @brianokkenDan Bader @dbader_org10: Python 3.7: Cool New Features in Python 3.79: Changes in versioning patternsZeroVer: 0-based VersioningCalendar VersioningSemantic Versioning 2.0.08: Python is becoming the world’s most popular coding languageEconomist article7: 2018 was the year data science Pythonistas == web dev PythonistasPython Developers Survey ResultsCovered in depth on Talk Python 1766: BlackProjectSoundgarden : “Black Hole Sun”5: New PyPI launched!Python Package Index4: Rise of Python in the embedded worldCovered at Python Bytes3: Legacy Python's days are fading?Python 2.7 -- bugfix or security before EOL?Python 2 death clockhttps://pythonclock.org/2: It's the end of innocence for PyPiTwelve malicious Python libraries found and removed from PyPI1: Guido stepped down as BDFLpython-committers: Transfer of powerProposals for new governance structure

#110 Python Year in Review 2018 Edition

2018-12-2600:56:541

Sponsored by DigitalOcean: pythonbytes.fm/digitaloceanBrian #1: Python Descriptors Are Magical Creaturesan excellent discussion of understanding @property and Python’s descriptor protocol.discussion includes getter, setter, and deleter methods you can override.Michael #2: Data Science Survey 2018 JetBrainsJetBrains polled over 1,600 people involved in Data Science and based in the US, Europe, Japan, and China, in order to gain insight into how this industry sector is evolvingKey TakeawaysMost people assume that Python will remain the primary programming language in the field for the next 5 years.Python is currently the most popular language among data scientists. Data Science professionals tend to use Keras and Tableau, while amateur data scientists are more likely to prefer Microsoft Azure ML. Most common activities among pros and amateurs: Data processingData visualizationMain programming language for data analysisPython 57%R 15%Julia 0%IDEs and EditorsJupyter 43%PyCharm 38%RStudio 23%…Brian #3: cache.pycache.py is a one file python library that extends memoization across runs using a cache file.memoization is an incredibly useful technique that many self taught or on the job taught developers don’t know about, because it’s not obvious.example: import cache @cache.cache() def expensive_func(arg, kwarg=None): # Expensive stuff here return argThe @cache.cache() function can take multiple arguments.@cache.cache(timeout=20) - Only caches the function for 20 seconds.@cache.cache(fname="my_cache.pkl") - Saves cache to a custom filename (defaults to hidden file .cache.pkl)@cache.cache(key=cache.ARGS[KWARGS,NONE]) - Check against args, kwargs or neither of them when doing a cache lookup.Michael #4: Setting up the data science toolspart of a larger video seriesset up. Tools to keras ultimatelyToolsanacondatensorflowJupyterKerasgood for true beginners setup and activate a condo venv Start up a notebook and switch envs use conda, rather than pipBrian #5: chartify“Python library that makes it easy for data scientists to create charts.”from the docs:Consistent input data format: Spend less time transforming data to get your charts to work. All plotting functions use a consistent tidy input data format.Smart default styles: Create pretty charts with very little customization required.Simple API: We've attempted to make to the API as intuitive and easy to learn as possible.Flexibility: Chartify is built on top of Bokeh, so if you do need more control you can always fall back on Bokeh's API.Michael #6: CPython byte code explorerJupyterLab extension to inspect Python Bytecodevia Anton Helmby Jeremy TuloupYou’ll see exactly what it’s about if you watch the GIF movie at the github repo.Can’t think of a better way to understand Python bytecode quickly than to play a little with thisComparing versions of CPython: If you have several versions of Python installed on your machine (let's say in different conda environments), you can use the extension to check how the bytecode might differ.Nice visualization of different performance aspects of while vs. for at the endBrian: “How the Internet is made.”

#109 CPython byte code explorer

2018-12-1800:20:451

Sponsored by DigitalOcean: pythonbytes.fm/digitaloceanBrian #1: pyjanitor - for cleaning dataoriginally a port of an R package called janitor, now much more.“pyjanitor’s etymology has a two-fold relationship to “cleanliness”. Firstly, it’s about extending Pandas with convenient data cleaning routines. Secondly, it’s about providing a cleaner, method-chaining, verb-based API for common pandas routines.”functionality:Cleaning columns name (multi-indexes are possible!)Removing empty rows and columnsIdentifying duplicate entriesEncoding columns as categoricalSplitting your data into features and targets (for machine learning)Adding, removing, and renaming columnsCoalesce multiple columns into a single columnConvert excel date (serial format) into a Python datetime formatExpand a single column that has delimited, categorical values into dummy-encoded variablesThis pandas code: df = pd.DataFrame(...) # create a pandas DataFrame somehow. del df['column1'] # delete a column from the dataframe. df = df.dropna(subset=['column2', 'column3']) # drop rows that have empty values in column 2 and 3. df = df.rename({'column2': 'unicorns', 'column3': 'dragons'}) # rename column2 and column3 df['newcolumn'] = ['iterable', 'of', 'items'] # add a new column.- looks like this with pyjanitor: df = ( pd.DataFrame(...) .remove_columns(['column1']) .dropna(subset=['column2', 'column3']) .rename_column('column2', 'unicorns') .rename_column('column3', 'dragons') .add_column('newcolumn', ['iterable', 'of', 'items']) )Michael #2: What Does It Take To Be An Expert At Python?Presentation at PyData 2017 by James PowellCovers Python Data Model (dunder methods)Covers uses of MetaclassesAll done very smoothly as a series of demosPretty long and in depth, 1.5+ hoursBrian #3: Awesome Python Applicationspypi is a great place to find great packages you can use as examples for the packages you write. Where do you go for application examples? Well, now you can go to Awesome Python Applications.categories of applications included:internet, audio, video, graphics, games, productivity, organization, communication, education, science, CMS, ERP (enterprise resource planning), static site generators, and a whole slew of developer related applications.Mahmoud is happy to have help filling this out, so if you know of a great open source application written in Python, go ahead and contribute to this, or open an issue on this project.Michael #4: Django Core no moreWrite up by James BennettIf you’re not the sort of person who closely follows the internals of Django’s development, you might not know there’s a draft proposal to drastically change the project’s governance.What’s up: Django the open-source project is OK right now, but difficulty in recruiting and retaining enough active contributors.Some of the biggest open-source projects dodge this by having, effectively, corporate sponsorship of contributions.Django has become sort of a victim of its own success: the types of easy bugfixes and small features that often are the path to growing new committers have mostly been done already in Django.Not managed to bring in new committers at a sufficient rate to replace those who’ve become less active or even entirely inactive, and that’s not sustainable for much longer.Under-attracting women contributors tooGovernance: Some parallels to what the Python core devs are experiencing now. Project leads BDFLs stepped down.The proposal: what I’ve proposed is the dissolution of “Django core”, and the revocation of almost all commit bitsSeems extreme but they were working much more as a team with PRs, etc anyway.Breaks down the barrier to needing to be on the core team to suggest, change anything.Two roles would be formalized — Mergers and Releasers — who would, respectively, merge pull requests into Django, and package/publish releases. But rather than being all-powerful decision-makers, these would be bureaucratic rolesBrian #5: wemake django templatea cookie-cutter template for serious django projects with lots of fun goodies“This project is used to scaffold a django project structure. Just like django-admin.py startproject but better.”features:Always up-to-date with the help of [@dependabot](https://dependabot.com/)poetry for managing dependenciesmypy for optional static typingpytest for unit testingflake8 and wemake-python-styleguide for lintingpre-commit hooks for consistent developmentdocker for development, testing, and productionsphinx for documentationGitlab CI with full build, test, and deploy pipeline configured by defaultCaddy with https and http/2 turned on by defaultMichael #6: Django HunterTool designed to help identify incorrectly configured Django applications that are exposing sensitive information.Why? March 2018: 28,165 thousand django servers are exposed on the internet, many are showing secret API keys, database passwords, amazon AWS keys. Example: https://twitter.com/6IX7ine/status/978598496658960384Some complained this inferred Django was insecure and said it wasn’t. Others thought “There is a reasonable argument to be made that DEBUG should default to False.”One beginner, Peter, chimes in:I probably have one of them, among my early projects that are on heroku and public GitHub repos. I did accidentally expose my aws password this way and all hell broke loose.The problem is that as a beginner, it wasn't obvious to me how to separate development and production settings and keep production stuff out of my public repository.Extras:Michael: Thanks for having me on your show Brian: https://blog.michaelckennedy.net/2018/12/08/being-a-great-podcast-guest/Brian: open source extra:For Christmas, I want a dragon…pic.twitter.com/RmFAEgqpSr — Changelog (@changelog) Michael: Why did the multithreaded chicken cross the road?road the side get to the other of toto get the side to road the of otherthe side of to the to road other getto of the road to side other the get

#108 Spilled data? Call the PyJanitor

2018-12-1100:21:51

Sponsored by DigitalOcean: pythonbytes.fm/digitaloceanBrian #1: glom: restructuring data, the Python wayglom is a new approach to working with data in Python, featuring:Path-based access for nested structuredata\['a'\]['b']['c'] → glom(data, 'a.b.c')Declarative data transformation using lightweight, Pythonic specificationsglom(target, spec, **kwargs) with options such asa default value if value not foundallowed exceptionsReadable, meaningful error messages:PathAccessError: could not access 'c', part 2 of Path('a', 'b', 'c') is better thanTypeError: 'NoneType' object is not subscriptableBuilt-in data exploration and debugging featuresglom.Inspect(``**a*``, ***kw*``)The [**Inspect**](https://glom.readthedocs.io/en/latest/api.html#glom.Inspect) specifier type provides a way to get visibility into glom’s evaluation of a specification, enabling debugging of those tricky problems that may arise with unexpected data.Michael #2: Scientific GUI apps with TraitsUIvia Franklin VenturaThey support: PyQt, wxPython, PySide, PyQt5People should be aware of and when combined with Chaco (again from Enthought) the graphing and controlling capabilities really are amazing.Tutorial: Writing a graphical application for scientific programming using TraitsUI 6.0Really simple UI / API for mapping object(s) to GUIs and back.Brian #3: Pampy: The Pattern Matching for Python you always dreamed of“Pampy is pretty small (150 lines), reasonably fast, and often makes your code more readable and hence easier to reason about.”uses _ as the missing info in a patternsimple match signature of match(input, pattern, action)Examplesnested lists and tuples from pampy import match, _ x = [1, [2, 3], 4] match(x, [1, [_, 3], _], lambda a, b: [1, [a, 3], b]) # => [1, [2, 3], 4] - dicts: pet = { 'type': 'dog', 'details': { 'age': 3 } } match(pet, { 'details': { 'age': _ } }, lambda age: age) # => 3 match(pet, { _ : { 'age': _ } }, lambda a, b: (a, b)) # => ('details', 3)Michael #4: Google AI better than doctors at detecting breast cancerGoogle’s deep learning AI called LYNA able to correctly identify tumorous regions in lymph nodes 99 per cent of the time.We think of the impact of AI as killing 'low end' jobs [see poster], but these are "doctor" level positions.The presence or absence of these ‘nodal metastases’ influence a patient’s prognosis and treatment plan, so accurate and fast detection is important.In a second trial, six pathologists completed a diagnostic test with and without LYNA’s assistance. With LYNA’s help, the doctors found it ‘easier’ to detect small metastases, and on average the task took half as long.Brian #5: 2018 Advent of CodeAnother winter break activity people might enjoy is practicing with code challenges. AoC is a fun tradition.a calendar of small programming puzzles for a variety of skill sets and skill levels that can be solved in any programming language you like.don't need a computer science background to participatedon’t need a fancy computer; every problem has a solution that completes in at most 15 seconds on ten-year-old hardware.There’s a leaderboard, so you can compete if you want. Or just have fun.Past years available, back to 2015.Some extra tools and info: awesome-advent-of-codeMichael #6: Red Hat Linux 8.0 Beta released, now (finally) updated to use Python 3.6 as default instead of 2.7First of all, my favorite comment was a correction to the title: legacy python *“Python 3.6 is the default Python implementation in RHEL 8; limited support for Python 2.7 is provided. No version of Python is installed by default.“Red Hat Enterprise Linux 8 is distributed with Python 3.6. The package is not installed by default. To install Python 3.6, use the yum install python3 command.Python 2.7 is available in the python2 package. However, Python 2 will have a shorter life cycle and its aim is to facilitate smoother transition to Python 3 for customers.Neither the default python package nor the unversioned /usr/bin/python executable is distributed with RHEL 8. Customers are advised to use python3 or python2 directly. Alternatively, administrators can configure the unversioned python command using the alternatives command. Python scripts must specify major version in hashbangs at RPM build timeIn RHEL 8, executable Python scripts are expected to use hashbangs (shebangs) specifying explicitly at least the major Python version. Extras:Michael: We were featured on TechMeme Long Ride Home podcast. Check out their podcast here. Thank you to Brian McCullough, the host of the show. I just learned about their show through this exchange but can easily see myself listening from time to time. It’s like Python Bytes, but for the wider tech world and less developer focused but still solid tech foundations.Brian: First story was about glom. I had heard of glom before, but got excited after interviewing Mahmoud for T&C 55, where we discussed the difficulty in testing if you use glom or DSLs in general. A twitter exchange and GH issue followed the episode, with Anthony Shaw. At one point, Ant shared this great joke from Brenan Kellar:A QA engineer walks into a bar. Orders a beer. Orders 0 beers. Orders 99999999999 beers. Orders a lizard. Orders -1 beers. Orders a ueicbksjdhd. First real customer walks in and asks where the bathroom is. The bar bursts into flames, killing everyone.— Brenan Keller (@brenankeller) November 30, 2018

#107 Restructuring and searching data, the Python way

2018-12-0700:22:50

Sponsored by DigitalOcean: pythonbytes.fm/digitaloceanBrian #1: Dependency Management through a DevOps LensPython Application Dependency Management in 2018 - HynekAn opinionated comparison of one use case and pipenv, poetry, pip-tools“We have more ways to manage dependencies in Python applications than ever. But how do they fare in production? Unfortunately this topic turned out to be quite polarizing and was at the center of a lot of heated debates. This is my attempt at an opinionated review through a DevOps lens.”Best disclaimer in a blog article ever:“DISCLAIMER: The following technical opinions are mine alone and if you use them as a weapon to attack people who try to improve the packaging situation you’re objectively a bad person. Please be nice.”Requirements: Solution needs to meet the following features:Allow me specify my immediate dependencies (e.g. Django),resolve the dependency tree and lock all of them with their versions and ideally hashes (more on hashes),integrate somehow with tox so I can run my tests,and finally allow me to install a project with all its locked dependencies into a virtual environment of my choosing.Seem like reasonable wishes. So far, none of the solutions work perfectly.A good example of pointing out tooling issues with his use case while being respectful of the people involved in creating other tools.Michael #2: Plugins made simple with pluginlibmakes creating plugins for Python very simpleit relies on metaclasses, but the average programmer can easily get lost dealing with metaclassesMain Features:Plugins are validated when they are loaded (instead of when they are used)Plugins can be loaded through different mechanisms (modules, filesystem paths, entry points)Multiple versions of the same plugin are supported (The newest one is used by default)Plugins can be blacklisted by type, name, or versionMultiple plugin groups are supported so one program can use multiple sets of plugins that won't conflictPlugins support conditional loading (examples: os, version, installed software, etc)Once loaded, plugins can be accessed through dictionary or dot notationBrian #3: How to Test Your Django App with Selenium and pytestBob Belderbos“In this article I will show you how to test a Django app with pytest and Selenium. We will test our CodeChalleng.es platform comparing the logged out homepage vs the logged in dashboard. We will navigate the DOM matching elements and more.”Michael #4: Fluent collection APIs (flupy and asq)flupy implements a fluent interface for chaining multiple method calls as a single python expression. All flupy methods return generators and are evaluated lazily in depth-first order. This allows flupy expressions to transform arbitrary size data in extremely limited memory.Example: pipeline = flu(count()).map(lambda x: x**2) \ .filter(lambda x: x % 517 == 0) \ .chunk(5) \ .take(3) for item in pipeline: print(item)The CLI in particular has been great for our data science team. Not everyone is super comfortable with linux-fu so having a cross-platform way to leverage python knowledge on the shell has been an easy win.Also if you are LINQ inclined: https://github.com/sixty-north/asqasq is simple implementation of a LINQ-inspired API for Python which operates over Python iterables, including a parallel version implemented in terms of the Python standard library multiprocessing module. # ASQ >>> from asq import query >>> words = ["zero", "one", "two", "three", "four", "five", "six", "seven", "eight", "nine", "ten"] >>> query(words).order_by(len).then_by().take(5).select(str.upper).to_list() ['ONE', 'SIX', 'TEN', 'TWO', 'FIVE']Brian #5: Guido blogging againWhat to do with your computer science careerAnswering “A question about whether to choose a 9-5 job or be an entrepreneur”entrepreneurship isn’t for everyoneworking for someone else can be very rewarding shoot for “better than an entry-level web development job”And “A question about whether AI would make human software developers redundant (not about what I think of the field of AI as a career choice)”AI is about automating tasks that can be boringSoftware Engineering is never boring.Michael #6: Web apps in pure Python apps with AnvilDesign with our visual designerBuild with nothing but PythonPublish Instant hosting in the cloud or on-sitePaid product but has a free versionCovered on Talk Python 138Extras:Second Printing (P2) of “Python Testing with pytest”

#106 Fluent query APIs on Python collections

2018-12-0100:26:21

Sponsored by DigitalOcean: pythonbytes.fm/digitaloceanBrian #1: Colorizing and Restoring Old Images with Deep LearningText interview by Charlie Harrington of Jason Antic, developer of DeOldifyA whole bunch of machine learning buzzwords that I don’t understand in the slightest combine to make a really cool to to make B&W photos look freaking amazing.“This is a deep learning based model. More specifically, what I've done is combined the following approaches:Self-Attention Generative Adversarial NetworkTraining structure inspired by (but not the same as) Progressive Growing of GANs.Two Time-Scale Update Rule.Generator Loss is two parts: One is a basic Perceptual Loss (or Feature Loss) based on VGG16. The second is the loss score from the critic.”Michael #2: PlatformIO IDE for VSCodevia Jason PecorPlatformIO is an open source ecosystem for IoT developmentCross-platform IDE and unified debugger. Remote unit testing and firmware updatesBuilt on Visual Studio Code which has a nice extension for PythonPlatformIO, combined with the features of VSCode provides some great improvements for project development over the standard Arduino IDE for Arduino-compatible microcontroller based solutions.Some of these features are paid, but it’s a reasonable priceWith Python becoming more popular for microcontroller design, as well, this might be a very nice option for designers.And for Jason’s, specifically, it provides a single environment that can eventually be configured to handle doing the embedded code design, associated Python supporting tools mods, and HDL development. The PlatformIO Core written in Python. Python 2.7 (hiss…)Jason’s test drive video from Tuesday: Test Driving PlatformIO IDE for VSCodeBrian #3: Python Data Visualization 2018: Why So Many Libraries?Nice overview of visualization landscape, by Anaconda teamDifferentiating factors, API types, and emerging trendsRelated: Drawing Data with Flask and matplotlibFinally! A really simple example app in Flask that shows how to both generate and display matplotlib plots.I was looking for something like this about a year ago and didn’t find it.Michael #4: coder.com - VS Code in the cloudFull Visual Studio Code, but in your browserCode in the browserAccess up to 96 coresVS Code + extensions, so all the languages and featuresCollaborate in real time, think google docsAccess linux from any OSNote: They sponsored an episode of Talk Python To Me, but this is not an ad here...Brian #5: By Welcoming Women, Python’s Founder Overcomes Closed Minds In Open SourceForbes’s article about Guido and the Python community actively working to get more women involved in core development as well as speaking at conferences.Good lessons for other projects, and work teams, about how you cannot just passively “let people join”, you need to work to make it happen.Michael #6: Machine Learning BasicsFrom Anna-Lena PopkesPlain python implementations of basic machine learning algorithmsRepository contains implementations of basic machine learning algorithms in plain Python (modern Python, yay!)All algorithms are implemented from scratch without using additional machine learning libraries. Goal is to provide a basic understanding of the algorithms and their underlying structure, not to provide the most efficient implementations.Most of the algorithmsLinear RegressionLogistic RegressionPerceptronk-nearest-neighbork-Means clusteringSimple neural network with one hidden layerMultinomial Logistic RegressionDecision tree for classificationDecision tree for regressionAnna-Lena was on Talk Python on 186: http://talkpython.fm/186Extras:Michael: PSF Fellow Nominations are openMichael: Shiboken has no meaningBrian: Python 3.7 runtime now available in AWS Lambda

#105 Colorizing and Restoring Old Images with Deep Learning

2018-11-2300:24:152

Python Bytes 104Sponsored by DigitalOcean: pythonbytes.fm/digitaloceanMichael #0.1: Chapters and play atChapters are now in the mp3 filePlay at button on the website (doesn’t work on iOS unless you click the play to start it)Michael #0.2: Become a friend of the showhttps://pythonbytes.fm/friends-of-the-showOr just click “friends of the show” in the navbarBrian #1: wily: A Python application for tracking, reporting on timing and complexity in tests and applications.Anthony Shaw (aka “Friend of the Show”, aka “Ant”)(if listing 2 “aliases, do you just put one “aka” or one per alias?)I should cover this on Test & Code for the content of the package. But it’s the actual packaging that I want to talk about today.Wily is a code base that can be used as an example of embracing pyproject.toml (pyproject.toml discussed on PB 100 and T&C 52)A real nice clean project using newer packaging tools that also has some frequently used bells and whistlesNO setup.py filewily’s pyproject.toml includesflit packaging, metadata, scripts tox configurationblack configurationproject also has testing done on TravisCIrst based docs and readthedocs updatescode coverageblack pre-commit for wilypre-commit hook for your project to run wilyCONTRIBUTING.md that includes code of conductHISTORY.md with a nice formattests using pytestMichael #2: Latest VS Code has Juypter supportIn this release, closed a total of 49 issues, including:Jupyter support: import notebooks and run code cells in a Python Interactive windowUse new virtual environments without having to restart Visual Studio CodeCode completions in the debug console windowImproved completions in language server, including recognition of namedtuple, and generic typesThe extension now contains new editor-centric interactive programming capabilities built on top of Jupyter.have Jupyter installed in your environment (e.g. set your environment to Anaconda) and type #%% into a Python file to define a Cell. You will notice a “Run Cell” code lens will appear above the #%% line:Cells in the Jupyter Notebook will be converted to cells in a Python file by adding #%% lines. You can run the cells to view the notebook output in Visual Studio code, including plotsBrian #3: API Evolution the Right WayA. Jesse Jiryu Davisadding featuresremoving featuresadding parameterschanging behaviorMichael #4: PySimpleGUI now on QtProject by Mike BCovered back on https://pythonbytes.fm/episodes/show/90/a-django-async-roadmapSimple declarative UI “builder”Looking to take your Python code from the world of command lines and into the convenience of a GUI?Have a Raspberry Pi with a touchscreen that's going to waste because you don't have the time to learn a GUI SDK?Look no further, you've found your GUI package.Now supports QtModern Python onlyMore frameworks likely comingBrian #5: Comparison of the 7 governance PEPsStarted by Victor StinnerThe different PEPs are compared by:hierarchynumber of people involvedrequirements for candidates to be considered for certain positionselections: who votes, and howterm limitsno confidence voteteams/expertsPEP processcore dev promotion and ejectionhow governance will be updatedcode of conductPEP 8000, Python Language Governance Proposal Overview:PEP 8010 - The Technical Leader Governance Modelcontinue status quo (ish)PEP 8011 - Python Governance Model Lead by Trio of Pythonistaslike status quo but with 3 co-leadersPEP 8012 - The Community Governance Modelno central authorityPEP 8013 - The External Governance Modelnon-core oversightPEP 8014 - The Commons Governance Modelcore oversightPEP 8015 - Organization of the Python communitypush most decision-making to teamsPEP 8016 - The Steering Council Modelbootstrap iterating on governanceMichael #6: Shiboken (from Qt for Python project)From PySide2 (AKA Qt for Python) projectGenerate Python bindings from arbitrary C/C++ codeHas a Typesystem (based on XML) which allows modifying the obtained information to properly represent and manipulate the C++ classes into the Python World.Can remove and add methods to certain classes, and even modify the arguments of each function, which is really necessary when both C++ and Python collide and a decision needs to be made to properly handle the data structures or types.Qt for Python: under the hoodWrite your own Python bindingsOther options include:CFFI (example dbader.org)Cython (example: via shamir.stav)Extras: Michael: Mission Python: Code a Space Adventure Game! bookMichael: PyCon tickets are on saleMichael: PyCascade tickets are on sale

#104 API Evolution the Right Way

2018-11-1700:30:07

Comments (7)

Antonio Andrade

I think you missed to highlight all the nice work of realphlython and your podcasts, these are key stuffs for Python in 2018!

Dec 27th

Vignesh Anand Krishnan

The jokes are good but let brian do it. 😂

Dec 13th

GreatBahram

Congrats Python Bytes. This episode was really great 😎

Oct 27th

pyguy

Joel Grus talk can be found here: https://youtu.be/7jiPeIFXb6U

Oct 8th

víbora means in Spanish: snake. umm, just thinking about Phyton

Aug 4th

It's intetesting the title is flask but you guys spoke more about Django? kidding? hahaha please dont mess with us《Mico framework fans Thanks

Jun 28th

nice, another super good Python postcast

May 20th