Joe Bostian and Chad McIntyre - Python and Open Source Update
Description
Returning to the show are Python and Open Source Experts Joe Boston and Chad McIntyre. Keeping up to date on how Python on IBM Z is advancing and improving is important, and we've got an episode chock full of info. From packages to ports, AI to Flask, it's time to refresh that knowledge.
Auto-Generated Transcript (may not be 100% accurate)
Frank Welcome to the Terminal Talk podcast on mainframe and mainframe related topics.
Jeff Frank Hi, Frank, It's really nice to meet you. I'm Jeff.
Frank And with us today, we have two very important guests. We have Chad MacIntyre, who is the lead for Python and U.S., and Joe Basson, who is the lead for Open Source NZ. And so both of these guys have been on the podcast before. You probably remember the great episodes that they did. Mm hmm. We wanted to kind of get together and hear about what's been happening since the last time they went on. Yeah, some good.
Jeff Sounds good to me.
Frank You feel so engaged as the important thing.
Jeff At least I'm not drawing this time.
Frank Yeah. So last time Jeff actually created a doodle of me that was maybe not the most complimentary. So it really felt like it was. It was with me, so.
Joe Yeah, I saw that, though. It was probably about the best Jeff could do, though, don't you think? Well, I.
Frank Thought about the.
Jeff Worst and not the.
Frank Worst. I'm going to make it my I'm my profile picture for the company.
Jeff For those in the room.
Frank And that's why.
Jeff I sent it to you, Chad. So you can see it, too.
Frank Okay. So let's start I think we'll start with the Python stuff, because some of that was has really changed quite a bit since last time we talked and. So, Chad, you want to talk about, you know, your adventures since the last time we talked to you?
Chad Certainly, Frank. So we've definitely been through a lot of growth and expansion, I would say, and exciting things going on in the Python universe. And just to kind of bring everybody up to speed in terms of some of the important things that are happening there. You know, Joe's on the call today for I'd say probably one of the things that's been troublesome. The most troublesome for everybody on the python front and those consuming packages. So you know, we'll get into that as we go down the line. So there's good news there. And the other one that everybody, I think from day one of release was asking was the question, you know, can Python run on Zips? And then the answer to that question was certainly no for quite a long time and has now changed within the last couple of months, too. Yes. So that's a very exciting development. And we're seeing a lot of people really interested in adopting Python workflows and running those things on the Atlas machines. Beyond that, you know, we've been doing our work to keep up with the community. So, you know, we're it's it's core to our mission to make sure that we've got the latest and greatest python supported on the platform. But not only that that we're making it take advantage of those special things that Z has to offer. Right. So in the latest release, 311, we've added support right within Python to take advantage of the enterprise data compression accelerators, accelerators. And so this is a nice feature because you get it for free, right? You don't have to do anything to kind of code to it or take advantage of it. If you've got them in there enabled, it will just work with Python. So lots of great work and lots of great developments on the Python side, and that's just a teaser from there.
Frank Well, in Python 311, across the board was a better performing python. Are we seeing that on the U.S.?
Chad Absolutely, yeah. So there's been some terrific work done in the community to rework some of the guts of the python the python implementation specifically to perform better. And so, you know, this was a concerted effort by the community to take a look at, you know, what are Python workflows, how do they run and why are we here, how can we do better at running them? And so there was a whole kind of re-organization and a breaking down of byte codes and stuff like that that happened in The Interpreter. And the question is, does that pan out into Z? And absolutely, it does pan out into Z. So this is kind of an across the board thing in terms of performance benefits. So I don't want to go into quoting numbers or things of that nature. But on the Net Corp performance benchmarks, we're definitely seeing benefits that match what the community was seeing there.
Frank Yeah. So how long before Python takes over the Z OS world?
Chad Oh, can I defer that one to Joe?
Frank No.
Joe It already has.
Frank Happened.
Joe We just don't know yet.
Chad Exactly. I think it's it's one of those things where the roots are already there. It's just a matter of time for this time the show is.
Frank Well, and that's that to me, a big part of this. And I kind of want both of you guys will weigh in on this. But in my perspective, though, the python work that we you know, that you guys have done, along with things like Zillow or U.S. Open automation utilities, see, I got it all. You know, it really starts to democratize the platform in a way that really has never happened on the platform. People now have a lot more choice and they don't necessarily have to do things the way IBM tells them they should. And so I that's a yeah, I think it's a really important piece. Imagine it. My, my future is always managing the platform with Python instead of JCL. Right. And I'm oh, pause for some listeners to have, you know, a complete nervous breakdown.
Jeff That's obviously one of the big. Hearts of Python success on Z is going to come from packages being available for it because Python is a very extensible language. I'm curious what the process is to in identifying those important packages, because it's not just the big ones that everyone thinks about. It's these tiny little ones that you need that one weird bolt to make a project work. So how do you how do you make sure you have all those bolts in place?
Joe A lot of it's trial and error and inexperience. And one of the reasons that we have gotten to the point now where, you know, between Chad's team and our team here in Poughkeepsie, that we're at least reasonably competent. And what we do these days is that we've made all the mistakes before. Right. I mean, we we originally started diving into the python pool in 2016. We came out with some of our AI related stuff and we pretty much did all the wrong stuff, right. We tried to shoehorn we tried to shoehorn code into packaging mechanisms and delivery channels that didn't make sense. And so what happened is you would you would come out with a product or a function that did something very useful, very cool, but the code would just sit there and rot over time because you had no capability to keep up with the open source community.
Chad Right.
Joe And so there were so many technical issues that were in our way. And over the years we kept identifying what these problems were. And one of the very key ones right from the beginning was to settle on a standardized first class python that we could make part of our regular language environment on us and rely on it and lean on it as heavily as we do the C compiler or any other language on the platform. And once, once, once we put that that group together and Chad became the lead of that, that that's when things really started to take off. That was the major inhibitor that we really that we really faced. And then from then on, it's like, okay, now we can tackle all the other kinds of things primarily how do you stay current? And even just two years ago, currency was all about how do we put the latest function out there, as everyone knows, right? More and more security has become the focus of the open source community. And now currency and security are tightly coupled with one another. So. So as we look forward to how do we best manage the open source community? Number one, we've got our good, solid current language to work from, which solves all a whole host of problems. And from my point of view, the next question is how do we keep up with the open source community when we've got hundreds of packages that we need to put out there? Right. And so you ask the question, how do we make sure that we've got the one little utility that is the, you know, the bolt that holds everything together is even.
Frank Yes.
Joe Yes. Something something profound like that, you know. And in the answers, we just spend time installing things and look for stuff that falls out. Right. And it's not actually even that hard. Python is such a flexible environment, and the package management system that's built into this ecosystem is so flexible and so useful that you go, Well, I've got this set of say, 15 different things that I want to do, but me install 15 and then look at everything that gets dragged in and maybe do some analysis on the dependency trees. And that's everything that I need to know. And you're not going to you're not going to get bitten by that. One thing that's missing, because in order for you to even have installed it in the first place, all these pieces had to be okay.
Jeff So from a currency perspective, how much of that falls on staying up to date? How much of that falls on the developer themselves versus like the mechanism there they're using?
Joe Yeah. The open source community, I saw a statistic that we put in several of the presentations that we've done, which I believe is really true when you when you look at all of the vulnerabilities out there that are exploited in the wild, they're 97% of all of them already have fixes waiting are already available