00:00:03

Good morning, everyone. My name is George D can-do. I'm the CTL of the mainframe division in Broadcom. And it's a pleasure to be able to host for you today. A panel discussion on extending mainframe, extending dev ops to the mainframe. So we were, um, planning on having three panelists here today, but we've had an emergency come up. So we're only going to have two. So let me introduce to you, um, these two panelists. Oh. Um, as I said, my name is George. We've got here, Matt and Vencat would you like to introduce yourself map? Sure.

00:00:36

Uh, I'm Matt Holmstrom. I work for IBM. I'm based out of Raleigh, North Carolina. And my day job is working on how to manage the platform, better monitoring system automation, um, incorporating things like Ansible, um, into our runtimes. Um, as well as I spend a good portion of my time working on projects, Zoe, which is an open source project, helping them modernize the mainframe.

00:01:03

Hello everyone. My name is . I'm the solution architect for the DevOps set up a portfolio of products in Broadcom. Uh, I'm based out of Plano, Texas, and, um, essentially the goal or my job day job is to make sure, uh, you know, the topic that we're talking about today, right. Extending DevOps to the mainframe and how do to enable that.

00:01:29

Okay, great. So the, the, um, format we're going to use here is I'm going to pose questions to the panel and you will, um, answer and just interrupt with, uh, any insights that you have any, any time we're going through. So the first question is just about mainframe and dev ops. What, what's the main difference? The difficulty in integrating dev ops with mainframe? Why, why is it so hard? Why is it the final frontier? And, um, why don't we start off with you, Matt?

00:02:00

Um, well, the, one of the big challenges with the mainframe is it's got 50 years of investment in it, which means customers have been building, you know, business logic. Um, they built processes, they have culture, and they've got, um, a substantial, I'll say velocity of what they've been doing for quite some time. And so, um, when you bring somebody to the platform, oftentimes they feel like they have to go through a whole set of different training. You know, we're going to use something like eyes. How many of you folks are mainframe people? Okay. Ooh. So when I say like 30 to 70 you'll know exactly what I mean. Right. Um, and so it's not exactly the, the, um, best IDE for developing complex programs. And so there are big challenges. How do we take the investment that we have in the platform, the existing processes, et cetera, and then how do we enhance those and make them so they are just as familiar from a process standpoint to new developers, um, coming from say, cloud native.

00:03:06

And I'll, I'll just add one, one little bit. That's may, um, uh, as an anecdotal data point, um, in the development that we have for systems, meaning the things we're building with, um, you know, our monitoring solutions like omega Mon or net view and things of that nature. Um, I have a number of new hires. And so these are our, um, I'd say kids, but that might be considered derogatory, some really bright folks coming out of college that never worked on a mainframe before. And what they find really compelling is leveraging rest API APIs, angular, and new technologies so that they can build the kind of the next generation of system functions. And we want them to be up to speed as quickly as possible. So making sure that we have the right platforms, the right rest API end points, et cetera, makes them productive on a platform and continues that investment.

00:04:02

So, you know, I think kind of build upon what Matt was saying, right? Why, why are, why is the mainframe considered as the final frontier? And I think the key thing that Matt hit on is, you know, the history and the heritage, right, as you heard this morning, during the general session, there is a 50 or 60 years of, you know, processes, tooling that is already in place and customizations that organizations have put in place. And when you take a step back and look at dev ops in general, right? One of the key words that you heard this morning repeatedly is the word culture. There were people, you know, starting with that. And, and the whole idea of, of, you know, dev ops is going through that cultural transformation. And when it comes to the mainframe that cultural transformation, eh, you know, the, the complexity of it is a notch higher compared to any other, you know, distributed systems that you would see.

00:05:02

And the reason is because of the history, right? And internet, you know, traditionally mainframe has grown up to be change, averse, risk averse, right. You know, it's traditionally been siloed and it operating its own environment and, and, and that's not good or bad, right. That's just how things have evolved or over the years. And, and now in this current digital age, when we're actually talking about, you know, rest API APIs, you know, mobile to mainframe bringing mainframe into the fold of the enterprise and making mainframe be the same as any other platform and driving that cultural transformation is key, right? That's, that's the primary aspect of it. And when it, with, with culture, we talk about people, right. And when it, when we talk about people, the, the, the whole idea of skills actually come up, you know, the, the willingness are to adopt new technology is I think the key aspect of it as well, right.

00:05:59

When, when you're bringing in. So Matt referred to the kids coming to the platform, the fresh college grads who are actually coming to the platform. And, and, and that's, that's inevitable, right? There's a lot of COBOL code out there. There are a lot of mainframe applications that are pretty much running the, the organizations and the whole world. And, and you need folks to actually manage, maintain these applications. So how do you actually bring those people in? Right. And part of it is actually, um, you know, changing that culture and having more open-mindedness to embrace new technology, embrace new set of tools that are actually coming out. And keeping up with that I think is, is, is, is the, the key aspect of it.

00:06:42

Okay. So it sounds like it's a combination of process tools, culture, all of this plays into making mainframe a difficult environment for adopting dev ops. Yep. Okay. So we're going to switch gears here. And so those are some of the problems, but what are some of the techniques that we can use to incorporate modern DevOps practices and tooling into mainframes? So how can, how can we overcome these problems? And specifically we've got, um, moving around here has joined us. Would you like to, um, introduce yourself real briefly

00:07:17

Enact bank? We also use mainframe mainframe is the main, uh, they can't platform for us. So in order to be successful, uh, for customer satisfaction, uh, we need to improve ourselves, uh, in distributed side. And also on the mainframe side, uh, the technology is, uh, now emerged in mainframe helps us, uh, in that domain. Uh, so we are looking forward to modernize our application, our way of working now. So it includes, uh, version control systems and also pipelines management. Like we do in the distributed side, we can do, uh, the same, uh, uh, same technology. We can use the same tools, technologies, and also most importantly, uh, our developers can also, uh, feel the same, like, uh, in the distributed side. So, uh, I feel, uh, that, uh, it, it is very important to use those technologies, uh, uh, wisely. Uh, so, uh, I think, uh, we will be more productive with, uh, using such tools.

00:08:40

So Matt, you mentioned Zoe and your introduction. So I think Zoe is an important part of trying to modernize the mainframe that's part of, part of, one of its goals. Can you describe a little bit about Zoe for this audience and how it, how it

00:08:54

Will? Yeah, sure. Um, so Zoe, if you're not familiar with it, um, was a, um, an interesting, uh, transformation that occurred about a year and a half ago about a year, year and a half ago. And, um, what happened is IBM. Um, we're looking in, we're investigating how we can modernize the mainframe and including new rest API APIs, et cetera. Um, and, uh, rocket software approached us and said, Hey, we built a new desktop interface. So basically we can put a desktop motif on the mainframe, um, because we, we think it would be more engaging because people know where to go on a desktop, right. I can go, oh, lower left start. Oh, here's all the things that are there. So it was more of a guided activity than being put in to say, like TSO. But we looked at that and said, well, doesn't really make sense to do a proprietary solution.

00:09:46

And we need, we need more than that. And we ended up having this conversation with Broadcom and Broadcom. It was in exactly the same place we were investigating. How do I improve access to the mainframe, et cetera. And they, um, had created a technology called Brightside, which we now refer to as the Zoe CLI. And so if you're familiar with the story of stone soup, we went from a boiling water with stones in it to some vegetables, chicken, and other things that made it actually, um, very flavorful. Um, and what we did was we practiced the three of us coming together who compete in the market, I think pretty aggressively, but we knew we had to cooperate with each other as well. So we wanted to open source the three technologies. And by combining the three together then, um, Broadcom brought an API mediation layer, which gave us a way of federating, disparate rest services on the platform.

00:10:43

Um, we've introduced single sign-on. And so Zoe has given us a way of cooperating and giving us the platform, I think, a better, uh, position to be, um, competing, um, for the, um, new work that's coming out from new applications and things of that nature as well. And so, um, we just passed the year and a half mark back in August. We announced it share. And we're in right now in the process of coming up with version two and, and from, for mainframe guys, um, we're releasing just about once a month with new features functions, capability, I'd say bug fixes, because that would imply we had bad code. So, um, but bug fixes as well. And the point is we've got pretty good velocity. In fact, um, uh, we've also seen other vendors, um, Syncsort recently joined the open mainframe project, which is where Zoe is hosted. And so we have other vendors now coming alongside as well. And so we've got a really great collaboration point where we can make the platform better. And as the phrase, a rising tide lifts all boats. And that's what our strategy is with Zoe.

00:11:51

So just, just curiosity question, how many people in this room have heard Zoe or at least the other key word? I don't know if you guys caught what Matt said, right. It is an open source project hosted under open mainframe, right? So again, going back to the earlier discussion that we were having, why is mainframe, you know, the, the, the final frontier, right. We were talking about cultural change. Now, think about this open source software on mainframe, right. An API. So, so these are some of the things that were, you know, kind of foreign words, uh, you know, about six to eight, 10 years ago, but now these are pretty much the standard. This is the, the, the new way of actually working with the mainframe and the key thing there is as part of that evolution, right. You know, mainframe is becoming like any other platform, thanks for API and CLS. And the beauty of the CLI is, is scripting, right? Which is the whole purpose of dabs. Right. You know, the, the key thing that we talked about, the blocking inhibitor, one of the things that is lacking on the mainframe is automation. When you have CLI and you have the scripting abilities that kind of opens up the doors to do more automation. And that is the key part that, that Zoe is actually bringing to the table and as become an enabler for DevOps adoption on the mainframe.

00:13:23

Do you have any experience with those?

00:13:25

Uh, yeah, actually we plan to use the same tools, as I said, uh, in the distributed side, like, SonarQube, it's a good tool, for example, in order to use such a product, uh, we need to have sources outside mainframe for that purposes we can use, uh, Zoe, if you have, uh, called, uh, also synchronized also in the Git repository, for example, we can easily use, uh, sonar, lint and SonarQube. And also, eh, another aspect is we can also use, uh, actives ID, uh, another, uh, vs code ID, like we use also in the distributed side. So some kind of sexualization

00:14:15

Enables you to use

00:14:16

ID of your choice standards, operation lecturer.

00:14:21

Okay, great. So

00:14:23

It was one way, yes. I was just going to say to the IDE point, everything doesn't run on the mainframe, um, the CLI doesn't run on the mainframe and we also made, um, vs code extensions available. That'll plug into vs code that will integrate with the mainframe as well. So it's, it's, um, we have, uh, lots of distribution channels for improving, um, the velocity for developing and interacting with the mainframe. Sorry.

00:14:49

So clearly Zoe's a key piece of opening up the mainframe to scripting opening up a web desktop. That's easier for, let's say, younger developers to use the kids, and then w so really clear, and we're going to have a deep dive on Zoe. We're going to have another panel discussion on next half hour. So if you're interested in deep diving in Zoe a bit, come join us for that. So still talking about opening up dev ops for mainframe, um, command line interface is always one area. Any, any other areas that we, that people should be looking at to try to move into dev ops on the mainframe?

00:15:33

So I think that the key thing to keep in mind when we're talking about DevOps on the mainframe, and this goes back to, you know, what you guys heard during the general session today, right? Everyone who talked about dev ops independent of whether it is my mainframe involved or not, right. The Jew, you know, the key thing to remember is basically it's a journey, right? Dev ops won't end, it's about continuous improvement and you actually continually improve what you're doing today. So in terms of, you know, talking about what to do, where to begin, which is the common set of questions that pretty much everybody has in their head, right? Where do I start? Um, the, the key thing is to, you know, to figure out where you are today, right? So from a business perspective, what are your key, uh, you know, business challenges or business goals that you want to achieve, and then, you know, picking, starting small, trying to figure out what are your, what is the current state of affairs?

00:16:33

What are the key bottlenecks? And then having a priority of those bottlenecks and say, okay, here are the things that I'm going to address that is going to give me the biggest bang for the buck, right? And then, you know, going back to that, that, you know, crawl, walk, run, and then come back to crawl because the key aspect of dev ops is actually learning from what you've already done, because it's about continuous improvement. So in terms of, you know, the practices and enabling that culture change starting small is, is the most important thing. And remembering the fact that it's not going to S you know, it's not a flip of a switch, right? Things are not going to change overnight. It's a journey. And it's about continuous improvement. And having an understanding of that as part of, as an entire organization, that's part of the cultural change that you need to, uh, to, to, to have.

00:17:26

And then there are, you know, enablers like Zoe, which are actually gonna help you through the journey, because if you look back right in as part of this, my own experience, as part of this DevOps journey, you know, four or five years back, we've been talking about dev ops on the mainframe for a very long time, right. And it's like, okay, here are the things that you need to do. But now there are concrete set of tools, practices, examples that are open and available for everybody to, to enable that journey. So there are these enablers like Zoe, which will help you go faster in that journey. Right. And, you know, it's one thing to say, Zoe, right. Um, is, is, is basically the enabler. Um, and the other part of it is, you know, walking the talk, right? What, so what we are doing within Broadcom, there are several, you know, traditional mainframe teams who have adopted Zoe to just change the way they actually work.

00:18:32

Right. We be at, you know, automating their bills are creation of automated test cases, running their existing test cases of the platform, right. Improving their, their build time. Um, and you know, one classic example that we heard they had recently is there is a classic, you know, traditional mainframe product. The developers were spending 40% of their time coding. And 60% of the time in actually doing the bills, setting up the environment and writing the test cases, running the test cases and things like that. So 40% actual coding, 60% on the real work. I mean, that does set of work, right. And with adoption of Zoe and the automating of their tests and the innate, the provisioning of their environments, it went to about 85, 90% of the time. Now they're able to, their teams are actually able to spend on coding. So again, what we're the, the they accomplished is they eliminated the waste through the automation process and elimination of waste is a key principle of DevOps. So the key thing, again, just to recap is start small. Remember that it's a journey and remember that there is a community, right. And set of enablers that are there to actually help you go through that journey.

00:19:56

So do you have, do you have any experience in how you got going in and doing dev ops on mainframe?

00:20:01

Uh, actually we are trying to improve the customer experiences. So, uh, it doesn't matter if you are on the distributed side or on the mainframe side. Uh, you, you need to, uh, your customer actually doesn't care about your technology, but, uh, what, what is important for them? What is important for a customer should be actually improved? So agile and DevOps methodology is now, uh, compliments each other in, in this work. So, uh,

00:20:40

You've integrated your main, your main frame and your, your distributed dev ops processes together.

00:20:45

Yes. Uh, I mean, it shouldn't be any different, it should be , uh, efficient. So, uh, your developers, uh, working in distributed side and on the, on mainframe side should actually leave the same experience towards the, uh, customer satisfaction. Uh, so we, we think that we need such tools in order to, uh, improve their daily life and also, uh, the way they work, uh, towards customers that's affection.

00:21:23

That makes a lot of sense. Yeah.

00:21:26

And, and the other thing that I'd like to add in here is, you know, there was one point that Murra made about, you know, using SonarQube as an example, right? One of the things that organizations are actually looking at is standardization of tools across the board, right? You don't want to use a different set of tools for distributed application development versus a different set of tools for mainframe. Because again, you don't want mainframe to be different than any other thing you don't want mainframe to be siloed. So the, the, the tools like Zoe, right, enable you to, to, and, and as proven by, uh, you know, more on steam is they're able to take mainframe source code right. Of the platform and use tools like SonarQube solar lent, which are the ones that are actually being used to do the code analysis and code reviews on your Java, or, you know, other distributed languages. Now it's possible with mainframe. So again, the key thing, again, due to go back, well, the word that stuck in my head this morning during the general session was heritage. Don't be a legacy organization, be a heritage, you know, leverage what you already have. And there are tools, techniques, and methodologies that are actually available to, to tie in your heritage, to the modern world. And that's, that's the key, uh, message, you know, from where the current things stand from a, from a DevOps on the mainframe world.

00:22:51

Yep. Um, so as we're having the conversation, um, I think the real key point it's already been said, I'll stress. It is, there's not like a thing that is the answer, right? If I'm, I'm driving a, uh, Jenkins pipeline to do build, um, Zoe brings, um, aggregation or Federation of rest API APIs. We have CLS, we have lots of different tools of dependency based build for how we can integrate with those tools. Um, but one of the things we really didn't talk about and, um, we're, uh, at IBM we're, we're moving ahead, very aggressively at how do we leverage, um, automation like Ansible so that we're not just talking about, um, doing a COBOL build and I'm pushing it and it's getting compiled and loaded, but how do I change that, that whole process into something that's a hundred percent so that, um, it's not just the, the building and the promotion of the artifacts, because the, the ability to really get the DevOps power is not just in development, but it's being able to take the packages that you're building or the, the, um, iterations that you're releasing and push them through and to test from test into QA, into pre-prod, into prod.

00:24:08

And so you have to see in that pipeline, what are the collection of tools that you're going to need? Zoe is certainly a key resource, but there's many others as well.

00:24:19

Yeah. So I'll, I'll mention as well, coming in as a CTO of a Broadcom, it's been, um, we have the same challenge ourselves trying to modernize and our, um, processes and our, our, uh, different, um, development teams. And I was looking, we have over 250 million lines of COBOL code that we maintained, uh, sorry. I've, um, assembler code that we maintain just as, um, as part of the products that our customers use, a lot of assembler code. But if you look at trying to improve dev ops across our teams, it's not a one size fits all story. You have, it has to come from the bottom and the program we've implemented sort of enables that, so that we call it engineering excellence, but it's pretty much every agile iteration, the teams have to improve their, do something to improve their process the same way they're doing a feature. And, and they identify what that's going to be, and it may be different for each team. So they measure before they measure, after they see the improvement. So every iteration we're spending time improving and moving towards a more agile dev ops environment. So I highly recommend, you know, these edicts from the top that everyone's going to do dev ops the same way. My experience with that is it is it doesn't work. I don't know if you guys are nodding your head to experienced a similar,

00:25:43

Oh, he completely agree. I mean, there is, there's no one size fits all. There is no silver bullet when it comes to dev ops and that's independent of, you know, whether it's distributed or mainframe, right. Every organization is different. Every, you know, organizations priorities are actually different. So, you know, as they're saying, analyzing what you, where you are, which is exactly what George was referring to is the engineering excellence that we did is we had every team go read a review, right? What's your bottleneck, what is that you want to improve on? And then, you know, set goals and make sure that they measure themselves and continually improve. So that's

00:26:18

The things that teams can improve themselves. They need help externally. And that's where we need to be there to get, get them that help 10 to enable them to make the changes that they really want to make. Right.

00:26:29

Can I ask, can I just ask you another qualifying question? How many people are in development only, or how many people do systems programming? So if you do development only regime. Okay. And are you on the systems or infrastructure side of the house? Okay. Couple of you. So one of the things we do at IBM, um, you know, design thinking, and one of, one of the, um, challenges is kind of feeling, figuring out an empathy for the other people in the organization, right? So one of the barriers or challenges that we have typically in many organizations is, um, uh, the developer's not the first person that gets called when something goes wrong. It's the operation staff. So the operations staff, like, ah, I'm going to control as much as this as I can. Cause I don't want to have to explain why we had an outage and that's not productive.

00:27:19

Right. So understanding, um, you know, what some of their constraints are, um, as well as helping them understand what yours are. I had my very first job, speaking of assembler, it was a batch assembler programmer back in 1852. And, um, I was writing a, um, a set of programs and I wasn't getting any love from the operations team. I think they hated me. And so I volunteered, I went down and I said, I will work in operations. Second shift from four to midnight for three months with no pay. I'll just do that in addition to my day job. And when I did that, all of a sudden, I, you know, mounting tapes and grabbing them from the library and seeing the production schedule and whatnot. And I went holy smokes. I was like totally making their lives miserable. So by actually entering their world, I was able to work a lot better with them. And then after I finished my three months, stint never waited for a tape Mount after that. So, um, just something to be aware, as you're trying to do this cross-organizational thing, you mentioned outside help, um, because the infrastructure teams really do want to change. Um, and they want to kind of unleash you guys. We have to figure out collectively, how do we change some of the contracts we have

00:28:37

And, you know, talking about outside help, right? And one of the reasons why we're going down the opensource route is essentially to create the community and you can do what Matt was saying, how it came about, you know, Broadcom, IBM, rocket coming together to create the community, to enable you guys and actually provide help you guys,

00:28:56

Chile, actually, we should say Phoenix software in there. They just, yes. So,

00:29:01

So we're out of time. Thank you guys very much for your time and the insights you've shared here. I'm hoping many of you will join us to do a deeper dive in Zoe over in Mon blond launch one, which is just on the hallway here and just a few minutes. Okay.

00:29:17

Thank you all.