00:00:02

Good afternoon, everyone. Welcome. Thank you all for, uh, sticking around and coming to this late day session. We're glad to have you here.

00:00:09

I'm David Kennedy, a solutions architect for Compuware.

00:00:13

And I'm David Rizzo, vice president of product Engineering for Compuware. And, uh, we're excited to be able to share with you a little bit about where Compuware has been, uh, over the last four or five years. If any of you were here, uh, two years ago, or actually it would've been in San Francisco two years ago. Uh, there was a session there that I did to talk about our journey, and we're gonna give the extension of that, what we've done with DevOps. And today we're specifically gonna show you how we measure DevOps, how we use real data to show what we're doing with, uh, DevOps and how our engineering organization works. And we're gonna, uh, fate the gods of live demo. And we're gonna show you some live, uh, demo as well. So we'll see how that all goes. We'll keep a good thought. Roll the dice, right?

00:00:56

If that goes well, maybe we'll go downstairs. Alright, so Compuware <laugh> is a, uh, mainframe software vendor who has a mainframe. Awesome. Alright, that's great. So we are a mainframe software vendor. We provide developer productivity tools and operations productivity tools to help developers and people and operations develop on mainframes. We've been doing that for over 45 years, and, um, we had the opportunity five years ago, we became a, uh, private company. A lot of you, hopefully you were in the opening session this morning. Our CEO and CFO uh, gave you a little bit about what we've done. We're gonna go through from a little bit, from the engineering perspective, share with you, uh, what we've done, how we've got here. Some of it may be a little bit of a repeat, but we'll make sure you know who we are. So as we think about, uh, Compuware, and look where we were, the first thing we did about five years ago is we sat back and had to look at ourselves in the mirror and identify a problem.

00:01:56

The problem we had is we were a 40-year-old company at that point. We hadn't delivered a new product in a long time. We had been stagnant. We were just kind of a, a company floating along, deciding what we wanted to do or trying to find our way. And we identified the problem that we weren't being innovative, we weren't being aggressive, and we made a decision that we needed to change ourselves. So what we did is said, we need to move forward. We need to find a way to get better. So we did implemented Agile, we started working on DevOps, and we did that by burning the boats. So to win you must burn the boats. And we did that. We transformed our company in a very short period of time. One morning we woke up and said, we're going to, we're gonna implement all of this within 60 days.

00:02:39

We had it all implemented, and we haven't looked back since in being a full agile DevOps company. And we deliver new software updates to our existing technology, new partnerships, new products, and we've done several acquisitions. We do all of that every 90 days. So the first business day of every quarter, we deliver, uh, faithfully to our customers. And so our, looking back at what we did and thinking about it, a great quote, uh, from Mark Twain, continuous improvement is better than delayed perfection, especially for those of us in the mainframe market. We always, we wanna be perfect. We know we run the world, so to speak. The world depends on us to be perfect, but sometimes you have to make sure that you're moving fast enough to keep up with the world and not worrying about perfection. Our goal is always to continuous, continuously improve.

00:03:29

So we work hard to do that every day. And how we do that is look at ourselves as we did, and we continue to do that today. And how we, how we really have to look at things when we, when we look back five years ago and we say, okay, we went agile, we started developing faster, we started delivering every 90 days. We've brought out new technology, we've done all these things. But the reality is when we ask ourselves, we all say we're doing a great job. How do we know it? How does anyone know what they're doing? They have to have real data, real metrics to look at and say, Hey, this is what we're doing. This is how we've progressed, or this is how we haven't progressed. The ups and downs, no journey. You've heard a lot of stories here this week. You hear a lot of people talking about transformation moving forward.

00:04:19

None of that is without some ups and downs. There's some good days and some bad days. If everything's always going up to the right, you have to question yourself. You try and do that, but you have to have real numbers. You'll hear a lot of people, uh, in the different organizations talk about how you do it. Surveys are nice to send out to see what people are thinking, and we send surveys out. Uh, we do Gallup surveys, we do different surveys for our organization to see how we think we're doing. And what we've done over the last couple years is we've actually created an entire product. And that's our terminology because we, we deliver products, uh, our Z advisor products, which is a product that measures how we're doing with our DevOps, and we use it internally. We, uh, have created within Compuware value streams for how we deliver our different types of work.

00:05:11

We classify our works in four different types. Uh, the standard, um, you know, the types of work being enhancements, um, defects, bugs. We do currency work for, uh, to keep up with what we're doing and, um, keep up with the market. So we deliver, we look at those types of works, and we have value streams for each one of those. And for each one of those value streams, we have KPIs that we've created to say, how do things move through our value stream? How do we measure all of that? So that's something that we've worked on and we've come up with, uh, what we feel is, uh, some very good dashboards. We look at them at the end of every quarter, we actually do a review of how we've done. We use the numbers to say, how have we done this quarter? And always looking to be continuously better. One thing you won't see in ours right now, and don't know if you ever will, but we don't show necessarily a trend. We do at times, but we look at how are we doing today and how can we be better tomorrow, continuous improvement.

00:06:13

All right, so three of the, the, the main KPIs that really focus on as an organization is innovation, meantime to resolution and velocity. And what you see here is our, our current state, uh, the quarter now and the, the trailing seven quarters. And what we're looking for from an innovation perspective is consistency and making sure that we're constantly focusing on delivering value to our customers, um, by releasing new enhancements. So everything that's not a bug, uh, we're considering that as innovation, um, but also focusing on those external bugs, those defects that reach the shores of our customer. And we have a careful balance of those two. Uh, but here, uh, you know, consistent, 70 to 80%, we're always working on new enhancements, uh, that are going out to our customers meantime to resolution. How many, how many days, uh, are our customers waiting from the time that they report defect until we actually deliver that back to them?

00:07:11

And you can see, uh, that is decreasing. Uh, and, and like David just said, we have good days, we have bad days, but we, we have a constant measurement that we focus on. And from a velocity perspective, uh, how much are we closing, uh, from a work perspective in Jira, uh, quarter over quarter? And because we have this data and because we have a consistent model, uh, within our Z advisor product, we can start to forecast. And you can kind of see those in the parentheses at the bottom. You can start to see expectations, right? We can use these numbers, we can use these metrics to determine how much work we can do in the future and use those as business drivers.

00:07:53

So, uh, during the conference so far, uh, what you've seen is a lot around value streams and, and flow metrics. Uh, flow velocity is a, a, a key, uh, piece, uh, to understanding what's going on in our organization from a work perspective. But everyone has to have a consistent definition. If you go to anyone in our organization and you ask them what flow velocity is, and this is the definition you'll get back, everyone has a common understanding of how we measure flow through, uh, flow through our system. But it's the absolute con of each, uh, type of work, uh, that's been closed in the period of time that we're evaluating. And David's gonna talk about how we use this.

00:08:33

So, you know, we, we look at our flow velocity, how much is going through, we can break this down. We do it at an organizational level, team level, and we look at two different things that we've accomplished during a time period. So if we look at a quarter or a year, and you see a couple numbers up there, there's a total, and then there's two numbers. Uh, one of those, if you can read the fine print says, uh, requires development. Those are actual issues where we did work and we actually delivered something out to our customer. So it went through our whole value stream, we updated our software, and we delivered value to our end user. The other number is the number of issues that didn't actually result in us delivering something out of the value stream, which for us is delivering code changes in our product.

00:09:20

So that is work that is done, potentially looking at what we might wanna do as a new enhancement, potentially just answering some questions for people that are using the products, but it's developers' time and work that's being done that doesn't ultimately deliver what we consider to be value, which is, uh, updates to our software. It's not all bad, but it's all things that potentially could be done elsewhere. The goal is when you have your value stream, get as much as you can, as much of the capacity within the value stream, delivering value out of the other side.

00:09:55

Uh, the next item we're gonna talk about is me. Time to resolution again. Uh, the amount of time or the average time elapse, um, from when a customer tells us about a, uh, an external defect or report, uh, an external de defect until it's gone through the entire cycle, uh, engineering cycle, uh, and, and we've released that back to them. Um, so if we're gonna work on, uh, something today, if we're gonna receive a bug today, um, it's gonna take about 56 days, uh, to get that back. Uh, so 56 days, 66 days to get that back, um, to our, our customers. Um, but why this is so important, David.

00:10:31

So the important, so MTTR and David mentioned external bugs and in comper terms, external bug is, that's a bug that an end user of our software has reported to us. It's the worst thing that we can deliver or do, is to have our customers experience an issue. So this is a key metric for us to look at. How long does it take us to get something back to our customers, uh, to get them back up and running or not experience that same problem?

00:10:59

Now we get a little bit more complex, and this is is flow time. So when we do decide, uh, to work on an item, how, uh, how long does it take, but broken down, uh, by each of the stages that that work item is gonna flow through, and we've broken it down, uh, into individual cycle times. And, and these are the ones that we measure today. And we're continuously looking, uh, at measuring other segments of work, but backlog cycle time, the amount of time that it sits on the backlog before engineering puts it in progress code cycle time, when the developer, uh, puts that that work item in progress and it's hands-on keyboard time, the time that they're actively working on, uh, providing value back to the customer code review cycle time, the amount of time, once the developer feels that they're done, that it gets reviewed by, uh, a peer, uh, developer and everything's been validated and validation time, the amount of time after that state until it's actually closed and released, uh, the customer. And so we roll that up to the, a total development cycle time accumulation of all those stages and then total flow time, the time that a customer can actually download, uh, that fix and deploy to their development or production environments. So this view, David, why is it so important to you?

00:12:19

Well, this is very important to understand how things move through our stages and how long it takes us, again, to actually deliver value. One of the numbers that's up there is our little iceberg that shows 79 in the middle. And that's, um, that's a, the number of days where it can sit and wait to, in some stage when a developer finishes coding, it has to wait for another developer to look at it to do a code review. So that's waiting. And that's as much as we reduce that wait time. That means that the quicker we get something out and we, so we get to look at this and say, how long does it take us to actually, from the day we start looking at something that we get a request to deliver something till we actually deliver it? How long does that take us? And in this context, um, this is our focuses on our new development, the new things that we're delivering, the fact that we're on a 90 day cadence where we deliver every 90 days, we expect these numbers to be somewhere in that neighborhood.

00:13:15

And that's one of the key factors when we look at all of these numbers, there's some business knowledge that has to come into it. Just the numbers themselves don't mean a lot unless you know, and put it in context of your business. And what you're looking to do for us, 90 days is about as fast as our customers can consume, sometimes faster than they can consume new technology and updates. But that's what we've decided our cadence to be. So when we look at that, see, can we meet that cadence? And that's where, uh, that's what it means to us to see how long it takes us to do something and how much we can shrink that time. If we wanted to deliver sooner, what would that mean? Where can we take time out of the system? Where can we increase, uh, our value to our customers ultimately in the end?

00:13:59

Good question. Is this a whole organization or just specific some sample?

00:14:05

Uh, this is the, this for us is an example of the whole organization would be at the organizational level. Okay.

00:14:12

Alright. So the, the last one here is innovation, and I alluded to it a little bit earlier, uh, but it's the, the time spent, uh, time spent, uh, working on non-customer defects. So this is stories, tasks, enhancements, currency, technical deck, everything that isn't an external, um, defect as, as David alluded to. Um, and so we, we really focused on, on this NU number, uh, but we can't focus on it too much if this gets too high. As you can see there at the, on the left part of that graph, we, we focus too much on new work and then we, then it started to drift off and we are focusing a lot more time on, on defects. So we really have to be able to see this in time series data and really develop a consistent measurement and really focus the teams and making sure that we're working on the right, uh, the right stuff. And then that, that's a careful, uh, balance and relationship that we have from an engineering perspective with product management, uh, making sure that we have both, we're working on new stuff, but we're also taking care of the customers and what they already have and what they're already using,

00:15:16

Right? And, and that is when we, we talk about that innovation percentage, right? Innovation is when we are ultimately delivering new value. Unfortunately, all of us who write software, we, we write bugs and we have to fix it. And how we can minimize that amount of time, uh, that we spend fixing bugs and reduce the number of bugs we have. ul ultimately that's the goal. And we've done a lot of that, and we'll show a little bit of that too, but as we go, how, what can you do with automation? What can you do with testing, automated testing? How can you reduce those bugs that ultimately go out? Because that innovation percent is the higher you get that, that's the more value you're ultimately delivering to the organization. And you know, something that goes back to, uh, what they talked about this morning, Chris and Joe, when we talk about, uh, customer satisfaction, customers are satisfied when they don't get, when the software works for them and where they're getting new features, new functionality that helps them be more productive, that's when we get the highest customer satisfaction.

00:16:16

So driving that customer satisfaction ties directly to this innovation. And that's when we look at our numbers, we can look at, uh, customer satisfaction and then we talk about employee engagement. Developers are happiest when they're writing new code. They love to write code, they love to create new things, and that's when they are fully engaged and wanna do something. No one likes to fix bugs. We all have to, but no one necessarily likes it. So when you, by re by making it easier to get, uh, more quality into the software by adding automation, you get that innovation up that helps employee satisfaction and that, uh, also helps customer satisfaction. And as was said, that then leads right into cash flow.

00:17:00

Alright, so I'm gonna switch over to, uh, live demonstration. So we really focused on, on showing, uh, each of the individual components, right? Uh, flow velocity, meantime to resolution flow time and, and, and flow cycles. Um, and this is the view that we give all of our development managers and our product managers the ability to look at each of those, but then look at their business specifically, um, from a work type, an agile team, uh, the product line or line of business that they're responsible for, or the family of products if they're responsible for more, more than one. Um, so if I'm, we have to a product called Topaz. This is a, an eclipse space, IDE and this is what, uh, a developer that receives our software, a mainframe developer has access to, and that's the entry point into all of our other products. So this is the main thing that we focus on at, at Compuware and delivering a great experience, uh, to the mainframe developer. And so from that, from this line of business and the, and the family of products, it's surround that we can see flow velocity, meantime resolution and the cycle times, uh, whether it's new work or a bug, uh, a defect that's reported, and how long that takes to go from when it's reported, uh, or when we decide on, uh, to do that new work until we deliver that to, uh, the customer.

00:18:23

And we also show on this dashboard, we show on the right side flow distribution, and that's signifies the four types of work. So we categorize all of our work into four types, as I said, and that shows what balance you're doing of all four. You have to give time to all four or something's gonna get outta balance, and it's not an even weight, but you have to look at that. So this gives, uh, your a development team, a scrum team, the opportunity to look and say, how are we balancing our resources? How are we balancing our time? Because we actually use time that they, uh, work on. So this is, you're actually seeing our actual time spent working on things. Uh, so that's what we showed there. And you get one look at how you're doing.

00:19:10

All right, so this is a little bit different view, uh, into the data model, uh, within our, within Z advisor. And looking from a value stream perspective, if you decide that you want to focus on a key specific piece, um, of the value stream, uh, where do you start, where do you experiment, where can you find the bottlenecks within the organization? So what we're viewing here is really a social graph of the communication that's happening, uh, within Jira. And so what we can quickly highlight and see are the cues in the system, and a queue by definition, uh, is waste. So looking at all the different components that are connected to this queue and being able to identify and start to drill down into, uh, where that work is going and the communication channels between each of those nodes. Um, what's unique about this too though, is we, what's most important is the people, right?

00:20:10

Within the, within the organization, the engineers that are contributing to the work that's happening. So this is the relationship between a queue, the products that, that people work on, but how those people interact with those products. Uh, so you could start to see and identify, uh, those bottlenecks. But when it's in this form and I decide that I really wanna focus on what's the impact of a certain item, um, I can start to remove items and start to see its effect on the organization or the products and how work is flowing through the system and decide what can't I lose, what's important so I can start to prioritize bottlenecks if I am able to identify them.

00:20:55

The lines aren't visible,

00:20:57

I'm sorry.

00:20:58

Yeah, it's just a video.

00:21:05

Yeah, it's a little bit washed out. So, um, there are, there are lines that are connecting all of these nodes and between each of those nodes, it's actually showing the weight, uh, of the relationship and the strength of the relationship between each of those nodes. So we can start to see how much is flowing between each of those nodes and the, the strength of that, uh, that social relationship.

00:21:31

Excuse me. What tool did you use to produce this? Uh,

00:21:36

Uh, great question. So the Z advisor product is actually built on the elastic stack. Uh, and so what we're actually using here is the, the graph technology that comes native, uh, with, uh, elastic. Uh, but a lot of the work that we've put into this, uh, is really around the data model. That's that and how the data flows into that system. Uh, the elastic stack allows us to visualize the decisions and how we, how we built that, that data model.

00:22:03

So it's a custom by yourself.

00:22:07

Uh, there's a group of of individuals within, within compre that actively develop this, uh, every day.

00:22:13

Is it open source?

00:22:15

No, it is not <laugh>. Sorry. Uh, something that we should consider. I agree

00:22:23

<laugh>.

00:22:25

Um, so one last, uh, quick, uh, quick demonstration here. So, uh, Michael dot Magnet, he's an architect within our organization, and, uh, he's a peer of mine. Um, and I wanna see, or I want to discover, uh, Michael's blast radius. I like to call, uh, in the organization, what products does he have an impact on and the strength, um, of relationship between the products that he's helping influence in the organization. So if I hit the plus sign here,

00:23:00

What it's gonna do is actually pull in all the products that this architect Michael, uh, has had an influence on. And what you can see by the strength of the line here, the width of the line, likely what he has had the most impact on. But also you have to consider is that potentially a bottleneck, right? Is all too much information flowing between one person and, and one product. It's a neat thing here, uh, that we can also do is see and, and start to expand and look at beyond that product, who are the individuals, uh, that Michael is likely working with, uh, on that product. So you could see here is myself, the UX engineer, the product managers who actually are sitting in the crowd here, um, and the data science team that helps build the data model behind this. So if you want to see how many people or what's the impact and the social graph of an application in respect to the value stream, if you've identified an area that is slow and not moving fast enough, you can start to experiment and understand how information and work is flowing, uh, through your organization.

00:24:19

And this is, this is a pretty powerful graph, uh, and pretty power powerful capability within, um, the, the product that we have. But to be able to look at your organization and see what happens if you move people, we're always planning as managers and, and in organizations we're looking to say, I need to, I need to create a new product. I need to do a new application. I need extra people over here and I'm gonna take 'em from here. And a lot of times it's, I have a problem here, let's go put out the fire, let's grab somebody with the capability there. You can actually remove a person from a product or f just take 'em out completely and say, if I take this person and put 'em on a special project, what falls apart? What isn't connected? What isn't necessarily being used? And that's a very powerful capability that, uh, I know when, when the first time I saw this, I was, uh, very excited to see it because we know our Brents in the world from the, using the Phoenix project analogy.

00:25:19

We know our bottlenecks and we know some of those key people. Sometimes we don't know who they are. Sometimes we don't realize that the quiet person sitting in the corner is doing a whole lot of things for different areas. And what happens if you move them? What happens, uh, to, to your organization and how does it, how does it stay together or not stay together? So it's, it's pretty cool what you can do with analytics. And I know it came up with, you know, what Z advisor is built on and the Elastic stack and we talked about, you may have missed it, we use Jira. So all of our issues are put in Jira. We do all of our, uh, agile tracking. All of our boards are in Jira. This data is coming from tools that are out there and just being put together and the brains behind getting all those analytics and getting those KK key KPIs. When you look at measuring how you're doing, uh, that's what you wanna look at.

00:26:17

All right, next one.

00:26:22

Oh, this is me. Yeah,

00:26:24

<laugh>,

00:26:25

We were doing pretty good. Um, so, so uh, you know, the, as we see in all the, all the, uh, speakers in, in all the sessions, it's what's the problem that remains? One of the key things is data. Data and more data. How we have been able to gain so much value. And from a comper perspective, we showed you data from a small period of time. We've been using Jira and we've been tracking things for over 10 years and we, we weren't actually tracking the way we track, but we were using Jira and we realized that we could pull all that data in and analyze it in many different ways. So we had the benefit of data, and that's how you get accurate measurements. That's how you can look at are things changing over time? Are you getting better or are you getting worse? And so having more data is key.

00:27:13

And metrics, standard KPIs to measure DevOps. We, we hear a lot about metrics. We hear a lot about KPIs. How do you measure that in a standard way? And that for us as a community, to make sure we're all measuring it the same way. We, we, uh, I know there was session earlier that was done by tasktop. They talk about, uh, their metrics very similar. We're getting very close. There's a lot of things there, but it's key that we get that all together and we say what is important for us to measure and make sure that we can show our CFO show our leaders that we are doing well. And it is helping. And again, sharing it's all, it's always about sharing in this community and all of us providing information to each other. A lot of organizations, and I know we work with a lot of very large organizations, a lot of organizations find this data.

00:28:02

They're concerned about showing it. We showed you live data. What our, that's compuware. We had good quarters and bad quarters and we shouldn't be afraid to show what it is. 'cause it gives us perspective on everybody has good times, everybody has some failures. There's things, the adjustments that you go on. We had 30 minutes to do this. I could give you a two hour presentation on the adjustments and the tweaks that we made where we added automated testing, where we changed how developers work, well, how we realign the organization, all of those things that go into it. That's the sharing amongst all of us. That gives us real value and really helps us to be able to move forward as we all look at moving our journeys along, getting DevOps implemented in our organizations, delivering more value to our businesses. These are the types of things that will help you. So I encourage everyone to share your data, share your information. Don't be afraid of it. You will learn something as we've done with many, uh, of the companies that we work with. We've shared our data, they've shared their data, we look at it, we learn things from each other. We continually get better by looking at what we have. So continue to share. Don't be afraid. And, um, it will be, it will all help you in the long run to move you along your journey.

00:29:17

Alright, so now for, uh, some, uh, exciting giveaways.

00:29:22

So one of the things I talked about was, um, we do value streams and we did a whole value stream exercise. And actually last week we had, uh, Gary Groover. Some of you may be who's heard of Gary Groover? Uh, a few people. Okay, good. So we had Gary Groover actually come in, uh, to our organization last week. He spent a couple days at Compuware and he looked at our value streams and we walked through and we actually presented to him what we present to each other to get a third party feedback for somebody to give us some valuable input. We, uh, he has a recent book, engineering the Digital Transformation, uh, just came out not too long ago. It's a great read. It's got a lot of good information in it for all of us who do, uh, this kind of thing. It's very informative and I asked him if I could give a few away because it was so valuable to me in helping with our value streams.

00:30:08

So the first three people that go to gary groover.com and download the book and use Rizzo, that's me. You pick my, for the the code, you'll get a free ebook. So please take advantage of that. Also, if you're not one of the first three, you can still download it there. Um, I don't get paid for it, but it is a good, you know, in the essence of sharing it is, uh, it is a very good book. It provides some insights into value streams and different things. And like I said, we spent two days with him. It was a great, uh, couple days. So, uh, that's Gary. And, um, the last, uh, thing, we'll take any questions. We are out of time. We just hit the zero marker. We'll take any questions. We will be here, uh, afterwards to answer any questions. We will be, we have a booth in the trade show. You can stop by and see us. We'll be here, uh, through tomorrow. You can always reach out to us at Twitter email, happy to answer any questions, give you more information about it.