Las Vegas 2020

VendorDome: The Time is Now - Accelerate DevOps & Agile with VSM

Can value stream management really solve all of your software delivery challenges? Join the live VendorDome session with HCL Software DevOps and Plutora to hear first-hand how Value Stream Management (VSM) is helping enterprises accelerate their DevOps transformations.


Moderated by Helen Beal, Strategic Advisor for the DevOps Institute, this session will include discussion with leading experts in the VSM space including Jeff Keyes, VP Product at Plutora, and Steve Boone, Head of Product Management for HCL Software DevOps.


Attend to get your questions answered live and gain insight into each vendor's unique approach. You’ll hear more about:

-Core challenges complex enterprises are facing

-What VSM is and how it helps

-How to align technology and business value

-Additional benefits of VSM for remote work, continuous delivery and hacking bureaucracy.

-Plus, how to get started with VSM in your own organization


This session is presented by HCL Software DevOps and Plutora.

SB

Steve Boone

Head of Product Management, HCL Software DevOps

JK

Jeff Keyes

VP Product, Plutora

HB

Helen Beal

Strategic Advisor, DevOps Institute

Transcript

00:00:14

Welcome to this DevOps enterprise summit, Las Vegas, 2020 Ventadome. But this is not a vendor dome. It's a vendor loving today. I'm Helen bale and stay. I'm going to be playing the devil's advocate. So, Jeff, please go ahead and introduce yourself.

00:00:29

Uh, thank you. My name is Jeff . I run the product team here at Plutora.

00:00:34

Okay. And on the other end of the rainbow, Steve, tell us all about you.

00:00:38

Hey everybody. My name is Steve boon. I lead up product management for our DevOps organization at eight sales software.

00:00:44

And so the audience we are taking questions live today. So please put them in the slack channel. That's the ask the speaker track for channels specifically, and let's get this party started. So what is the problem, guys? Is stuff really broken or are people really make me fuss about nothing? Are we saying that everyone's been a bit dumb and we've been doing it wrong all this time? What have we been doing wrong, Jeff?

00:01:07

Well, it's kind of funny. It's fun to be here at the birth of a new category called value stream management. And yet we're at a DevOps enterprise summit, a conference. And I remember walking around the booth last year and everywhere there was value stream value stream value stream, but I don't think people really get what the goal is. They just know it's cool. And so, you know, go get me to that tool or some of that, the core problems of value stream management focus on really four key areas. One people still can't see along the application delivery pipeline. They don't know what's happening from idea all the way to production one. If he can't see across a single pipeline that brings you the problem too. You can't see what's happening with the portfolio at the portfolio level, you have an additional problem of dependencies timelines, how they all fit together and they yet, they all have to merge going into a production environment.

00:01:59

The third core problem of value stream management focuses on the ability to manage having a control plane. If you will, to manage what's happening across different methodologies. Not every team has dev ops. Not every team has at the same level of dev ops. Oh, to be all automated all the way through, but innate reality and most enterprises. And the fourth problem is if you haven't solved all three, for sure you don't get metrics of delivery and you're not having any system of continuous improvement. I had a, uh, one of the partners that we have, I remember talking to him yesterday, uh, or sorry, last week about one of the problems they were facing in their dev ops transformation. And one of the best messages he said is, you know, we did a lot of really cool things, but we couldn't prove it to anybody. We couldn't show them we'd done better. So value stream management acts as a, as an umbrella to actually handle and solve these problems of visibility.

00:02:55

So we're flying blind, are we?

00:02:58

Absolutely

00:03:00

We're, we're, we're flying, uh, and as much as it's like blind, but I definitely think there's a problem with rolling this information up. You know, Jeff, you talked about this idea of continuous improvement. I think that's exactly right. You know, when we talk dev ops, we always talk about this concept of a fast feedback. And to me, the idea of, you know, visualizing your value stream or being able to look at your value stream in real time, to me, that's providing you the ultimate feedback loop, um, and in all different levels, not just at an individual development team level, how we communicate, how we train and plaque our work, um, how we can help one another and communicate better. Uh, but also when we look at it from an organizational level, you know, how do we identify high-performing teams? How do we know what their best practices actually are so that culturally we can embrace those.

00:03:42

We can share those, we can educate other folks. Um, that's been one of the biggest challenges, more and more. I see companies that are turning around and saying, okay, engineering team, uh, use whatever tools you want to be as effective as you possibly can. And that's great, but then you have the problem of, well, how do we make sure from a compliance and security standpoint, that all those different teams, all those different applications, the decades of technology are putting our business at risk. How can we actually start to validate what they're doing, learn how they're doing it and really use it as kind of a north star, right? So that we can start changing conversations from being reactive. Hey, what do we think is wrong to proactive? Here's how we're going to go fix it.

00:04:21

So we've been doing this stuff for decades, like you just said, so why haven't we got it right yet? Why it's, why do we still need to identify a north star?

00:04:28

Yeah, I think the big challenges, you've got a lot of large enterprises that would like to be unicorns, right? They want to do true continuous delivery, but oftentimes they feel more like horses. Uh, and the reason that is because we are still stuck in cab meetings, we're stuck, still stuck with a ton of regulation that we need to get through. A lot of folks are just, you know, blankly saying, yes, look, we've released this number of defects. We've ran. These security stands, we've done all of this great work. Somebody approve our release and that's taking a whole lot of time from some of your most talented and successful people. When we're spending, you know, hundreds of thousands of dollars in meetings validating this stuff. When the reality is we have the data, we could look at it, we could trust it. Um, and, and just start to lean out a lot of the different inefficiencies that we have on a regular basis so that we can start to trust our processes a little bit more.

00:05:16

We can start to trust the data a little bit more, um, and really start to moving towards embracing, Hey, if we're going to do all this time, uh, modernizing our applications, embracing Kubernetes, Kubernetes, embracing microservices, we should be able to push a button and push those things out whenever we want and have peace of mind of knowing that that's not going off the rails. And I think the only way you can really start to embrace that and, and celebrate it is if you've got visualization of that entire process and, you know, you've got the right safety checks in place. Um, so then you're not losing sleep over it.

00:05:46

I think, I think another problem is, is we've all been hearing all these benefits of, you know, go dev ops, go, uh, gosh, implement CGI and, and implement a new way of testing. It's not that these things are bad. It just, it may not be your slowest and biggest pain point in your entire delivery process. One of the things that value stream management really focuses on is take a look at the whole process. Does it really matter if you take your build time from 12 hours to two, when you're spending six months, um, ideating over the things you should build or another six months to deliver, um, as you said, Steve, from, you know, it checked in now, it's still six months to get through the process, man, focus on the right stuff. And I think the power of using these metrics to run micro experiments, to run evaluations of those choke points and fixing them, that's where this is all changing. I think there's one other point we've long had it beaten in our heads that, you know, software, development's not a factory. It's not just a simple process. Things are changing all the time. We have to stitch these pieces together. Uh, the idea that why not use metrics and, and tooling and bring this data together is now this big aha like, oh, I get it. That's what value stream management does now I can have visibility. So all of a sudden it's a hot topic. Right?

00:07:02

Okay. So we're kind of saying that maybe people aren't that dumb, but they're flying blind, which is not really their fault. They just haven't been given the dates that they need, but it sounds like we're still doing some pretty daft things, perhaps. I mean, Steve, you mentioned calves then cabs, are they, are they a stupid thing to do? Or if they're not, why do we do them if they just get in the way so much?

00:07:21

Yeah, no. I mean, I think we need to still have, it's always good to be able to look back and say, okay, yeah, we're doing right by the business. Right. We're making sure it's always good to double check risk, but at least a lot of companies that I've worked with these meetings go on all the time for every application. Sometimes they're week long meetings and you're getting to come back in and justifying your release process and justifying your quality. Um, we should just be able to trust those. When I think about things that we could do to really improve time, we'd start looking at business alignment, right? What are we working on day to day? Is it actually aligned with the business goals? How do we prevent things like unplanned work from coming in and taking us away from our key objectives that we want to deliver?

00:08:01

You know, now, especially something like 20, 21 more people are working remotely than ever. We find ourselves getting pulled away from different tasks. People need help. We don't have that single place where we can really come together and say, Hey, here's the company strategy. Here are the goals. Here's the alignment that we want. How do we make sure that every time that we're checking in COVID, we're writing code, but it's valuable, but it's what we should be spending our time in. And I think that if you can start to eliminate some of the waste and automate some of the governance checks around the cab, if you can get the team aligned on the prioritize thing, the business ones, then all of a sudden, you feel a little bit easier to go, okay. Yeah. Use those tools that you want to use, take whatever processes you want to do.

00:08:38

Experiment with two week sprints, one week sprints, whatever it might be. But the business still has peace of mind that, you know, we're not putting them at risk and putting it in jeopardy. And we're not just going through a process of rubber stamping, some of these things, right. We've all been in those meetings and they're like, well, have you ran your security scans? Yes, absolutely. We did. And we probably did, but no one in these meetings is analyzing them. No, one's really picking through them. They just want to know that you ran them. So how can we actually really go to bed and sleep well at night knowing that, you know, the police is just a hot mess that we can actually put good quality software. That's going to delight our customers and make them happy and actually deliver value.

00:09:12

And it sounds lovely, stupid. I've got to tell you that a lot of people feel that their business and technology teams are not like that United. So what does the person do if they feel that there's the business and a completely separate part of their organization, that is the technology team. So Jack any insights into the business technology divide?

00:09:32

Well, that's kind of the funny part about this whole world that we live in. You know, when I started being a developer, one of the best pieces of advice I got was go work at a small software company because software is the product that is the business. And it's funny that, uh, you know, now we're starting to be in this world where we're realizing that software really is the business of every company who would ever thought that we'd live in a world where the differentiating feature of a car, is it software? And yet here we are, we got customers where they're a power company. Yet they're producing things that are in-home monitors, street-based monitors, and the software that drives it is the differentiator software is eating the world. And as soon as we realize that, that changes the story. Now the next cultural shift, isn't just that software is the prime. One of the primary focuses that we deliver, but it's that we're all delivering that value across the value chain and helping people see the entire value chain and doing something like a value stream mapping exercise is really effective to bring the whole pieces of the puzzle together.

00:10:37

Uh, have you been looking at the questions That's asked is the question Gloria would like to know what is the difference between valley stream management and value stream maps? And since you said Bonnie stream maps, you can start with software.

00:10:51

Well, I will add value. Stream map is an exercise. It's a workshop. If you will, where you put people in a room that are involved in the, you know, hopefully as much as broad of a, of the delivery cycle as you can get. And it's an exercise of evaluating it. Step-by-step, we've, uh, actually on the Plutora site, you and I have done, um, webinars on that topic. Helen, you can go read all about that and watch a webinar on that exact topic highly recommended. Most importantly, not to, um, uh, not for the efficiencies that you'll gain, but for the alignment that you'll get of the entire team on one goal. And that's delivering software value, value stream management is the set of practices, tools, and, and, um, uh, culture, if you will, that come together to manage the entire delivery from beginning to end, that integrates into the tool chain that you've got, where you can implement those changes that you've defined in your value stream mapping sessions.

00:11:49

Yeah know. So two things I'll try to condense the previous question a little bit more to this. We talk about the value stream mapping, right. And one of the questions you talked about was, you know, how do we kind of bridge that gap between the business and the people? Um, I'm doing a talk later this week on the third day on humanizing dev ops through data, right? And to Jeff's point, when you start looking at the whole end to end spectrum, it gives people more empathy of what it actually required to re release quality software that makes our customers happy. So when, and what am I talking about there? Well, um, I think it was DevOps Institute earlier this year, put out a whole report around this idea of, of human skills and dev ops and what employers are looking for. And some of the top things that they're looking for, um, are in these, what we call ease, shaped type, uh, skill sets, right?

00:12:38

Which means people have experience they're energized, they're ready to go out there. They're not just T-shaped, which means that they're, you know, you know, broadly focused on, or I should say broadly knowledge of the company and then specialists in one they're, they're much more well-rounded in that. And in part of the way that we do that and help people become, you know, from T shapes to shapes is actually by exposing them to everything that it takes to get that software out. So it's not just about engineering, did our part, QA did our part, designed our part operations is doing their part, but they understand the business requirements. They understand why each part does the things, the way that they do what their expectations are with their management things, how they're being, um, you know, bonused and credited on their own work. And when we look at the whole bigger picture, then all of a sudden, it's a little bit easier to go, Hey, you know what?

00:13:22

This is challenging. We need to work together to solve this problem. It's not a development problem. It's not an operations problem. It's a business problem. Um, and that's, that to me is a, is a huge light bulb for a lot of companies. They think today we're doing DevOps. We're, we're doing a lot of the tactical things, but where they really struggle is the culture side of it. And actually being able to work together to bridge those gaps. So that value stream mapping exercise that, that Jeff talks about. That's huge. I mean, that is, it is as important as the retrospective. Uh, some companies are only doing it once every six months, if that at all. And we have an opportunity now to actually digitize that, make that an everyday conversation, make that something we live and breathe.

00:14:01

Okay. So I was about to make that point that it sounds lovely it's value stream mapping, exercise that I think most people feel that they spend far too much time in meetings. Uh, anyway, so how do we make it easier for them to do that continual inspection and adaptation? And I think you've just said that we digitize that. So Steve, can you expand on that point on how that helps people pivot and persevere?

00:14:24

Yeah, absolutely. I mean, think about all the tools we interact with on a daily basis, as an engineering organization from source control to ITIL, to issue tracking tools, CII, CD, you name it, uh, how we engage with those tools, the digital fingerprints that we leave behind, actually tell us a lot about how we form and storm our normal everyday, uh, activities. Um, a great example is working with a team who essentially was struggling on code reviews and they were wondering work would get finished, but it would take so long for it to get reviewed. And it wasn't that people weren't doing code reviews, it's just, they didn't have enough people to actually be a plus to code reviewer, right. It was a management delegation on the team of who could do what, and when your senior people are getting pulled into fire drills and they're getting pulled into other directions and you have to explain to management what's going on and why they're not always around to do those types of code reviews.

00:15:12

So those little pockets of waste, right, when we look at where the work actually sits, where a work item gets held up, um, is, you know, how long what's our work in progress, right? Do we, are we taking on too much work? Um, that, that kind of information, that kind of visibility by digitizing, that we can easily come in it's product managers, it's development managers is, uh, C-level executives, you know, search for the things we care about, search for the business value. We said, we were going to deliver a new front end in 2020, where are we on that? Right. We get these big business goals to get broken down into sometimes hundreds of different, uh, various different work items. So where our team spends our time, how we interact with those work items, how we interact with the code changes, all of that can be visualized and bubbled up to give us this really unprecedented visibility into how our teams actually work.

00:16:00

Yep. Yes. Tools, tools, tools,

00:16:07

Question from Carlos, some companies simplifying it VSM as a tooling issue.

00:16:14

Sure. Just buy it widget and you're good. Um, yeah, no, um, it, I, you know, I tool isn't going to solve your problem, uh, per se, there's the tool is an enabler. And, but you have to combine that with a whole bunch of other things, just like I said, you know, uh, culture eats process for lunch. Um, the, the point is, is there's a culture here that we're working on transforming along with architecture, process trust and everything else. The cool thing that value stream management gives is visibility and visibility creates that collaboration and efficiency that you need. What's going to change the culture is when you can run these experiments or have the ability to compare different teams and team styles and say, Hey, this one's working and then ask the critical questions. That would be part of the cultural naysayers. If you will, oh, they can't be secure enough or they can't be compliant.

00:17:08

Oh, they're not going to deliver the right kind of quality. Oh, whatever else is in there. And you know what, when you can correlate their MTTR, you can correlate their, um, bugs escaping into, into production, um, or even correlate it to customer sat and, and correlate it to your net promoter score. Oh my gosh, you're not proving something. Hey, this is the new way. And everyone has that same goal in general of wanting to deliver value to their customers. And if you can show it's being done here and it's being done well, you might still run some experiments that aren't successful. You know what a faster you get to quote unquote, failing the, and that's what the metrics are for.

00:17:54

Oh, sorry, Steve. The vision you're painting here, guys. This is beautiful. This beautiful place where we can see everything, but these teams have been building these DevOps tool chains. These teams have got tools already that tell them about that. MTTR their service desk. Us, for example, we know about the ideas. So they sit in the product backlog. So then we just need a dashboard, Steve.

00:18:15

Yeah, no, that's, that's, that's exactly what I was thinking of. The last thing we need is another dashboard. Um, absolutely. The worst thing anybody could say is let's put up another, uh, dev pit dashboard if you don't have any great, cool. Um, but it's not about the numbers, right? If you're looking at just a mean, you know, MTTR, if you're just looking at lead time and site time, those will give you a general sense of the health of your value stream. Right. But I bet that most people listening to this right now, if I asked you, Hey, what do you think about your value stream in the quality of it? You could probably put a finger in the air in ballpark. And you know, we have some issues right now, the rubber meets the road. When people talk about, well, what are the real issues?

00:18:49

How do we fix them right now? All of a sudden, instead of people saying, I think we want to be able to say, I know because we trusting the data, the data tells us. Um, so to me, it's not about a dashboard. It's, it's much more than those things, right? It's about throughput, right? It's about distribution. So what's your velocity. How many story points are you able to get done? That's great. But also what's the breakdown of those. Are we delivering more features than we are defects? Are we working on things? The business actually cares about more frequently if we're throughputs through the roof, because we're just knocking out defects all the time, really not delivering a whole heck of a lot of new value to our customers, right? Um, so there's a lot more there to me. It's, it's not about a number of the magic is in the relationship.

00:19:27

It's how all that data fits together. It's how you can look at lead time cycle time MTTR with the number of contributors you have with the number of defects that are going out the door with your distribution, all of that together tied with your quality results, your security scans. That's going to give you a way, much more broader representation of actually how healthy and successful a value stream is. And if you're making business decisions on, we want to put out this new feature we want to invest, or we want to build this new application. Wouldn't it be nice to be able to know, well, what are the best frameworks that we can build upon? Where is our best experience? What's our most, uh, healthy value stream. What's our best way to get from idea to production, because then I already know that I've got a leg up. I can already say, Hey, we've got a lot of expertise in this area. We might even be able to look at, do we have people in this area that it can, ER, you know, available right now to help us. Those are the types of insights that we start gaining that now aren't even necessarily at a developers mindset, but are more rolled up to, you know, executive level decisions.

00:20:24

Exactly. And if you think about the foundations of what makes up a value stream management platform, Steve, you know, you were talking about that. You have to, you need a common data model, a common way to look at every tool across the tool chain that you can ingest data and stitch it all together, correlate from planning to build dev test all the way through the release, uh, whatever the criteria is and on into production so that you can see the entire journey and then measure it from predicted value to what actually happened. And that correlation that's the magic

00:20:58

That's right.

00:20:59

So that's about flow. I think we're talking about here. So we're talking about the visibility of the work as it travels through the end-to-end system, from idea to value realization. But we have a question from James James wants to know, does maximum value require visibility of features from end to end in the value stream? I think this is an interesting one because it's not just talking about speed. It's talking about prioritization, Steve.

00:21:24

And then I, I read that too. You know, the visibility of features we're talking about, I made a release, there could be some feature flags that are on and some are not, I want to be able to look at what's the performance of some of those features. So, absolutely right. I think that's where this space is growing into is really start saying, Hey, I pushed out some things where I'm running. Um, I, the system is live right now, which feature flags we'll have on there that are actually, you know, high performing that we feel confident in compared to which ones I don't and who's taking advantage of those different things. Right? I mean, that to me again, we start talking about that fast feedback cycle. That's extremely important.

00:21:57

Yep. I think there's a trend to, from a product perspective that, uh, you know, just as much as we don't measure our developers and the number of keystrokes, they press we're, we're not going to measure our delivery teams by the number of story points that they produce, because really, you know, if you want to take an entire team and make them more successful, just increase your story point estimates and why you're now more successful. Um, the point is we have to look at the actual value delivered and what does that actually mean? Um, there is a prioritization, but we want to maximize that flow through the pipe and make it the most efficient that we can and measure the value that, um, comes out the other side,

00:22:35

All of that, that we can stop gaming the system. It's like Steve, what you were saying about data-driven conversations, not opinion driven conversations, I think.

00:22:43

Yeah, no, that's, that's a hundred percent, right. And just that other idea of, of, uh, you know, features and, and end to end, it's like if we are tracking the features, if we know where our features are at any given time or that breakdown between here's my epic, and here's all the linkage between the stories and underlying behind that. Now all of a sudden we can come in from a business perspective and say, well, where is the business value is at 30% on, is at 75% on, we could have clear cleaner conversations and set expectations accordingly with our clients. Right? We can say, Hey, here's when we think this will be done. As a matter of fact, we contextually say w way more clear than that. Here's precisely when we believe we'll be done, because we already know, like we've got such clear visibility and it's, it's so much better than a conversation of just going back to your product managers and going, ah, it's going probably going to have to wait.

00:23:27

And one more release will Y Y like, so, so often we just go, all right, that's fine. It's slipped. We'll get it to the next one. But why that, to me, that, that question of why something might slip, what else were we doing? Like we were probably spending our time on important stuff, but we should know that we should be able to plan for that. So this idea of being able to plan for the unplannable right is, is, you know, it's, it's impossible to do. We can start looking at just the way that we track work items. We for, you know, for planned work, we also track incidents, vulnerabilities, support tickets as they come in from our customers. And so being able to look at how our teams manage that work, um, who's available how that all bubbles up that directly correlates to how much business value can actually deliver per sprint per release per quarter.

00:24:17

So you mentioned agile and ethics there. Um, I don't know whether you'd caught sight of that next question from Keith, but I'm going to draw our attention to it. Now. I think it requires us maybe taking a few steps back actually, and thinking about what a value stream actually is, which we haven't talked about yet today. So for the uninitiated, what is the value stream? Because what keeps saying is that a value stream is like an agile, uh, epic versus user stories. So saying that we can get very large value streams that we may need to break down into pieces.

00:24:48

Um, I completely agree with that. In fact, I, you know, it's like when the story of the blind men describing the elephant, if you're, if you're trying to describe, it depends on where you're coming from a way that I like to think of value streams or describe value streams is we take all this stuff that's happening with agile, and you take all this stuff that's happening with dev ops, and you add some connectors to the outside for the inbound expected value and some connectors to the, after you put it in production to adoption, NPS customer sat and other value related metrics, that's value, stream management. And that sense, it acts as a control plane across these, these agile release trains, following every step of the process from ideation to production.

00:25:30

So what's the natural value stream. And, you know, I like to use the definition it's anything that delivers a product or a service, but am I massively oversimplifying state what's the valley stream?

00:25:40

I, I look at it as like, let's look at a practical example, right? Uh, take a single team. That's responsible for delivering an application. It can be many different parts to it, right? They probably know the different components that they're building for that application. Um, depending on how that team is structured, there could be multiple value streams there, right? If that application has four different core parts to it, each one is different, small micro teams within that, that are responsible for those. They could have different value streams, um, documentation design, all of those groups that we interact with to get software out the door, have their own value streams. So if we want to make a prediction or tell the business the commitment on when we're going to get something done, well, what does done mean done means it's ready to go. We can push it out the door.

00:26:23

We have the documentation, we have all of the marketing collateral that we need to get it done. We have all of the, your support ready to go. All that stuff has to be considered. And so when we want to push a product, we want to get that out that door. We want to have an idea of what, where all these other value streams, how do they impact what we're doing in engineering? So there's several different ones and it's important. Like we look at, as I try to break it down to let's get as granular as we possibly can. Right? So it makes sense that if you've got a small documentation team, that's just writing documentation for this one application. That's what they have a backlog of stuff that they work on, and they want to track that. And if we're going to say, Hey, when are we gonna get this latest documentation up?

00:27:00

Is it going to be in line with what we're pushing? The software would be really good to know, right? Not guess not hope, but actually know and plan for it that way. So a hundred percent, you're going to have a lot of different value streams. And I encourage all of the teams to manage, make them responsible. They will want to manage their own value stream. They're going to get enjoyment out of managing their own value stream because it's going to allow them to see all the inefficient inefficiencies of how they operate with different parts of the organization. Today

00:27:27

Sounds like an anti-pattern to me, having a documentation team, going back to your question,

00:27:35

We do away with document,

00:27:41

But you bring up a good point. I want to take you back to a question that I'd kind of skipped over. That came from the audience. Um, you talked about each shape, people in the context of the DevOps Institute, upskilling reports earlier on and James. No, not James. Uh, Nick wanted to know, is this each shape skillset related to what Gartner is calling a burst list? So yes or no. And then perhaps why multi-skilled people are important when we're trying to do this stuff.

00:28:09

Yeah, it, you know, I don't know specifically the question, uh, from the Gartner report. Um, but if it really, when I think about east east, east shaped skillsets, we're really talking about as people that not just have the technical chops to do the thing that we hired them for. Uh, but they have the ability to be successful collaborators within the business, right. They're able to go out, they're able to work with the different parts of the organization that can represent themselves, their team, others. And instead of just having a little bit of an idea of how that works, they're interested in it. They want to be engaged with the other parts of the business. Um, that to me is a solid shape skillset. A lot of that comes from experience, uh, in, in, you know, being curious about the different parts, you know, willing to experiment on how you interact with those various different groups. Um, so it sounds like there's overlap there without actually knowing what, what gardener is defined that as I hesitate to say, yes, that's the thing. Um, but that's, that's the way I look at it. Uh, we're going to be talking a whole lot more about that on, um, Thursday this week. Um, really just about how data in general helps make people feel more connected in a day and age where we're definitely not connected. We're the least connected we've ever been, uh, at least from a human to human conversation standpoint,

00:29:21

Jeff it's valley stream management humane Steve's claiming it is technology.

00:29:27

I think, you know, one of the biggest fears that people have of interconnecting their tools and having it come up into anywhere central, it's like, oh my gosh, people might see that, you know, we blah, blah, blah. You know what? It's not as fast as it could be. And I'll big people come and beat on us with sticks. You know, the whole culture is got to change the best part that you, that anyone, especially if you're on a team can do is to say, here's a baseline. Maybe you don't promote the baseline, but it's what happens after four months after you've made some pipeline differences and you say, look, we increased our throughput by 3000%, no matter where you were, that's the number that matters. We increased, improved our quality by X percent. We increased, improved our adoption. Our customer sat score by X. Those kinds of things are hugely not only validating but exciting as a team because we want to deliver stuff.

00:30:24

Is that humane? Absolutely. We feel like, I feel like so often the problems people run into, uh, of culture are, you know, it's the psychology of, how do I, how do I make this happen? I can't because so-and-so is over here and they didn't let me and I can't get access to you, do a value stream mapping session, all the ideas pop out. And everybody's like, well, that's stupid. Why are we doing that? The whole team like gangs up on the stupid things that are happening, and then you have a chance to actually prove it. That's powerful. And that's frankly really humane.

00:30:55

Yeah, I completely agree. I mean that, to me, that's the beauty of the value stream as I look at it is just how we communicate in an individual daily level, right. With a small team. Uh, especially if you're at a team that's not used to working remote and you've always worked, you know, in a, in a central office together, it's really easy to turn around and raise your hand and say, Hey, can I get some help on this? Or, Hey, can we look at this? You can say, oh, we've got slack, we've got all these different tools. I can just ping people for help. But when the fire drill happens and we need people to help right now, we need to get on with the customer, that unplanned stuff, right. That's really hard to manage right now. So what's going to drop on the floor, right?

00:31:30

We only have capacity to do so much as a team. And so we need to make trade offs. And if we want to make trade-offs and we want to say, okay, I need these two people to go help this customer because we have to solve X, Y, and Z. Okay. Well, whatever they're working, I need to either be covered or we're going to make decisions that it's not going in the sprint, but how are we going to justify those? What are the impacts of those decisions? All of that is extremely harder to do. It has consequences. People like checking things off their list and feeling like we got that done. So at the end of the day, if we can show everybody the bigger picture and why certain things are happening when they're not, it just makes it a much easier conversation to have. We don't find ourselves repeating or justifying ourselves to everybody.

00:32:07

Um, and to me that is extremely humane because right now, one of the things that I think everybody struggles with is just that overall, where are we going? I feel isolated. I feel alone when reality is, we're not, you're working with a team of people to solve really challenging problems and bring great value. Like let's not lose sight of that. And to me, that's one of the biggest things we can do is by making invisibles, we're making sure we're not losing sight of any of those things. We're not, we're not losing sight of quality initiatives. We're not losing sight of security initiatives. We're not losing sight of, you know, business goals. Everything's right there at our fingertips for us now to query and look through

00:32:40

The ad to put up more finer point on it. Can you imagine how humane it is to the team that's currently being managed to story points delivered, and there's a developer there, this like, you know what, I want to improve my pipeline. I want to do more automation, more test, upfront, discover problems before they get to production and having some testing, that's trying to, you know, repeatedly over and over test the same features. And hopefully they find all the problems and I'm going to go automate it. And yet what's in humanities. We're going to penalize that guy cause you didn't get enough story points done.

00:33:09

That's right. Yup.

00:33:12

Okay. Well, we're talking about connections. We're talking about discoveries. So making local discoveries and turning them into global improvements, I think across teams and this question reflects back to something. Steve said a few moments ago as well about the complexity of value streams. And perhaps that value streams may look separate behalf pieces or things in common. So Roman has asked, is valley stream component reuse count in different streams, a good performance or value indicator for

00:33:44

Component where you use kind of I'm guessing is, Hey, we've got a component that can be reused across many different applications. Um,

00:33:52

Yeah. And, and, and I, and I think what you could look at is is that there will be data that would show as long as that team can deliver consistently high quality that the teams consuming that component won't be negatively impacted. Uh, but I would think if, you know, you have a package or a group or a single reuse component that's being pulled in by tons of stuff, what I would really want to make sure that it's got a really good value stream around it, right? I want to be making sure that the security scans that they're running are running often that the quality and insecurity tests that they're running are published and we know what they are. And I want to make sure that we're getting for improvement on that, right? It's going to be hard to go to any team and say, Hey, you need to have 80% code coverage unless you started with that in mind from day one.

00:34:35

But you can go and say, is Jeff, you were talking about what's the current baseline and how are we going to get better? If it's 25% today, can we make it 26 or 27 by next quarter, if that's a goal for us, we need to plan appropriate. We need to, you know, taking those goals into consideration with the businesses goals, and we've got to juggle and weigh those appropriately. But I think had all of that visibility would allow you to know that if you've got that individual component, that's being pulled down and it's a poor quality value stream, that should be a red flag. You should want to fix that right away.

00:35:08

I want to ask if we should be talking about applications anymore in the context of valley streams or whether we should be talking about service components.

00:35:17

Yes,

00:35:22

Jeff,

00:35:24

I might even say people, just the people representing those service components. Right. I get back down at the end of the day. I'll all of those things. There's people behind all of that. And I think that's an important part as well, is that, you know, we're, we're, we're more than the IDs are the services that we log in with each day and to put these things together.

00:35:42

Okay. This is a great question here from Denver. Can you correlate dollars dollars, say from attitudes feeds, defect, reduction dollars generate by new features by retaining clients, getting new clients and features where additional billable items. Can we visualize that too? Can we put the dollars on the board people?

00:36:01

Uh, yes. Uh, I'll give you a couple that are really interesting. Uh, one interesting aspect, uh, a sub piece of the value stream is test environment management. I think it's one of the Achilles heel of any dev ops transformation. Um, we actually have a good, uh, piece that was done by EMA, um, on calculating the dollars of it, um, lost time downtime, lost development, um, and so forth. Uh, I think as you look at the, uh, quality and the software quality problem, um, you know, higher quality is less expensive to maintain and there's plenty of stuff that's been, um, uh, researched and presented along those lines as well. Huge business case for implementing advisory management, frankly. Um, just as go ahead, Steve.

00:36:51

No, it was, we asked the answer. Absolutely. I think the, the, the, his question is what I talked to a lot of people that asked me that question. My question is, is, yeah, where's your data at for that? And that's, that's the challenge, right? People will go, yeah, we absolutely can do that. Right. Because if I can look at, Hey, you had a customer, they were on version X and then they signed up because you put out a new version, went on that. Great. But that information is usually hard for a lot of clients to bring to me. Now, what we can do is we can look at what's the value stream costing you to operate, right? If we can look at the number of contributors you have, just from a straight engineering perspective, what's it cost me to run this thing, right? You could look at something simple, like head count, but if we can start showing that over the course of leveraging value stream management, we were able to identify and reduce the amount of time it takes for us, for the handle outages to identify and fix, uh, security vulnerabilities.

00:37:39

I think you can make a huge case and actually show that the things we're investing in other areas is a good payoff. I mean, that's, for me, when I started getting excited about value stream management is because I had people coming to me and saying, we've been doing DevOps for six, seven years. How do I know I'm getting the ROI out of it? How do I know that the things we're doing are actually beneficial to the business, that we are delivering faster, that we are reducing the amount of bugs. And so we started saying, well, we have to track that, right. We have to show that. Um, and so putting cost around it. Absolutely. It's just a matter of having the data.

00:38:09

If you're going to do a presentation to management and you want to, you know, have a case, have one slide, just one slide with this quote, the price of light is less than the cost of darkness.

00:38:24

That's the man,

00:38:28

There you go.

00:38:29

Well, other than that, rather than having one side clearly to chip, and we all know that leads to finally care about their KPIs. And anyway, but rather than having that one slide, would we not be able to just show them a live screen, showing them the incoming value with the product and the current cost of it?

00:38:45

I think that's the whole point is showing visibility what's happening in the pipelines, where are things going? How are they improving? You know, cause at the end of the day, people just want to see that we're improving and that we're doing better than we did before. And we're delivering more than we did before.

00:39:00

Yeah. And, and I, I think the, the thing that really, uh, I just speak from my own experience with the companies I've, I've worked at, the question is, is, Hey, you said you were going to deliver this thing. Where is it? How did we do it? And if not, what what's holding it back. Right. And, and even from those perspectives, those are sometimes hard questions to answer. People don't want to hear excuses. They want to see the actual data. And so anytime you try to explain the well, and I think in the bots, that's a little bit harder, but if we just show you transparently where we're at our time, then we know it's spoken for. Right. And, and I think that's the reality is engineering is pulled in a million different directions. And at the same time, the business is constantly changing their mind on which way, where they want to go, what their new priorities are, the new things they want to chase. And so how, how do we balance all of that and go back and, you know, prevent that conversation of engineering going well, we could've got it done, but then you guys wanted us to do X, Y, and Z, right? It's like, if we forgot clear understanding of where we're going and we can prioritize ourselves accordingly and the business, I think respects the heck out of that,

00:40:03

Let's go back a couple of steps. Let's talk more about business and leadership. Jeff gave us a great example of how the sell to leadership. And I made a very flippant comment about how leadership only cares about their KPIs and their bonuses. What do we do with our leaders? If we feel that they're not leading, listening to us, um, we, we talked about the message. We've talked about the data. How do we get that clarity of alignment that you're suggesting stay through the whole value stream, like blurring that gap between business and technology. How do we use value stream management to align the CEO's goals with my user story?

00:40:44

Yeah. Well, not only just the CEO's goals, right. But our clients and our stakeholders as well. And I mean, so it's just about executing the same things we've been doing, right. We should be taking that feedback. And that's the kind of driving goals that we have. Those should be in tools like aha. They should be epics and tools like JIRA. We should be, we should be tracking them somewhere. I think the problem is, is they tend to fall by the wayside. How many of us are dealing with backlogs right now that are just not super clean, probably plenty of and prioritize stuff back there. And every time we go to, you know, draw up another sprint, we could go, gosh, there's no shortage of stuff to do around here. Is there. Um, and that becomes, I think the overall underlying problem is if we can just look at that thing, if we can start coming in and each week I can show you a swim lane of the work that is already planned and prioritize, who's working on it where, and I can watch as they kick off builds, commit code, run tests, that piece of value flow through my organization.

00:41:37

Cool. Because I also get to watch where it stops, where it gets hung up and all of a sudden it's like, I'm coming in, I'm looking around and I'm saying, okay, where's this epic, well, three of these things have been stuck in development for three days. There's only two days left in the sprint. That's probably not getting done. Why what's that person do? And it's not the point of finger. It's just as a whole, we need to have each other's backs on a lot of stuff. There's always things flying by. So if we truly want to make something a priority, we have to do that. We have to free people up to get that work done. We can't have them being pulled and ripped in a million different directions. And so I think there's still some work that needs to be done. We're trying to take that a step farther by looking at integrating with tools like a hot to say, Hey, here's the big ideas. Here's the kind of two three-year plan that we want to go for. If we can bubble those things down, we can start to pull that information in. We can start get teams act on that more readily.

00:42:28

Yeah. I completely concur with that. You know, Steve, you and I are both vendors in this space and I'll tell you one of the most powerful sales messages we have in our sales motion is to say, here's, here's one of the dashboards you can have that show you light show you visibility of, of delivery. And, and here's the number of story points you delivered and here's the correlating effect. And you can trace it all the way back to the initiatives that were, uh, started with at the beginning.

00:42:55

Yeah.

00:42:56

So there's a question here. That's very pertinent to what you're talking about right now, such as I'll ask. Um, I was wondering if there's any part of value stream mapping dedicated to decision-making and the process inherent to it. So stakeholder involvement, discussion and validation. So I think what he's referring to here is the fussy front end. So you we've just talked about a heart, so let's talk about value stream mapping. So it's, we all know it's tool from lean manufacturing. Um, I think what often happens in our industry when we use value stream mapping is people start at the developer task and look at it as a CD pipeline. How do we use value stream mapping to get into the fuzzy front end?

00:43:34

Yeah, the fuzzy front end, I think, is where all the interesting stuff happens, right? Because it's where we have a ton of good ideas and we have to weigh the ways, the pros and cons with other investments that are going on within our business and with our clients. So, um, the data data's there, as long as we have it, as long as it's sitting in these tools, we can pull it in your value stream. Isn't linear. It has different shutoffs all around it. And so it doesn't, it's not just a pipeline. We'd like to think of it as a bill. I build, I deploy, I test, I released there's literally a million other things going on that aren't represented in those things. Right. I mean, I always tell folks we want to get as granular as possible. We want to outline our value streams to mimic how we communicate about work on our day to day basis.

00:44:17

So you just don't want to say, Hey, we built it there's before that there was probably a code route code review or two steps through that. Right. We should look at those. How do we communicate? We say, Hey, well this needs code reviewed well in our team. That's a plus one, right? And if it's going to pass code review, that's a plus to our value stream has bubbles that represent plus one and plus two, because work gets stuck there. Work can sit there. So any place work might be, we need to represent it. And if there isn't a tool today that I can tap into to represent that work, and we have to provide the API APIs so that you can get it in there one way or another so that we can start tracking it. Right. And so that's really where it's at. It's all about integrations, the better the integrations, the better the data and the better we can represent, how it's flowing through the organization.

00:45:02

You mentioned swim lines a little while ago, Steve, that was a question from the audience, which is there, is that a correlation from G sorry, this is from James. Is there a correlation between Kanban and V S M?

00:45:16

Yeah, absolutely. And, and not only Kanban, but lean and agile and in a number of different ways. Right. I look at all of those different methodologies is trying to provide the business with agility, right? You're trying to say, how do we do more with less? How can we plan something that we want to get done, but still be flexible in case things change. And all of these different styles and modalities use really are trying to say, Hey, we can help you do that. Right? Here's some best practices around that. So regardless if you're doing Kanban, regardless, if it's just straight agile, there's all different ways that you can help and just kind of visualize that work. But when we look at a swim lane, at least from, with, with our representation of it, absolutely. We think about it as a Kanban board, really hard to go move.

00:45:56

And I know a lot of teams have started visualizing their Kanban boards. And then it's nice to be able to move that physical thing across the board when you're done with it. But when you're looking at doing it across hundreds of applications or your organization, and you really want to be able to search, query, identify, highlight, and track those different pieces of information, it's really hard to do that with combine boards. Um, and so there is some aspect to that. We want to show that work moving, who owns it, where is it at all the way that's, that is a core concept and a core visualization that I think a lot of folks are driving to. Uh, but to me, it's about being able to roll that up. So it's just not a, it's not a single team or a single, you know, service or application, but it's it's for an organization and you can split that apart. Right. Um, yeah, hopefully that answers that question. It's a really good question.

00:46:41

Yep. I think the 0.1 of the core tenants of value stream management is to be indifferent, agnostic, not caring about what the development methodology is, whether you're agile waterfall or all points in between it should, it's agnostic for whatever level of, of dev ops automation and processes. You are, it doesn't care. The point is, is that you've got metrics with just are, um, in you in the larger enterprises. I mean, you've got decades of M and a divestiture, um, uh, you know, in, in some banks, for example, or power companies, even when they talk legacy systems, they might be talking about something that is, you know, uh, 50 years old. Oh my gosh, I complain about stuff that is, you know, nine, eight years old. I can't imagine some of these systems that have been around that long and yet it doesn't make sense to modernize these. Does it make sense to, uh, it may be, but maybe not because there's not that much investment going on there. Maybe it's just keep that system alive for however it is. Should you not value, you know, bring in metrics of it. No, you actually should least then you can have some visibility and, and see what's happening as you correlate these different, um, value streams together as they go on into production. You need still the visibility to see what's happening.

00:47:57

Okay. So I'm enjoying these conversations. We're having about kind of nested and bronched valley streams. And that's a question that's coming from Rochester as well, which is any tips to reach into silos, to get their value streams, to feed into the overall value stream. Kind of like a valley stream of valley streams that reminds anyone,

00:48:19

The team of teams talk we heard earlier today, right? I mean, that's A hundred percent

00:48:26

And I kind of feel like silos, we know are an anti-pattern. And I kind of feel like the whole purpose of thinking like valley streams is to break those silos and make them into a tube instead of a load of bubbles.

00:48:42

Well, look, there's a, there's absolutely value in every single team, just starting for themselves, because if you want to break down silos, as you've moved from project oriented to a cross-functional product oriented team, you're now, you know, we build it, we own it. We own the delivery, the maintenance, everything we care about what we deliver. So start there, I think it will create, um, you know, sort of, uh, a desire for other folks to say, well, I want that there's sort of feels like there ought to be, you know, you might be a redneck if kind of talk about, you know, you, you might need value stream management. If you're tired of getting asked the question of, you know, where when's that feature done, you, you might need value stream management. If, if you know, people keep trying to figure out what the quality of the releases and what is it going to be done. Uh, you could probably go on and on for all these questions that are silo oriented kinds of questions. Because if you're seeing silos, the answer is implement a system where all that data gets brought into a commonplace that's called value stream management.

00:49:47

Uh, Chris Novak, uh, who is, uh, runs our advisory and adoption at HCL software, uh, comes from a highly regulated banking industry. And he used to love to go and tell his teams, look, if you, if you buy into the way that we're trying to operate and you want to get people to support you, you gotta, you gotta show them what the trade-offs are, right? So he'd like to go and tell people, look, if you want me to keep them who want to keep audit off of bothering your team, let me automate your processes. I will pick tear of audit for you. You won't have to spend any time talking to audit. And that was a great way that he helped get people bought into saying, okay, Hey, we're going to let you help us try to automate some of our stuff. A value stream is a similar thing, right?

00:50:23

It's a gift to get relationship and just damn near everything we do. So what can we provide people, especially engineering teams? Well, we can obviously we've been talked a lot about visibility. Um, developers want to get better. They want to do the most efficient job that they can, uh, this will help them with that. I also think that if you say, Hey, if you want us to keep the release engineers off your back and asking you 30 questions before, you know, every couple of sprints that we want to push out the door, this is a great way to do that. If you want to spend less time in your calves, this is a great way to do that. If you want to make sure that, you know, you don't have to go and vet every little thing you do and explain every little thing you do, because it's already out there.

00:51:01

People already understand security. The CSO knows what you guys are up to, right? Quality leads. Know what you guys are up to. This is a great way to do that. All you're doing is helping be a better communicator and I'm not poking a new fun. We don't have a lot of great communicators. Um, and, and all the engineering teams that I've been a part of and people are very focused on their day-to-day tasks. They're great at going out there in Boston, not code. We need to communicate now more than ever. We could use better empathy, better communication. That's just the world. We're in call it the new normal, call it whatever you want. But that's exactly what this is designed to do, right? It's just make everyone's lives a heck of a lot easier. And put us all on the same page.

00:51:38

I'm going to table what you're saying. A line us up with a question from Eduardo. So problem is everyone's really busy, right? The problem is what we're talking about is an emerging market and emerging platform tool set, whatever you want to call it. So at the moment, we've got maybe a minority of people that have got the proof point. It works and people are just going to say, yeah, it sounds great, but I'm far too busy. I'm trying to keep the lights on I'm firefighting all the time. So it Waldo's made this point about what you said is why shouldn't we treat technical debt as any other item in the product backlog, to which my response is a normal human being, not the devil's advocate would be you absolutely shirts to technical debt. Put it backwards. No question. Right. But if he's having to ask them, I'm guessing he's being told that he shouldn't be. So how could he use value stream management to show people why measuring technical debt is important?

00:52:31

Well, I tell, I tell us the same thing. I have a conversation with my kids. If you go into the toy store, right, buddy, you got $10. How do you want to spend it? You can spend $2 over here. We can get some candy. It's been $5 over here. We get a toy, but when we have $10 and that's the time that we have, and we've got to think of those same kind of trade-offs and we're talking about paying our technical debt or paying anything else that we want to get out the door, how much allocation are we going to put towards features? How much allocation are we going to put towards technical debt? Because those are real conversations. And as technical folks, we want to come back and say, you're crazy. We can't do that. Unless we do these other things. First, we need to pay this debt.

00:53:07

That's a hard conversation to have with a C-level executive or some other folks that are really focused on closing that deal and getting stuff done. Right. So how do you make a technical conversation? A little bit easier to have shut down the breakdown? May I show them the data, the data doesn't lie, right? And that's, that to me is like the, the, the best part about this is we can come in say, look, if we want to get this done and we still want to pay it's financial advisory for your backlog, there's really one way you can look at this, right? If you want to pay these things down, if you want to get ahead at the same time, we have to pay into each bucket a little bit. And so what better way than to know and prioritize what those things are and actually show that we're making headway on them.

00:53:43

Yeah. And the key there is, you know, absolutely correct. And the thinking that, you know, everything we do or from a product perspective or just features problem is, is that when, when there's a planning organization, they don't think about technical debt and there's no visibility like, oh, there's a bunch of technical debt and here we'd better go fix the value here is making that visible and transparent. So now it's, it's back of like, well, I got, I got, how do I spend my money?

00:54:08

So we're into our final five minutes. Can you believe that this conversation is flying by? So I'm going to lead us into tying up. And I think the most useful take way that the audience is going to get to say based on the questions and in particular, the last two questions we have from Robin and Roger, I want you guys to, first of all, explain as succinctly as you can, how somebody that's never done it before, goes back to their organization and identifies a value stream. And then what the first three steps they need to go through in order to make sense of that value stream.

00:54:44

I'll just wing it. That, how about that? I'll go quickly. Hey, pick an application that somebody can concrete point to, whether it's a cloud solution or a portion of it. What have you, and look at the process from beginning to end, how to get started, pull the people in to a room for a couple of hours and ask, how do we decide and create from idea all the way to production everything that's required in this process. And every time we had a step in the process, put a box, go through the value stream mapping exercise. Next step, take the data from the different tools where stuff's being tracked and put it into a value stream management platform and show the data and baseline and see where you're at while you're doing the value stream mapping exercise. You're going to discover, Hey, it's really dumb that over here we do X, Y, and Z. How could we fix that? You'll have little Kaizen moments, great fix it, put it in the tooling for, by your value stream management platform, to have a newer model that will help you through the process measure the final result. When you get done with the updated process, whether you automated, you changed how you prioritize, whether you, uh, limited your work in progress, whatever the fix was great. And then you'll see if it was successful or not,

00:56:02

Th you spot on, right. I mean, we, we've got a couple of different eBooks available at our booth. Um, you know, one of them's on, on data-driven DevOps. The other one that we have out there I believe is, is, uh, you know, getting started with value stream management and, and kind of walks through some of those fundamental things that teams can go through to grab some markers and, and have the whiteboard conversations. When the reality is you can get started at this at the end of your retrospective, right? You're already asking, Hey, what went well? What went poorly? How do we improve? It's great, great play. But instead of really honing in on some of those things, I think it makes sense to step out of that, have a whole dedicated meeting where to Jeff's point, we're just looking and talking open and honestly, right.

00:56:39

And that's kind of, maybe the big thing is it's gotta be open and honest, what do we, think's going on? How do we think the work actually flows through? And I think it'd be really interesting as you'll see people have different opinions about what they think is actually happening with what we do. What exactly. Yeah. So, so you're a little core bubble of people that you hang out with all the time are probably in some sort of alignment of how you think the world is operating. But as soon as you branch out, it's like a game of telephone. It goes completely farther down that road, right? You start having conversations with the design team and they start telling you how their process works, who they have to do, who they got to speak to you to get stuff out of what their requirements are. And all of a sudden you're like, dude, I just thought you made the pictures for us.

00:57:17

And we put them in the product. I didn't know, were going to be that big of an ordeal, right there is that all over the place. And, and so having a conversation with your core group, um, being able to visualize that understanding where your handoffs and, uh, the groups you work with are important, then you can come over and say, I want to work better with you. That's a great way to let me, Hey, nice to meet you. I'd like to work better with you, right? And if we can have that conversation within our organizations, I think that just says, you know, tell me more about how you do your job. That way I can help you do your job better. And I can benefit. We benefit together. And to me, that I think is, is, is the great way to go about doing it and getting started is just ask people, how can I make a job better? What do you do? How does your job operate? And, and that'll, that'll initially start the, the appropriate things you need to do to start mapping those things out.

00:58:06

Perfect. So thank you very much. I've been had bail. I've been the devil's advocate today. This has been opened beloved. Thank you, Steve. Thank you, Jeff. Thank you for bringing humane technology to the masses.

00:58:19

Thank you. Thanks.