Las Vegas 2023
Generative Value Streams: When Code is not a Constraint
Generative Value Streams: When Code is not a Constraint
DM
Dr. Mik Kersten
CTO, Planview
Transcript
<silence>
Okay, so the next speaker is someone who should be familiar to most of us in this community. Uh, Dr. Kersten wrote the awesome book Project to Product five years ago, uh, which is being used in so many organizations. And, uh, uh, the language has showed up in so many of these presentations. Uh, today he's currently the CTO of Planview. So over the years, he's taught me so much about architecture, which dictates, uh, how organizations are wired. And so, uh, so much of that shows up in the book that, uh, Steve and I, uh, wrote. So Mick and I were talking earlier this year, and when he told me about a conversation that he had with a mutual friend of ours on how generative AI will change coding forever, uh, I immediately thought that this merited a talk, uh, here, uh, at the conference. So I asked him to share with us not only how coding will change, but uh, the changes that he sees, uh, coming in, how software is built. So, here's Mick.
Thank you.
Hello, everyone. It's great to be here. And I just wanna share with you that for my entire career, I've been looking for the these 10 x improvements and how software is built, and how our organizations can innovate and bring value to the market, and how to get that as broadly distributed as possible. So, to my surprise, in, uh, about a year ago now, uh, I, I'd always thought that this, these changes would actually come from programming languages. So I actually went back and saw some of the same messaging, uh, that's being used now around these, all these code pilots that help you generate code. This is actually a presentation from, uh, Java one in 2008. So 15 years ago when I thought these, these actually, these productivity benefits would come from open source tools, uh, they'd come from new programming languages, and they're the things that would take out all that friction and toil from our day-to-Day jobs as, as programmers, as developers.
And I, I spent over a decade doing that in building building code. And what's been so fascinating with some of what you saw in, in the programming at develops Enterprise Summit is that we're now actually seeing just complete step function change that's going well beyond anything that I was seeing as productivity gains through open source, uh, through agile methods, through some of the automation practices that we all have and that we all apply. And with that, uh, we'll have the quick emergency broadcast. I'll take that as a round of applause. Uh, and we'll continue in just a moment. So,
<laugh>,
So, uh, the point here is that we're actually now at the point where we're seeing it possible to multiply, I think we have line of sight to multiplying the productivity of an individual developer by 10 x. And this was kind of always the dream. And again, all that friction and distraction was always a frustration. And I'll now share with you some slides that I co-presented. It's called densmore, the head of GitHub copilot, uh, with the data that they presented in their own study of the application of GitHub Hope copilot. So we're seeing 55% faster coding, 46. This next one is fascinating. The next two, I think are fascinating. 46% of the code written in those repos was generated by GitHub co-pilot. And then the developers, and this is my favorite one, obviously were 75% more fulfilled because they had to write less boilerplate and less of that annoying code and tests that actually deliver less value.
So I think this is, this is just the start. And this is not about GitHub co-pilot. This is about the fact that these large language models can take our thoughts, our intentions, and actually generate amazing code. And I actually believe we'll go from 52% to five extra to 10 x of the scope of the next year or two. So this is just such a fundamental shift. And if you think back to, to five years ago, uh, when Project to product was, was announced on, on this stage, one of the main themes of that book was looking at the last technological revolutions. The conversation I had with Carla Perez most recently was in, uh, in 2019 when we were brainstorming, well, what is the next technological revolution? We went through the age of software and digital, what's actually coming next? And we actually started talking back then about artificial general intelligence.
Would that be it? And she said, you never know when you're actually, when it hasn't started. But she says it'll have the same traits as the previous revolutions had. So some means of production will actually be, go from being very expensive to much cheaper and more scalable. Before that, it was electricity that, that really amplified things. Uh, my apple watt's a little delayed. Um, it was, uh, and before that we had steam and transportation, mass production, all of those things really transformed how much we were able to deliver value to the market. Now, all of a sudden, if you believe some of those previous numbers and you extrapolate them, and I'll show you how we can extrapolate them, all of a sudden writing code, the main limiting factor on digital innovation is about to become 10 times cheaper. So the question really is, how profound will this be?
I think it might be as profound as the internet and electricity. And of course, how will your organization, how will your teams leverage it? How will you get ahead of this curve? Because what we've seen every time we go through these tech technological revolutions is some organizations get very fast at wielding all this new power, whereas other organizations slow down and languish and then actually get disrupted and, and decline. So really, the the point here is we are in this new age of artificial intelligence, and how will you and your teams apply this? And how will you accelerate some of the activities you're already doing around transformation, around digital, around DevOps, uh, and by harnessing what's now available extremely broadly through these large language models and and ai. So I see these three layers I think we all need to think about in terms of how to apply this to, to delivering value to our customers, to our organizations.
At first, there's this, this code level, so these code agents that can actually produce new code, produce value for us. So basically write the things that we wanna deliver to our customers through software. Now, I think that's just the start. I think we're actually gonna have this next level of agents, and I'll show you a preview of that, uh, that will operate in the value stream. So rather than just helping the individual developer, they'll help identify and resolve bottlenecks across the organization. Uh, and then finally leveling it up to, to the theme of Jean and Steve's book. So Wiring to Win the Organization. I actually think some of these concepts can then be applied, and some of those, the amazing heuristics and frameworks in the book, uh, can actually be applied at the organizational of the strategic level, which I think is a bit further ahead.
But I think each of these has the power to amplify the productivity at every level of the organization. And I think in each of these, what we'll actually see, uh, in the, in the coming months, and what I hope we see in the coming years as well, is that there's will actually, this will amplify the effectiveness of what we're able to do today. I think there's a, and I'll show you some stats on this as well. There's a lot of inefficiency in how organizations build digital experiences, build software, uh, to get that 10 x across organizations to be able to build 10 x the software is, I think, an amazing thing that you'll be able to harness. And, you know, Sam Altman himself said that he does not think, even though he's got a very good perspective on just how much productivity can be gained through coding, it's not that we'll have a 10th of the programmers, we'll have 10 times the software that's being written.
But again, I don't think we should stop there because, uh, for many organizations, we need to up level this as well. But in terms of how I've been thinking about the, uh, the code level and those kinds of agents and where LMS help us there, we've been applying this at planview. We're actually, of course doing the code generation. There's things like code brushes that can clean up code. I think we're going to see a lot more of actually the prompts becoming the code. So we have a lot of developers now who do prompt engineering. So I encourage you to try each of those things, whether it's through GitHub, copilot, um, Amazon Code, whisper, wherever it is, just start learning how to use code generation, because this truly is the future of, of the coding profession. Uh, prompt engineering. You'll have to have developers in your organization to learn how to do that.
There's been a lot of great content at this conference around that, and I think we'll have exciting new things coming. For example, I remember a few years back, it, it took our teams quite a bit of work to move one of our products from Angular to react. Those are the kinds of things that these LLMs are gonna make completely automatic through auto GPT, where you can actually incrementally modernize an application, write its test suites, and validate that it's working with very minimal user input. So all of a sudden, all of this tech debt and these, these legacy constraints organizations have, some of those things will actually start going away through the power of these, the, the growing power of these LLMs. So this truly will transform how, how software is built. Uh, the challenge that I see here is that if you look at the data, and this is the state of, uh, state of the project, the product industry report that we had planned, we ran earlier this year, we're actually seeing that only 8% of the end-to-end value delivery time, the end-to-end flow time is spent in development.
So 10 Xing that won't help in these organizations. These are 36, uh, sorry, 34 different enterprise organizations, 3,600 value streams. And this is all systems data. This is data that's been extracted from Jira, GitHub, GitLab. The actual systems and work is getting stuck upstream and downstream of development teams through all these dependencies, these weight states. So it's not enough just to do that 10 x unless your organization has already removed those big red boxes, which again, all these organizations, uh, haven't, these are, these are some very large, uh, organizations who are in the midst of these transformations. And we have to really get past this. So I'll, uh, dig into this data a little bit more. So right now, to show you how big a problem this is, leaders keep completely overloading teams. So all of a sudden, if a leader, and by the way, the data that we're seeing, we augmented that system data with some survey data, uh, lead technology and business leaders, there's things that they're value streams.
So their teams of teams have 10 times the capacity that, that we actually measure in those value streams. So planning is being done without a view and capacity, productivity is not being properly measured. Uh, and of course this is causing these tremendous wastes. 40% of team efforts are wasted due to overload and bottlenecks. So that's putting more overload and bottlenecks on teams. It won't help if they're 10 times more productive 'cause we're actually just making the problem worse, uh, and not addressing the constraint. 80% of value streams don't proactively invest in technical debt. So again, if this is not happening, we're having the teams do the wrong kind of work because we're not prioritizing the kind of work because all of these things are disconnected. And by the way, the power of these models doesn't work unless they actually have that data in the connected the way that they can reason about, as I'll show you in a moment.
So I think it's time. We, we, the main message here is we can't stop at those value stream that those code level agents, we actually need to uplevel this to the, to the level of value streams and actually help us solve these problems that continue to pla our development teams that really effects of organizational dysfunctions. So we've now got a whole library of flow optimizations and the scaled agile framework. These are called the flow accelerators. We know how to measure flow through the flow framework and the flow metrics. So we can actually apply these things automatically. And we've been experimenting with this. We've actually got flow optimizations that we will be applied automatically to value streams. If the value streams is overloaded, why not automatically reject the work or maybe prioritize some very critical work. All of these things can be done once you've got the whole organization modeled in a way that, that the large language model can access managing roadmaps, again, prioritize the key work, but create capacity for reducing tech debt.
If tech debt investment goes too low, of course the organization should be warned about that. We need to make this much more visible to our leaders, and that's not possible. And now through the magic of the approach of auto GPT, we can actually set things like objectives and key results for these agents to track to. So if we see that our modernization, our our dependency reduction efforts are going to slowly we're under, under investing in architecture or in in modularity, these kinds of things can be automatically implemented. And where, where the large large language model, basically the agents are in the loop and actually help us do this. So the fascinating thing is, uh, this is, this is now possible today. So for the last, I'll just give you a, a short preview of what I've been doing with our, uh, research and development teams where we're actually able to point an agent at the data in an organization.
Of course, the data has to be connected. So imagine the team level data, uh, the agile plan data, the OKRs, the investment data and so on. And I can actually ask an agent this question. So this is a, this is a, a copilot in this case it's the Plan V copilot. And I can say, what should I be worried about? And it's able to take all this data and tell me that the team is overloaded because the flow load exceeds 1.5 times the flow velocity. So it's actually able to apply from all the body of knowledge it has access to from the rag that we added by giving it project to product another relevant lean literature. And from the data that it's seeing here, it actually does all of the computations GPT-4 and to some degree Claude Claude two, but not nowhere near GT four can actually do all of this computations and all the data it's seeing and tell me that we're actually, we've overloaded the team and we're gonna miss our goals.
It's also noticing that the rate of defect fixing is, is taking more and more capacity with the value stream. So they have less ability to deliver features and innovation. And this is now possible to do at the scale of an entire organization, because once you connect the data, once you feed the right data, it can actually highlight these antipas and these problems for you. And of course, present it at the right level to the right stakeholder. So I can actually ask it. I've, I've asked that many countless things at this stage. I've asked it to, to explain flow metrics to me as I was, as if I was a 5-year-old old child. And, uh, it, it will like pick cookies and tells me how many cookies are being baked concurrently. Of course, that's the flow load, uh, on my, on me and my friends. I also, with the prompt engineering, I've, I've realized that whenever I'm demoing something like this to an executive, I actually always will append, uh, and explain this to me as a, as a, I was a 5-year-old child because then it uses very natural normal language <laugh>.
Um, and, uh, yeah, don't, don't read too deeply into that, but, uh, but the, and that is by the way, a trick that you can do with GPT four. If you ask you to explain things to you as a child, it will go through these very thoughtful, uh, processes. So the neat thing is if the data curate the right way, uh, we can now interact with it. And I think this will empower, uh, leaders at every level of their organization to help support their teams, remove those bottlenecks and remove the constraints. And then I think we can go to the next level. I think we're actually gonna get to the point where if we've got large technology to re platform things, re platform things that we need to do, uh, it's going to be possible to have those happen in a very automatic way. Um, if we have organizational structures, we need to look at, we now have from Jean and Steve, a whole new and extremely simple and effective vocabulary, applying things like simplification, ification amplification.
If the agents are not seeing that in your organization, where should you look at improving that and, and provide leadership with guidance of what those organizational dysfunctions are. Uh, and finally, investment allocation. If the investment data's there and you're not getting an an ROI for particular initiative, uh, if your latest launch or replatforming modernization effort is falling behind, of course this can now be much more visible, purely because once you send the right kind of connected data, uh, organizational data, basically this organizational, uh, work graph, uh, the LLMs have this incredibly powerful, uh, reasoning ability that, that I actually, you know, I, I know exceeds my own as a product manager, um, and as an organization leader. So I think the key thing is now leveraging this, making sure all your data's connecting it and just getting started. So I do think we're gonna have this, this cascade now where there's a 10 x, we're not there yet, right?
The numbers you saw earlier from the code level agents were 50%, but a potential 10 x capacity gain at that code level of individual developers, then another 10 x at the value stream level. I mean, we know that's possible from the data we're seeing in these value streams, the data that's flowing through them. So it's possible to unlock that and even at two x will have a massive impact on your organization, on your customers. And of course I think it's little much less clear what will happen at that strategy level. But the fact that you can use all that reasoning and again, apply the framework that we have, um, such as that from wiring, the winning organization will allow the LMS to apply those automatically and look for, uh, where we improve So quickly. In summary, I think in all of this, how we leverage nvi AI is all about people times machines, not people versus machines.
We want to amplify the work of our teams, of our leaders, uh, all the way up the, the organization to do that, you actually do need to make sure all of that work is connected because as soon as, for example, we see our, we're not feeding the LLMA dependency, it can't help the same way that once you're digging in manually through this yourself, you, you won't be able to make that same conclusion if you're missing data. So all work has to be connected for these LMS to leverage it for you. Um, and I would say just start adopting now, if you're not adopting code agents right now, do it. If you're not trying out these value stream agents, do it. And the other key thing of course is getting back to flow is, uh, you need to actually make the economic case to the organization.
So what we've done, of course, is looked at how did flow improve for teams that used, uh, GitHub copilot because it's not enough to look at how much code was generated, but how quickly that that value got to market, um, and how the flow velocity improved. So of course these things are, are very constructive and, and we now have these LS LLMs in the loop. So I'll wrap up now. But in terms of help 'em looking for any learnings that you have, applying it at these various levels, especially that, that code level, the use cases that you found really working, uh, as well as the value stream level you start experiencing with that, I would love it if you could share it with me, um, and the team. So it's just make a plan.com for more. So with that, thank you very much.