London 2019

Project to Product: Thrive in the Age of Digital Disruption with the Flow Framework

Based on his work with Fortune 100 enterprises, Dr. Mik Kersten presents a new concept, the Flow Framework, which unravels the enigma of enterprise software delivery, providing an approach that helps organizations better understand the human and system dynamic that underpins their IT infrastructure. Dr. Mik Kersten spent a decade creating open source developer tools before realizing that programing was not the bottleneck of large-scale software delivery. Since that time, he has been working on creating a model and tools for connecting the end-to-end software value stream. Prior to founding Tasktop, Mik created the Eclipse Mylyn open source project as part of his PhD in Computer Science, pioneering the integration of development tools with the delivery pipeline. As a research scientist at Xerox PARC, Mik implemented the first aspect-oriented programming tools for AspectJ. Mik is the CEO of Tasktop and loves working with IT and thought leaders on transforming how software is built.


Dr. Mik Kersten

CEO, Tasktop



One of the themes have showed up in every plenary talk over the last two days. There's this notion of project to product. The next speaker is Dr. Mick Kirsten who literally wrote the book on it. Uh, his book, project to product is generally a book that I admire and envy. When I first read the draft of it. My first reaction was I wish I were smart enough to have written a book like this. I think it's a very important book because it provides that last a language that can help span the business community and the technology community. And in his talk, he's actually going to also provide historical frame that suggests as a new, as a part of a new management mode that gets created only once or twice in a century. So without further ado, Dr. Mick, Kristen


Thank you, Jean. And hello everyone. So for the first decade of my career, I spent all of my time in the code and writing code and coming up with new programming technologies when I was working at Xerox park and new kinds of development tools are because I actually thought we could code our way out of pretty much any problem. And the really interesting things I learned as we were building these tools that were making us more and more productive and keeping more and more of our time in the code, creating value is just that feeling of focus and flow and joy that you get when you're delivering amazing things for your users and your customers. And I got so interested in trying to scale some of these concepts. They actually went and did my PhD in connecting some of the lessons that we learned from programming languages, from software architecture to end to end value streams into how business flows through software delivery and my PC supervisor.


And I decided to try out and see if these concepts could work for large scale and the priority organizations and founded the company Tasktop to, to experiment with this. So I then spent the last decade of my career in trying to apply these concepts and try to understand why that same feeling that I had of being in the flow of my work. And so engaged at both at an individual and a team level was so different to what I was seeing when I was traveling the world and learning to what's like what's like for these organizations and for it leaders out to try to manage these very large complex code bases and businesses that were trying to transform. And the main thing I noticed is that, that velocity, that I was feeling of delivering value in open source and with my open-source colleagues, that that was a hundred and something a thousand times slower than what I was seeing in the organizations with thousands and tens of thousands of developers.


So realize that something was really wrong in the way that we were looking at software and from all those journeys and lessons from all those amazing and very smart leaders, I actually came up with this notion of the flow framework on how we can bring this feeling to flow to our organizations as a whole and survive what's happening in terms of the, this era of digital disruption. So I'm going to go into some depth on the flow framer today. Tell me a little bit about the flow metrics and tell you a few of the stories in the book, project, or product. Now, the first story, this a, this, this was the one that really completely changed my world view. Uh, this was the first time I had had discussions with a CIO and this was when Nokia was trying to transform itself. So Nokia realized a year before we all realized that there's going to be a phone coming out of Cupertino.


That was going to have a much larger screen. And remember Nokia got very good at building these phones. That was, that was my phone up there with shiny beautiful stainless steel buttons, very elegant devices, uh, but with a very small and very limited digital experience. And they realized that in order to compete with apple coming onto the market where the phone was going to be all glass, it wasn't just about being able source glass capacity of screens. They were going to need to be able to innovate and software. The entire user experience was now going to be based purely on software with these devices. So it was just basically, you know, things were turning into glass rectangle. And so at a leadership level, at a board level, they realized back in 2006 that they needed to transform. We started working with them in 2007 and started apply some of the concepts that we learned in open source and connecting developers to agile tools on connecting the value stream at Nokia.


So I got to watch this firsthand and I got to see also that Nokia was becoming through this process, a poster child for how you scale agile. They brought in the best consultants, Dean Leffingwell was there the creator of the scaled agile framework, our Kent scraper was there helping them train everyone on scrum. And the entire agile transformation was actually being measured because it was such a key part, such a key thing to the business. So I remember when I was at the agile conference in Chicago and Nokia was up on every slide as the absolute proof point that agile works at large scale. And some of the ways that they were testing that agile was actually being adopted across Nokia, uh, was with the Nokia test create by Nokia Siemens networks, which basically checked if teams were using the agile tools. And if they were trained on scrum and what actually was happening is that while all this was going on, the business side was thinking everything was on track because of the way that they were measuring their agile transformation.


Meanwhile, the result was actually a local optimization of their value stream. So they were making things go agile in the middle, but nothing was actually moving faster end to end. The developers realized that I was working daily with developer managers and developers themselves interviewing them. And they realized something was fundamentally wrong. Someone as far as going to the board saying, we cannot do this. We'll never, we'll never actually meet these business objectives, but everyone thought everything was on track. And with this massive misstep and massive misunderstanding of something that the technical side understood that the business side didn't, our Nokia actually end up losing the entire mobile market that it created in failing to transform. So this is a profound lesson to me, as I realized that measuring the wrong things that measuring just this one silo, um, and using these proxy metrics of are people trained on agile or they're using my adult tool.


The one that we deployed completely derailed this transformation and ended up destroying this company. So I think nothing illustrates this kind of local optimization of your valley stream. Then John smart slide from this conference last year, where you can see, we are so freaking agile transformations on track, but meanwhile, if you take the customer's perspective, so for Nokia, they would need to innovate on new kinds of widgets on bringing an app store to market. They had no chance of doing it in a window that was meaningful. Now that was back in around 2007. And the concern that I started increasingly getting over the last few years is that not enough has changed. So as I worked more closely with large enterprise organizations, also very motivated and very driven to succeed in the digital transformations, I realized that there was, there were more fundamental problems. So this is another story of a top 25 bank by revenue.


It was on its third transformation. The first two were agile transformation. This one is the dev ops. So everyone was sure it was down to succeed. And even more interestingly, the budget for this transformation was approximately a billion dollars, an incremental billion dollars for this digital transformation. It sounds like a lot, but you'll actually hear it. See a bigger number in a couple of slides. Um, the thing I learned was that the way the transformation was being tracked was yet another set of proxy metrics of activities. Everything was being tracked. This project plan, which to me was completely foreign because coming from open source and from this product mentality, we always track results. How people were adopting, what we were doing, how much we were delivering, not this year-long plan of different activities and costs and budgets. And the amazing thing is that two years after the transformation was that two years into the transformation after its completion, um, things were successful in terms of the budget results.


In terms of the project plans being hit, everything was green, but that green was an illusion, right? It's this watermelon effect where in the project plan, everything looks green. You look inside and everything's actually red. You interviewed any person on the business side, on the UN or on the it side. And they were saying that we're actually delivering less than before. Because again, these proxy metrics, these cost centers and project management, as the guiding force connecting the business and the technologists were just fundamentally broken. So the bottom line is that there's something wrong there. And we now have companies on the market, they've been on the market for a while. They're doing it right? So if you compare the stock price change of this particular decade of Amazon, who has their product value streams aligned to their organizational structure and two pizza teams to their software architecture, microservices, you can see the basic creative disruption machine, the tech giants who've done this properly.


It's, they're just get to choose what market they go into next. Meanwhile, those other companies have long realized they need to transform are some of the ones on the right, who we've heard from at this conference were actually doing better than others, but things are still moving way too slowly. And they're actually moving so slowly right now, in terms of the large scale transformations that at the current rate of disruption, half of the S and P 500 is predicted to be replaced in the next 10 years. Whereas it's forecast to be replaced the next 10 years. Um, we're seeing this move from industry segment to industry segment. So even the banks, apparently aren't safe now. And this Bloomberg article from just last week indicates that banks have spent a trillion dollars on their digital transformation. But if you have refills of benefits, so we're talking about a trillion dollars I saw, I felt like I personally saw a billion dollars go up in flames at this one bank.


You take this collectively, it's a trillion dollars and very few are seeing the results. So there's just something very wrong with this approach because of course, all these organizations are very motivated. These are really good people with good intentions, trying to succeed, not trying to lose their jobs or have their companies decline in the process. So I got to wondering, as I was writing the book, I was actually in the middle of the book dis rate of turn this rate of disruption. The fact that we're all dealing with technologies changing and cloud runtimes and container orchestration methods changing, what feels like almost on a weekly basis is this normal is this is what we experience in our careers. Normal. And Jean pointed me at the work of Dr. Claudia Perez. So in 2002, Dr. Paris published a book called the technological revolutions and financial capital where she actually zoomed out for us to look at the last five technological revolutions.


So there've been five of these industrial revolution, steam and railways are still in heavy engineering oil and mass production. And what we're living today, the age of software and digital, and the really fascinating thing about her work is that she identifies and that in her models that each revolution has two distinct periods. It has this installation period where some new means of production becomes very cheap. So think back to the early 19 hundreds where Henry Ford created the assembly line and all of a sudden creating many cars became cheap at that moment, there's all this financial capital or this VC venture type capital that wants to make a huge return on that new means of production. And so in Detroit, and just that one city at that point, you had over 300 car startups, which we don't remember the names of today, but over time, then you giants who master that new means of production form.


And we have those giants today, those giants forum. And we ended up with this thing called the turning point where that just the creative destruction accelerates and accelerates the few companies like the Fords and the GMs, and the last revolution who've mastered this new means of production accumulate more and more of the world's wealth. And we're today at the point where the fangs of Facebook, Amazon, apple, Netflix, Google, Microsoft, as well as the bats. So Alibaba, Baidu and Tencent, their combined worth is the equivalent of the GDP of Japan. So those nine companies form a combined wealth of the third largest economy economy on the planet. And they're continuing to actually grow and disrupt further. So now what's happened is in past revolutions. Uh there's after the turning point, other there's a dissemination of technology and that this is why I think this conference and what you heard Jean say earlier today about this concert being, being for the horses, not just the unicorns and the tectonics who figured this out.


That's what brings about the deployment period where the rest of the world economy actually learned these new means of production, masters and scales them. This is actually where inequality in past periods has actually gone down and we get a much richer and wealthier economy and culture. The parents actually call these things the golden ages. So the question really is how do we collectively work to get through this turning point, get to the next golden age and make your company and your work, a key part of that. And so hint is that we're not going to do it with more project management because in each of these revolutions, we've come up with different managerial methods that are factory systems, subcontracting Taylorism Fordism. And each of them helped with that particular means of production. Now, as it turns out, project management came from Taylorism. It came from being, treating people as interchangeable and fungible cogs that a large machine, which you can actually do if you're building bridges or data centers, or really large, large buildings and things of that sort.


But Henry Ford realized it doesn't work when you're doing more complex work. He realized he actually needed to invest in his staff and Fordism came about. So trying to use Taylorism for creative work, 20 was project management. And assuming you have that much certainty over two years, multiple year projects is just a completely flawed approach to innovating, to surviving this turning point in the age of software. So to get a better sense of this, I actually went and visited one of our key customers and try to understand how that company, how the BMW group had survived. The last turning point and the age of manufacturing. They are one of the companies, the brands that made it, and not so much in terms of how they did the production itself, but how the business and production were connected. And it was an absolutely fascinating journey. Um, you heard a little bit from BMW yesterday, on terms of the it side.


This is purely on the car production side, where I went into their most advanced plant, the BMW Leipzig plant, where the one and two series car cars are made, and those cars come off the assembly line every, every 70 seconds. More interesting that I figured I ate there, electric cars are made there, and there was not a Ganter at insight, but what you did see all of the plant was just the complete embodiment of these lean principles to precisely specify value by product, identify the value stream for each product, make value flow without interruptions and let customer pull value from, from the producer. These are from Jim Womack's book, lean thinking and everything, but the plan embodied these, even the architecture, the building from space, you can see the one in the two series assembly line is very long. It's extensible. The buildings actually get extended as new manufacturing steps got added and it's completely automated.


The I eight line to my surprise was the shortest thing. And instead of having the 72nd time tack time, it actually the 30 minute tack time, the people working on it were generalists. And it was only the width of the building because this value stream was actually optimized for learning now for high throughput and velocity. And so now comparing car production and what we have today in enterprise, it, everything was up there and these integrated production lines and no notion of disconnected tool chains, you can't have disconnect in that kind of value stream. Everything was managed as product and price. They're connected to business value rather than projects. Everything was architect around flow. So there's not one big, large enterprise architecture that the shared. There was everything was architected around the flow needs of that particular value stream, not technology layers, not separating our, the persistence from the, from the data layer and so on.


And from the engagement layer, everything was optimized end to end. So we always start end to end with customer centric measures like lead time, which happened to be the best predictor of car company performance in the market, because of course they embody what the customer experiences, rather than optimizing and silos and looking how fast are we closing user stories and everything was measured first in terms of business results rather than these proxy and silo metrics. So the key thing I learned from this is that we need to apply these concepts of flow to software delivery, but to do that, we need to agree on the, what flows through software value streams. And we simply, haven't done that through academia in the industry. We know it's not function points and lines of code, but we need a common definition of what flows, what the flow items are.


And that's the key point of the flow framework. And remember flow is about what customers pull. They don't pull releases, they pull what's in the releases. They want your features. They want fixes to defects. They'll make them choose your solution, the features and the quality of it. Um, rather than a competitive solution, risks are now a core part of software delivery. So making sure that we have data privacy and security as key to the market and to our customers and technical debts are also a first-class part infrastructure debt, and resolving those debts is the first class part of software delivery. As we implement more features, we actually build up that's for taking shortcuts to get those features to market. So these are the four flow items, and these are mutually exclusive and comprehensively exhaustive. They're messy, which means these represent all the work that's done in your software value streams.


So if you're using something like the scaled agile frameworks, which has a very rich and refined taxonomy over of work items, then you might have dozens of those. Each of those work items actually maps into one of these flow items. So I'll give you a quick example now of what this looks like. And when I was first preparing these slides, Jean told me, make, you need to explain the flow items just with up and down arrows, because that's all executives will pay attention to. So this, this will explain everything purely through, up and down arrows, um, think back of your last big push to market and push us to market. And those, those releases that you're putting out there success is really dependent on the features that you deliver, right? Because that is what customers, what users consume as you do that. And there's the businesses wanting more and more of those features, debts and risks just rise.


Quality goes down. And then this interesting thing happens that defect works goes up and up to the point where you can get into a death spiral where defect work consumes all of the value stream. The business gets extremely frustrated, cause feature works. It goes down to zero. And this is exactly the situation that Nokia found itself in the developers understood why, because they were building on a burning platform platform that was brilliant technical debt, and that could never support announced app store or the Symbian operating system. They had to replatform. But of course the business, just one more features and just to prioritize more features, and this drove that company into the ground. Now here's another version of that story. This is in 2003, when bill gates wrote the trustworthy computing memo. So the SQL slammer thing happened, and bill gates actually said, we're going to bring fetal work down to zero across Microsoft value streams.


We're going to work on security. We're going to work on debt reduction, which no enterprise it company CEO has ever said ever probably is. We're going to stop all feature work now. And it's not that Microsoft didn't have competitive pressures. It just that bill gates understood that zero sum game and these trade-offs and knew how to steer Microsoft around that and focus on that work on, she couldn't replatform the way say Amazon has replatformed because their product portfolio is probably the most complex since it actually started the moment they just software started, they reduce debts and risks, and then of course, quality without fetal work or an app. And we now have the largest market cap company, the most valuable public company in the world, as partly as a result of this. And the interesting thing is that most other tech giants, most many other unicorns FACS had been through this.


They've had these near death experiences where their technical debt piled up, but they were able to steer their companies in the right direction. Amazons spend almost a billion dollars on its replatforming, but actually produced AWS. LinkedIn has had multiple replatforming and for each of the tech giants, the advantage I think that they've had is that all of them have as their CEO, a former software developer. So these flow framework concepts in these notions of these trade-offs have debts and risks and defects. They're just innate if you've spent part of your career building software, but we really need to bring some of that understanding of those concepts to today's business leaders. And that's really the goal of the flow framework. So I'll give you a quick overview right now of it and tell a few stories of some of these things that the flow framework has you track.


So the first thing you start with is you define your products, your product portfolio, these are internal products, external product platform products, and for each product value stream, you don't track how many user stories or issues were closed. You actually track each of the flow metrics. And so you've got this more abstract, high-level more business centric view of what flows, the flow metrics or flow velocity flow for since the flow of time and the flow of load, which I'll show you and correlate those flows to business results. So to some notion of value for an external product that could be revenue that could be active users, the cost you do not track costs for how much development costs, how much operations costs you track cost per value stream. So all of the people involved in that product value stream, even the hosting cost of that value stream or the cost of the value stream with those two things together, you can actually get to things like product, life, cycle profitability, you track the quality of the value stream and you track the happiness of the staff on the value stream.


So not how happy all your developers are, and I'm happy all your support. People are happy. All the people on the product value stream are. So you have that early warning indicator. If you've got an architectural problem, if you actually need to invest in architecture improvement for this value stream. And so I'll give you a couple of examples right now of how these flow metrics can answer the key questions that have been so hard for, especially for business leaders to answer and understand. So this is a story from DevOps enterprise summit in Vegas, this past fall that nationwide insurance shared. So they were wondering, we worked with them closely and we actually looked at this end to end flow time, why it was taking 120 days to deliver value to customers, delivering new features to customers. And of course the gut reaction for this is we need more developers and CEOs are always thinking, we meant we need many more developers, but when you dig into the flow time, as they did a few years ago, they realized only two and a half percent of the days spent delivering value to customers were in development.


So doubling the developers would have had almost no result. There were issues with upstream and downstream processes that needed to get resolved, and those were the bottleneck not developers. And so this is why the concept of flow time is so important rather than optimizing that two and a half percent for user story open to user story closed in JIRA. You have to take a more customer centric point of view and actually optimize the flow for your end to end value stream the way a car manufacturer does. Uh, as part of this, of course, we're always looking for where our bottlenecks are when you start thinking this way. And we start thinking in terms of value streams. And so this is one of my experiences, actually, that was quite interesting where we were hiring more and more developers because hands on keyboards and, and writing code, always to me seemed like the, the most important thing having come from that background.


And then we started realizing from what we were hearing while we dug into this, that we might have our ratio of developers to designers wrong because adding those developers was not producing more features at the rate that we thought, you know, you actually looked more close together. The fact that developers had Photoshop open, that was probably a, an odd sign, but I actually dug into this a little bit and I realized that we had a ratio of one to 40 of developers to designers. Meanwhile, companies like Atlassian companies like Dropbox. So they actually shifted one to nine ratio, one to six ratio because of how important user experiences and great user experiences are in today's consumer and in today's business applications. So by tracking flow efficiency, you actually learn where the wait states are. If developers are waiting on wire frames, you know, you might have a problem there and you start having the right conversations.


And that's how you actually do you do that by tracking flourish of fits and see, you see where things weighed in an end-to-end value stream. Again, not in a segment of the value stream, cause you could have different weights and different ones of your value streams. Then there's this other key one that I think a lot of you have probably experienced as well as why is delivery slowing as we add developers, this seems completely counter-intuitive, but it happens all the time as complexity rises. And this one's really interesting. And in the last few months of my trips of helping companies adopt the flow framework, this is actually one of the most common issues. This shows up a lot in Dominica to Candace book in terms of dependencies. But so often we see that value that architecture is the software architecture that we use are not actually aligned to the product value streams.


It's kind of the opposite of what you see at Amazon with those microservices and that loose coupling between their product value streams. You've got these enterprise architectures that were made to look elegant and be future proof in terms of extensibility. And it's the opposite of what you see in that BMW plant where the I eight has its own highly optimized value stream. So flow velocity, when you, for example, see, why does it take us so long to resolve risks? Well, maybe you should have a vice firm dedicated to your security framework because there's so much dependency on that security framework. And you've just asked everyone to implement GDPR. So one more story here. Why are the teams working on this particular product value stream, less happy than the others? And this was another kind of key lesson for me. Once you start measuring things like employee engagement or employee NPS for product value streams, you'll actually start to see this and you start to ask those kinds of questions.


And in our case, we noticed that it was a flow load issue. So the issue was that we were putting so much featured on this value stream. We were not allowing the developers to work on tech debt. They were completely overloaded with a feature backlog and we knew this. We were going to let them pay down the tech debt. We just didn't realize how much their engagement went down because there was, they felt like they were failing the company, which is exactly what happened at Nokia. We actually have now studied forensically the employee engagement logs from Nokia, and that's what was happening in play games was going down because developers wanted to fix those features and wants to help the company succeed. But they were completely overloaded with tech debt and the flow load would have indicated that as well as the value stream net promoter scores or other engagement scores.


So the question is, how do we do this? We have to start over focusing on the tools and the technologies, and really put a layer of abstraction over the tool network. So basically you make sure that we've got these flow items connected end to end so that we can trace them and connect them in these end to end value streams. And then from that generate these four metrics. And it's not as hard as it seems because once you've got those four metrics, you start getting these insights and there's simply a mapping and the modeling of the work that's happening across your entire tool network. So just as a quick example, our work at Tasktop, it actually starts a lot of it starts at Salesforce. It'll start in Zendesk where those customer requests come in. All of that goes into a product management tool, work gets done in JIRA.


Some of it goes to one of our partners, JIRA who help us with some of that work, but all of this has connected. And the really neat thing is once it's connected, once it's obstructed with these models, you measure these flow metrics and you can actually see the flow of business value and these bottlenecks. So here's an interesting one. And one of our products, we noticed that we had a very large feature backlog and we really needed to get through this feature backlog because is key to this product success. So as soon as you see that spike over there and flow load. So how many features that team has to work on and starts thrashing as they reprioritize things, the flow time goes up almost instantly. So it's now taking them way longer to get anything done. So what we did is we actually doubled the size of this team over here and flow velocity didn't go up immediately.


And then we got to really worrying because we realized that the core architecture was thing was, was actually built in Scala. It was our first experience with Scala and our VP probably development called Brian. I started wondering like, do we need to rip out the scholar or people way too slow to ramp up on it? And we actually thought, okay, well she'll probably move the scholar to things we know like Java and types type script. Then what happens to the team said, no, you know what, give us a chance. Let's, let's, let's give us a couple of sprints to ramp up. And we actually seeing the flow velocity rise and improve, and we realized we were wrong and that the scholar stuff was fine and that they were able to ramp up. But this shows you the importance of having that same common language. That's higher level of abstraction.


That's closer to the business value in terms of making it very easy for the team to show us that we were wrong. So really the goal here is to use these flow metrics as full rate work, this, this layer over top your agile tools and your DevOps practices to stop these local optimizations than basically having waterfall on the ends and agile in the middle. And to have this flow orientation where you actually measure flow, where you can stop using project management for visibility, and you actually have a direct visibility of what's flowing through your product value streams and what business results those are delivering. This is why companies like Google can actually kill products easily because they realize if they're adding a bunch more features, it's like getting adapted more. It's not delivering more business results. Well, it's time to kill that product that doesn't happen in the world of project management.


It actually has a really significant effect of how you think about budgeting. So in budgeting and project management, you bake in all the risks upfront. It's not optimized for any of the principles of DevOps of flow feedback or learning, because you're trying to pretend that you understand the whole life cycle of this product rather than learning through leases and through through iteration. Whereas in a product oriented mindset, you actually invest further when you see those business results happen. That's when you ramp up the teams, that's when you make further investment in the product, when it's incrementally delivering results. Another really important one is in a project oriented mindset. Again, that Taylor's mindset, you're bringing people to work and I'm still, it drives me crazy whenever I see this, but we see this all the time. You'll have developers assigned to eight, 10 different product values.


So it 10 different projects. And guess what? Their productivity, not that you see it in project management, but has tagged in the product oriented world, no developer or no team is assigned to more than one product value stream. They get to master the work. They get to master the insane complexity of the technology stacks they're working with today. And I think one of the most interesting and most impactful things that I've been noticing is the effects of product thinking on software architecture. So a lot of organizations are just putting a ton of effort on their business, their customer facing products. But as it turns out, if you actually look at the dependencies between your product value streams, you'll notice that your internal products, your platforms, your API APIs, your data pipelines, those things are actually a bottleneck for all those external value streams. And then you'll notice that your actual value stream network, the tool chain itself is one of your most important products because that's what makes all your developers productive.


So to tech times, this is kind of obvious sat in Adele said there cannot be a more important thing for an engineer than to work on the systems that drive our productivity today's enterprise. It's completely flipped. All the good developers are on the stuff on top, and you've got your complete stuff and stuff on the bottom. Tech giant completely flipped. The high-speed developers are on the bottom layer on the developer productivity as the value stream network. Then on the internal API APIs and platforms that enable everyone to build great software. And this is why companies like BMW realize that their most important product is not that I eight it's the factory. It's the plan. So in summary, disruption's going to accelerate. So we just need better business. We've I think we figured out the technical tools. We figured out the ways of agile and DevOps.


We don't get better business tools to survive it. And today's 80 practices of managing that value that we have two projects just won't work for the innovation that you need to survive. You need to shift from project to product at an organization level, and you get a shot to connect the business with technology with the simple language of the full framework. So I do hope this works for you to do that connected value stream network, define that product portfolio, track flow metrics and connect them to your business results into your plans. So the idea is that you go from projects within cost centers, the product value streams from silos and these proxy metrics that do straight to flow metrics and business results, and from fragmented value streams to an integrated value stream network. So you can learn more about the book over here in terms of the help I'm looking for. It's really doing a ton of learning and how this works in terms of changing your organizational steering model, connecting to finance. So any stories or wins or difficulties that you've had on that front, please reach out. And with that, thank you very much.