Las Vegas 2018

Project to Product - How Value Stream Networks Will Transform IT & Business

Based on his work with Fortune 100 enterprises, Dr. Mik Kersten presents a new concept, the Flow Framework, which unravels the enigma of enterprise software delivery, providing an approach that helps organizations better understand the human and system dynamic that underpins their IT infrastructure.


Dr. Mik Kersten spent a decade creating open source developer tools before realizing that programing was not the bottleneck of large-scale software delivery.


Since that time, he has been working on creating a model and tools for connecting the end-to-end software value stream.


Prior to founding Tasktop, Mik created the Eclipse Mylyn open source project as part of his PhD in Computer Science, pioneering the integration of development tools with the delivery pipeline.


As a research scientist at Xerox PARC, Mik implemented the first aspect-oriented programming tools for AspectJ.


Mik is the CEO of Tasktop and loves working with IT and thought leaders on transforming how software is built.


His book, Project to Product, is available here: http://bit.ly/ProjectProduct


Mik Kersten, CEO, Tasktop

MK

Mik Kersten

CEO, Tasktop

Transcript

00:00:02

Hello, everyone. I am just thrilled to be here today with all of you and sharing the story behind this book, project, the product. So my name is Mik Kersten for my entire career. I've been on this journey to help transform how software's built. And the first 10 years of that journey I spent as a developer, just writing a ton of code because I thought the problem was actually with the code, with the way that the code was disconnected from the value stream. And so I actually end up writing over a million lines of open-source Java code. That's still in use today in frameworks, such as aspect oriented programming that you might know from spring, uh, tools like the eclipse Mylon project that influenced how developer tools work and so on. And as gene said, what I realized is that if we do remove those impediments from developers and connect them to the users that they're supporting, uh, to basically the flow of business value, we do get the sense of focus and flow and even joy.

00:00:52

And I actually got my PhD in software engineering on exactly that topic. So when my supervisor and I realized this, we decided, okay, let's, let's try to scale this. Let's let's try and make this work for the very large scales. And we thought were better. Look for those very large scales than enterprise it. So we started a company called Tasktop to do just that. And I learned some very different lessons about what it's like to work in enterprise it, and that those kinds of scales, then it was like an open source. I actually noticed that a hundred fold productivity difference between how quickly open source developers. And I was managing the community of hundreds of what sort of contributors could create value compared to what I was seeing in these very large scales enterprise it organizations. And I realized the problem is actually much deeper.

00:01:35

And I think this morning, Jean gave us the inspiration that we can no longer target just improving the lives of 10 million developers. We now have to think of 40 million it specialists. And so I can quantify this past decade of my life, not with lines of code, but with traveling over a million air miles, which I can assure you is nowhere near as fun or rewarding as writing a million lines of code. But it did teach me a whole lot of valuable lessons by allowing me to meet with hundreds of it. Leaders who are struggling with this problem and struggling with this weird and broken layer that we've put between technology and the business, and to try to bridge that layer, bridge those gaps. I create this thing called the flow frame of that. I'll share with you today. And that's where framework is what's featured in project or product.

00:02:19

And the goal is to give us a common language between the things we've learned over the past 10 years of agile and of dev ops and bridge that gap between the way that the business looks at technology and looks at surviving and thriving in the stage, different digital disruption. So for me, the journey started at Xerox PARC. It's 1999. I just got my undergrad degree from the university, British Columbia in computer science and in anthropology. And I had this opportunity to work at, at Xerox park, which is amazing to me. This is the computer science lab, where they invented things like the graphical user interface, you know, little things like the mouse and our different programming was pioneered there as well. And I was just having the time of my life because I was getting to work on these super cool new programming tools that were supposed to bring business requirements, actually closer to the business to developers and express them in the code.

00:03:12

And I was actually working, started working basically 80 hour weeks and trying to do this because we had a thriving user community and I, I loved what I was doing. And then after some time the RSI started, so I got this really bad tendonitis. And then that progressed into some nerve issues in my arms. And I tried mousing with my other hand and so on. But at one point my boss told me that he'd seen several careers and this way, and that he suggested some, some paid time off. And this was the absolute, last thing you want to hear. I was having the time of my life building these tools and coding, and I was far from burnout. It was just, my body was not holding up. And I really then at that point, decide to figure out why this was happening. Why was it that I couldn't keep coding these longer hours with us having so much fun doing it?

00:04:00

So I started analyzing my own work and the most surprising thing to me was that it wasn't the coding that was giving me the RSI. It was actually all this thrashing I was doing with all these windows, which were invented at PARC. So it was felt a little ironic, but the windows and those windows had stack traces. They had tickets, they had instant messaging. There was email cause we were tracking user stories in the, at that point. And by the way, we'd done all our continuous integration, continuous delivery automation. I wasn't even thrashing that way. It was actually all the communication in terms of what we were supposed to be doing and how misaligned our software architecture was to that flow of value stream artifacts. And it's like, okay, if that's a misalignment, well, we can make a software tool to solve that. So that's why I did this experiment of creating myelin.

00:04:47

It actually did fix my RSI because it bridged this gap between what I was working on and the structure and the code. Other people seem to have this problem as well. So it started seeing millions of monthly downloads and it was kind of the foundation for my PhD thesis. And so I realized there was something really fundamental around this weird disconnect between the software architecture and the value stream. And that was my first epiphany that's featured in the book that these fragmented value streams kill personal productivity. In my case, that actually caused me physical bodily injury because I just kept wanting to code. In other cases, of course, you'll end up with burnout with people who are unmotivated because they just can't get work done the way that they want to. So now fast forward a little bit. It is now 2007 and that's what my phone looks like.

00:05:31

It was, I thought it was a beautiful phone. Uh, and it turned out that Nokia was, had started their agile transformation. So just like you've heard Jeffrey Stover say early this morning, they had realized that they were about to get disrupted. They knew about like the passive that screen, the large screen of the iPhone a year ahead of time. And so they realized that they better get better at software. Clearly this company was incredible at manufacturing, physical devices, but the screen was going to get so much bigger and going to involve so much software that they realized that software delivery would be their bottleneck. And they realized this in time they started adopting agile practices. They started hiring more developers. They did all the things that you hear companies doing today, and it appeared like they were doing it the right way. And I was invited to speak at the agile conference in 2009, about how to connect agile and open source.

00:06:23

And I was amazed because every room I went into, there was a consultant on stage saying how Nokia was a poster child for enterprise agile. But then I started working with Nokia and I realized that the truth was very different from the way that they were being described. Because what Nokia was doing was basically just the way that the business was looking at the agile transformation was how many teams were agile, how many teams were trained on scrum? So the measurement of the success of the agile transformation was this proxy metric. If you've seen Jeff Bezos letter to shareholders in 2016, he talks about proxy metrics, metrics of activities, not metrics of result. Now you've heard John smart talk about this as well. And this no get us for agility, basically tested where they were, whether they had optimized a very narrow segment of the value stream development.

00:07:13

You may have heard. Kevin Fisher mentioned this morning that when nationwide looked at their end to end value stream, only two and a half percent of the time was spent Ashland development. So now imagine Nokia could have hired 20 times that developers could have hired 2010 swipers to help them on scrum. And it would not have helped this problem. And as a result of this, that, and the business, not realizing this Nokia actually completely lost the market, the mobile market that it created. And this was really my second epiphany that the silos that looking at the value stream this end to end value some of bringing business value to your customers and measuring proxy metrics. Some of those proxy metrics are very important. Chain successor is important. Development cycle. Time is important, but it's not enough to make a transformation successful. We cannot measure these proxy metrics and the silos.

00:08:03

These are the things that destroy and derail transformations as we saw with Nokia. So now it's 2016 and we should have learned all these lessons, but I'm at this bank. This is a top 25 bank by revenue and they're in their third transformation. So the first two failed and very similar ways to Nokia. The third one was not just an agile transformation. There was an agile and DevOps transformation. So I thought, okay, this, this must have a better chance of success. And they actually were taking the approach of automation as well. So there was some fundamental positive differences. And the amazing thing was that it was such an important transformation that there was a billion dollar budget for this particular transformation. It's no everyone was pushing for making this thing work. Everyone wanted this thing to succeed. And then I realized that there was some that it was going to fail.

00:08:50

And this was two years ago, it's now happened. And I noticed that there was another very big silo. So there was this, it was doing the transformation basically within it. And there was a chief digital officers and those kind of people who were brought in, of course, they were creating these amazing dashboards for this bank. So imagine like this amazing car dashboards, because it was separated, those dashboards would never work in any of the regions that they were planned for. The software architecture was never going to support it. And the silo that was separating the business side, who was wanting to do the transformation and it was completely getting in their way in the way of this. So the transformation was being managed to of course cost because it was a project management transformation. And so, yes, of course the costs were reduced at the end of the two years.

00:09:38

In addition, after interviewing the people that next time everyone said the same thing, our ability to deliver value was also significantly reduced, of course, because it was just managed to cost alone. So this layer that was put between the transformation, that technology side and the business side completely got in the way it was the wrong lens, it's the wrong layer for this thing to succeed. And that led to my third epiphany, that project management and cost centers are completely the wrong model. If you want to become a software innovator, there are these things that completely derail these transformations, and we have to replace them with something better. And we have to do that quickly because other companies already have, so this is the stock price change between 2006 and 2016 of some retailers that you might know. Now, the retailer that's got that big bar, that 1900, 10% stock price increase that retailer Amazon has completely aligned its business strategy to what software products, to its software architecture with microservices, to its organizational structure with two pizza teams.

00:10:46

And they've completely out innovated everyone else in the process. Of course, some of that stock prices AWS. And there's more to the story. In addition, it's great to see that some of the companies that are, who are speaking here at this conference are actually making steps in the right direction. For example, Target's prices near its all-time high, but this is not just retail. We've got at this point, this world where some organizations have done this, they do not have this project management layer. That's assuming a high degree of certainty of what will happen for the next two years of the market. They've become software innovators and that the change is now so rapid between those who've managed software delivery at scale and other organizations that at today's turn rate, half of the S and P 500 will be replaced in the next 10 years. So this is happening and it's happening quickly.

00:11:33

And as I was wondering about this, I was wondering, has this ever happened before? Because I think for all our careers, we've been used to all this crazy change, then JavaScript, framers coming out every two weeks and so on. But have we seen this rate of change? And Dean introduced me to the work of an economic historian by the name of Carlota Perez. So she has studied technological and she studied the last five technological revolution sung with the industrial revolution, the age of steam in the railways age of steel and heavy engineering, a to foil mass production and where we are today, the age of software and digital, and a really fascinating thing that she discovered was that these revolutions come and have two phases. There's an installation period, and there's a deployment period in between them. There's a turning point. So in the installation period, there's a big frenzy because some new means of production becomes very cheap.

00:12:23

So you might have mass production. And this is the period in Detroit over a hundred years ago, where there were 300 car startups over 300 car startups in Detroit, and this is fueled by financial capital. So today that's venture capital. And there's another example of that. There's a conference on FinTech in Las Vegas today, there are over 2000 FinTech startups today. So these companies are mastering. The new means of production. And what's interesting is that this causes a tremendous amount of creative destruction, which has happened in every one of these technological revolutions and then some masters of the new means of production emerge. So we can now call those the fangs. So Facebook, Amazon, Netflix, Google can include Microsoft Alibaba, fd.com. And so on. These companies have mastered the new means of production production, and they're amassing great amounts of wealth. And I think the key thing right now is to help all the other companies in the world economy move into this wealth generation that happens when we have the dissemination of this new means of production.

00:13:25

So it really becomes the question, how do our companies survive this turning point? What did we learn from all those tech startups? What do we learn from the tech giants? What are they doing differently than enterprise? It organizations are still responsible for the majority of the world economy are doing today. And how do we have our company survive this mass extinction event that happens in the turning point because we are currently in the middle of that turning point. So if we look at those revolutions, each of them has brought with it, a new managerial discipline. So there've been new financial systems discovered, but things like factories have some subcontracting than Taylorism and then four dozen. So what's so fascinating. And from conversations with color Perez, this became clear is that Taylorism is, is actually where we learn how to do really interesting, amazing projects. The Hoover dam, not far from here that was created with Gantt charts, which is an amazing thing, but you could not create an economy of mass production that way, where mass production actually manage lean methods and products and value streams.

00:14:27

And so what I've noticed is happening to these companies that a lot of them were actually founded and established before the start of the aid of software before the seventies, we're still using management techniques from two ages ago to run our businesses and our strategies. And so I want to look at whether this is, how did this happen? How did companies mass production companies survive the last turning point? And I actually looked at a company that we've worked with very closely. So BMW they're clearly one of the masters of mass production and they made it, they made it through that last turning point. What did they do? How was their business connected to production? And could we learn anything from that? So I had this amazing opportunity to visit the BMW Leipzig plant, where this is Renatus strata on the right. He likes to be speaking with common Diardo tomorrow and can tell you more of the story and more of what BMW is doing in their agile transformation.

00:15:21

But the thing that was so amazing to me about this plant is that it's got that complexity, not only the architectural elegance, but the complexity of this plant is tremendous. So there's a new car delivered off the end of the production line, every 70 seconds. And you want, or two series model, and those cars are actually delivered in the same sequence. They were ordered. It's called Justin sequence delivery with an ecosystem of 12,000 suppliers putting in parts into those cars. And of course, many suppliers providing the different robots and parts on the, on the production line. So this actually, we don't have the excuse of complexity enterprise. It, this is somewhere near that same scale of complexity, but no one there is talking about project management. They're living the lean principles that here are listed from lean thinking. So those principles are principles are precisely specify value by product, identify the value stream for each product, make value flow without interruptions and let the customer pull value from the producer because in the end products are about customer poles.

00:16:25

That's what we're meant to deliver. And so in this plant, for example, everybody knows where the bottleneck is. The CEO knows what the bottleneck with the limiting factor of the planters. Meanwhile, in all those meetings I did traveling. I asked many, many times what is the bottleneck of your software production? And it executives or business leaders will just give vague answers or a blank stare because we're not even sure about what, what the units of production are in software in terms of the way that the language of the technology side and the business side and BMW is so amazing with architecting everything around these value streams, you can actually see the value stream architecture from space. So this over here is the one and two series building. So this is a very long high automation line because it's again, delivering those cars every 70 seconds, no manufacturing step can take more than 70 seconds.

00:17:16

The cars weave in there, and this is extensible. They can add new manufacturing steps by extending those buildings. BMW did not know how much demand there would be for their electric series. I, through RIA, to me, they've very configurable line that I ate the time to create an aid is actually for each step of production, 30 minutes, there's much less automation than in one or two series. So everything is configurable as these value streams and the business understands the product lines and the value streams. And let's compare this to what we're doing today with an enterprise. It in car production, we've got these beautifully integrated production lines, not these disconnected tool chains. Everything is mantas products, not as projects everything's architect around flow, not technology layers, front ends back ends and so on. Everything's architecting on these product lines. Everything's optimized end to end, not optimized in silos and everything of course is managed by business results, not these proxy metrics of one aspect of the silo.

00:18:13

So if we look at business value flow, because fundamentally I think that's the problem. We've not agreed upon what production, what productivity is and software delivery. We don't have a common definition. If we look at business flow that BMW plant it's about quality cars that deliver sheer driving pleasure. Interestingly, those cars are designed in yearly cycles, even though they're delivered every 70 seconds. So there's a difference between what they're doing and what we're doing, but the creative and the manufacturing processes are decoupled. They're actually in separate buildings. And of course, all of this flows across a linear line. The line stops, everything stops. That's a little bit different than what we see in it. And technology and software. We delight our customers, not with releases because those should happen continuously now. But with new features, they're pulling new features. They're pulling new value. Our customers are, are those things are designed sometimes on a daily basis, weekly or daily basis and the creative and the manufacturing processor wants.

00:19:08

So that's a pretty fundamental difference. And the flow is not linear. It's not a batch process of delivering a feature. It's a flow across a complex value stream network. That includes all the specialists, designers, developers, support staff, and operations involved. So the flow framework is to answer this core question, that what flows and software delivery, we need a common definition of productivity and a flow. And it's these four units features that's new business value. That's pulled by the customer defects quality improvements also put by the customer risks. So the reduction of risk, security, availability, compliance, and so on posed by people like risk officers, debts, technical debt improvements. And these are pulled by architects, by developer and so on. And these four flow items are mutually exclusive and comprehensively exhaustive, which means they represent all work that's being done. You can have these very specific taxonomies, for example, in the scaled agile framework, which has a very, very nice taxonomy that we use all of those types of work items map in to these four flow items.

00:20:09

And so I'll give you an example of how we can use these to express a common language between the business leaders and the technologists. So this is, imagine your last big push to market, of course, with a big push to market its features. It's your ability to deliver great features to your market, to your users. That's probably going to define the success of that product as you do that without push the features. There's a cost because it's a zero sum game between those flow of those four flow items, which means that you take on additional debt and risks and the result quality of course goes down as you do that. And defect work goes up and up and up to the point where he can easily get into this death spiral of been there more than I care to admit when a defect works, keeps rising, this is where we can actually get the burnout.

00:20:57

Developers want to be delivering features. They want to see the impact of their work on your customers. But of course they don't get to do that. And that's where we got, we actually measure as you'll see things like employee net, promoter scores drop as less than work done. And this is exactly the situation that Nokia got itself into. And just as a case in point, this is by John Cutler. This is the kind of things that, of course we've all experienced in 2015 have referenced features at 15 to 30 days in 2018. Comms is probably now bigger, 150 to 300 days. And of course the tech technologists understand why this is happening. They know they need to invest in technical debt reduction. The problem is the company, the organization, the business is not understanding that. So now contrast that to the security standard that Microsoft witnessed in 2003 with SQL slammer.

00:21:43

So what bill gates did is sent this fascinating memo back in 2003, before securities front page news and said across our value streams, we're going to basically drop the feature work. We're going to reduce our debts. And then the debts were reduced and micro. And this was of course the trustworthy computing initiative. So as a result of that, Microsoft assured its place in the future. And the way it's been able to do things like move to cloud, where in the end, they're going faster than ever and more securely than ever. So this is exactly what Nokia at a business level failed to do.

00:22:21

And the whole goal of the flow framework is to provide this common language. The thing of that, of course, the tech giants who have CEOs who were previously software developers already innately understand, but to make this more accessible to our business leaders and to come up with a common language and framework between the technologists and the business leaders. And so I'll just give you a really quick overview of that. So the idea of the flow framework is that you've got this flow distribution and you're measuring for flow metrics. How many of each of those flow items you delivered for a sprint, a release a year and so on the efficiency of that delivery, because we can measure that in the value stream network, the time, how long it took for a feature or fixed to get to market and flow load. If you're familiar with Dominica, the grand, this book or Donald Reinertsen's work, we know that putting too much load on a value stream causes things like burnout and actually decreases productivity.

00:23:13

So we've got a measure for that. And the key thing is you do this for each value stream. You don't do this for operations. You don't do this for one feature team. You don't do this just for a scrum team. You do this for a value stream, which tends to be bigger than feature teams. And you correlate that to business results that have to be specified for that value stream. So those business results can be revenue. If it's an internal product that you're doing that the internal product value stream such as an internal developer platform, API, that can be an adoption metric. You measure costs, but you measure cost for the value stream across all the specialists involved in building a value stream. You measure quality and you measure happiness, but you measure happiness with measures like employee net promoter score for the, all the people working on that value stream.

00:23:56

Because then you'll have an early warning indicator that if you put too much load on one set of teams, you could cause them to burn out and cause developers to start leaving because they're so miserable. So, or you didn't give them the chance to reduce technical debt. So this is just a high-level sense of the flow framework. And this is how you'll get to end of life. All those products, project management, things go on forever. If you see business results dropping while you're adding more features, maybe it's time for that thing to die. And on top of this, we can later on these metrics that are covered in the books that just flow efficiency. So for example, we measure end to end flow time, rather than again, just looking at development cycle time, where code commit to code deploy. We really need to take the customer's perspective of flow, which has to do of course, with the end-to-end activity.

00:24:40

Ever since that feature was submitted, that defect fix was submitted. We measure there and we can create these dashboards using those flow metrics. The key thing being those dashboards are all for each value stream. So you've got this level of visibility. This won't tell you if you bring the right features to market, but we'll actually show you when you've got a bottleneck. When you've got one of your time fees, such as too many dependencies between a value stream slowing single one of the value streams down. So the question becomes, how can we do this? And how can we really go from silos and proxy metrics to this connected value stream network that we can measure that we can optimize, that we can manage and that we can see. And the idea here is that you basically take all these different tools and create a tool network out of them.

00:25:28

You stop thinking about individual tools. You create this abstract model on top of that, that you can actually measure and you can measure cross tool flow. On that model. We call this the artifact network that sits on top of your tool network. You can then measure flows across that network and you can measure those flows and map them into your product value streams, and measure those flows and those product value streams. So you can go from this waterfall world of projects, where again, you're assuming that you know exactly what you should be living for years on end to this flow orientation, but measure the results of that flow orientation. You can go from this world where everything has to be specified upfront to what some very innovative companies, including tech giants or things like what we're doing at Tasktop, which is you do create a product development budget, but you allow the reallocation of budgets between product value streams on a much shorter timeframe than a year so that you can adapt to your market for wherever you are getting those business results.

00:26:26

And of course, this key thing is that you stop bringing work to the people, to the work you bring work to the people. So you allow those teams to form stable value streams that are bigger than just feature teams. Feature teams are a great start, but you give the entire value stream, the ability to have autonomy, mastery and purpose, and you allow them to set the flow distribution. Those teams, the product value stream teams will know when they should produce more tech debt, for example, to reach a north star, whereas where they should run faster on feature delivery. So just as a really quick example of how we do this at Tasktop because the flow framework has absolutely changed how I look at managing software delivery. Of course, we just connect our value stream network. We've got all our customer requests, similar to BMW, where they're coming in from customer orders.

00:27:09

Ours are coming into Salesforce. They're being going into product management tool with target process then to JIRA, which is connected to contractors, zeros and to our service desk and so on. But we've got this abstract model on top of that, and we can measure that model. So here's an example of the, of a live dashboard they look at on a weekly basis and what you can see here across these different product value streams. So we've got almost a dozen of these is that you can actually see something I did kind of wrong and I blame Dominica. The grandis for joining Tasktop too late. So I set a north star a year ago of feature delivery because I knew the more features we delivered, the more successful this new product would be. And so I actually put this on, on the strategic plan for the company, which means that some people in management were bonused on this, but I did that deliberately.

00:27:59

I knew we would pick up some more technical debt and that we would then pay off that debt once the product had succeeded in the market. So you can actually see that green bar just above this red box. There was a lot of flow load on this team, and there's a predictable rise in deaths in defects, of course, and we had to do more defect work and it took more time. So I kind of understood that was going to happen. I did not quite predict how much that flow load would affect the happiness of this team and the fact that I'm not even showing complete communicated to them, that they'd be able to reduce it at some point in the future, even though we were thinking it, and this actually, this, this became a real problem, but now I can now see the problem. I could also see that the team who was unhappiest the year before, who actually were given the time to rearchitect their continuous delivering and their entire test infrastructure is now the happiest team.

00:28:49

And so being able to see this, this is fine when you're like us, I'm looking usually at six of these, imagine this across a portfolio of hundreds of internal and external products. This becomes a really big deal in a really important tool. So my advice to people kind of more on the business side is ensure that you're defining this product portfolios and these product value streams, and that the metrics are being tracked and empower the delivery teams themselves to allocate flow distribution. They're the, they're the ones who know when they need to reduce tech debt. You just need to listen when they need to be given that chance. And again, this is why we need a common language. This is why the leaders of the tech giants, no one to listen to their delivery teams. Sometimes you'll actually need to do things definitely to hit a north star.

00:29:33

For example, you might need to do a major risk reduction, a major investment and things like GDPR. And you need to accept that. That's going to take from the flow of say, feature delivery to technologists, create your value stream network by connecting all those tools. Again, abstract over that tool network, define your value stream architecture. This is as important as your software architecture. That's probably one of the main things I've learned and then use those flow metrics to identify bottlenecks. You can then dig in and see, okay, is the problem that this team is actually there. They're testing pyramid is, is inverted, is the problem that they haven't they've said they've done it. The production done done their CIC pipeline properly. This will kind of be your early warning signals or you'll look at where to extract the platform because everyone's got dependencies on this one internal component or a team so quickly, the ideas that we go from project and cost centers to these product value streams from silos and proxy metrics to flow metrics and business results, fragmented value streams to this integrated value stream network.

00:30:34

And so the book will launch on November 20th, I'll be signing some copies of the book tomorrow. We're trying to document some more best practices around this@flowframework.org, or you can follow the book project to product.org. And then the main ask I have is that Jean and I have been trying to understand how best to present this to CEOs, to our boards, to our executive teams. So they can understand where to invest in helping make us all successful in bringing our companies through the turning point. So Jean had that cool idea of those system dynamic diagrams that you saw. If you have any ideas, how we can better presenters to your organizations, please get in touch and let us know. And thank you very much. I hope.