Now What?

Host Intro (Gene Kim)

All right. Welcome back. We have a bunch of kickass talks this afternoon, and our first speaker of this segment is Justin McCarthy.

So I was introduced to him by a mutual friend of ours, James Cham, who is currently a VC at Bloomberg Beta. James was actually on our board of directors back when I was the CTO of Tripwire back in the mid-2000s. And so James said to me, "Gene, you have to see what Justin is doing." And I'm so glad he made that introduction.

So Justin, at the time, was co-founder and CTO of StrongDM in the information security space, where correctness is not optional. And here was someone who was obviously thinking very deeply about what happens when coding agents write the software and about software factories at industrial scale. In July 2025, well ahead of the conventional wisdom, he and his team wrote their last line of code. And in an information security space, that's a heck of a claim to make.

Earlier this year, after StrongDM was acquired by Delinea, Justin threw himself entirely into the next thing, which is Diffusion AI, where their mission is to help change how teams of developers work in large, complex organizations by taking them out of their usual context and immersing them completely in this new way of working. So here to teach us what it looks like from his area of the frontier is Justin.

Justin McCarthy

(walk-out music: "Everything Is Awesome")

Everything is awesome. Everything is cool when you're part of a team. Everything is awesome when you're living out a dream. All right.

I know pretty much all of us say this every day, but what a time to be alive. Everything is awesome. I don't know — best walk-out music possible.

Okay, so the title of this talk is "Now What?" And as Gene mentioned, a year ago, not writing code and not reading code was a little bit ahead of the zeitgeist. Conventional wisdom has caught up, and I think maybe we're all seeing a pretty similar shape of reality in what's possible now. But gosh, given what's possible, there are still so many questions that we don't know the answer to — in terms of how we shape companies, how we shape teams, how we set priorities, all that stuff. So we're going to talk a little bit about that stuff.

It was a little more than a year ago now that we wrote on the board: "No human-written code." That was essentially my day one of building a software factory. And our day two of building a software factory was that second realization that if we weren't writing the code, there was simply no way to ingest, understand, and ultimately review and approve the code that was being written at machine scale. So fairly early on, I dedicated myself to that question, and found that there are answers. You can industrialize a lot of these review processes. You can have a mapping between your intent and what you're seeing in the product. It is achievable.

But I do want to just spend a minute on really the essential part of making that achievable. This will be familiar to most, so maybe this is more of a confidence-boosting speech for us — that really this starts with validation. So we're just going to spend a few minutes on this, because if you don't have validation, you don't have anything.

At Diffusion, the way we say this is: you only have what you can validate. And essentially everything else is just a rumor or a claim. All the agents will think they did the best job possible, had the best idea, until you ask another agent whether that agent did a good job. When you compound this together in a system, using engineering, and you design that system, it is possible to validate at high throughput. And really, we've heard from many speakers at this conference that that is the center of their new production process, and I think that's totally appropriate. So just a few minutes of encouragement on validation.

---

So what you're looking at now is a product you might find if you bust open some sort of gear, some sort of equipment. In this case, these are maybe in a transmission, or a reduction gear inside of some sort of robot or car or something. But you can kind of tell this is made of plastic, and probably the process that created this is one that some of you may be familiar with.

We heard from Art yesterday. We heard from lots of folks with manufacturing metaphors so far, so I'm going to also use a manufacturing metaphor.

Some folks will recognize this process that's up on the screen right now — it's animating. This is the process whereby we convert something with extremely low intrinsic value — in this case, little pebbles of plastic — into something that has a much higher utility. In this case, some useful gears and cogs. Those pellets, if I gave you a 50-pound bag of them, it would just be a burden. You wouldn't say thank you, right? You might say "thank you," but you wouldn't be happy. On the other hand, if you need some gears, this is the way you get them.

But if you consider this whole injection molding system and this whole process — this rig is obviously where the engineering went. But another way to understand this is: this is the validator of whether the plastic is in the right shape. So the ratio of the little pebbles of plastic — maybe those are our lines of code. Two years ago, three years ago, we thought those little pebbles of plastic were the important part. We thought those lines of code were the important part. Now, I hope you're all seeing that the rig — that's the important part here. That's where we're putting our creative effort.

So this injection molding system that makes all these lovely gears is mediated by this very, very precise — in fact, leak-proof, plastic-tight — mold. And for anyone that's done a process like this before, you'll again realize that so much of the design and so much of the cost of production goes into this. So that mold — that's what you're trying to build. You're building molds for your intent, and then you're flowing the code into it.

And if you take that mold away, what do you have? You just have Silly String. You have a spew of mold and plastic, and that's, again, not valuable. Nobody wants that. It's just a burden. You just have to clean it up. So we don't want plastic slop. We can't have product slop. We need molds that validate our intention.

---

There are other disciplines that have also felt this before. You have a printed circuit board with very fine features, or you have very fine features on a piece of silicon. How do you know you got it right? The way you know you got it right is with test equipment. The way you know you got it right is with different kinds of validation at different scales, different layers, with different processes.

I love this video because this is an illustration of a flying probe test, again on a printed circuit board. It's doing connectivity testing between different points on the board. You can't see it in this, but this little jig is moving super, super fast. And if you think about all the engineering and all the ingenuity that went into building this flying probe machine — this is something that in software, when we think of our CI/CD system, when we think of our DevOps process, maybe we're excited about it. We've heard a couple of people say how proud they are of their pipelines. But it never quite looked as Rube Goldberg as this. It never looked as embellished and specific as this one does.

Yet our mold in the injection molding process and our flying probe rig — these are very clear upfront investments that we've made to achieve high throughput with high reliability. So this feeling of what we're producing — maybe historically out of the lines of code — versus how we're validating it: I just want to help folks accept the reality that we're probably going to be spending one or two orders of magnitude more tokens on the outer compared to the inner, the actual code. That should feel normal, and if your teams are not feeling that yet, I would encourage you to talk them through this process, because it's just inevitable. We have to spend so much more on validation, for as long as — really for this period here where the agents are out front building for us.

And again, there are other companies that have gone before. In disciplines where you can't directly inspect the reality of what you're producing, we built up huge logos and huge companies associated with test and validation equipment. A lot of these occur in electronics. A lot of these occur in bio. In these disciplines, you can't inspect a cell. You can't inspect a feature of silicon with your eyes. You can't touch it and know if it's right. So you need test equipment to amplify up and down your intentions and match them to the actuals.

This represents actually a trillion dollars of market capitalization. On the bottom of the list there, you'll see the largest ever dedicated software company to software testing and quality. So there's a trillion dollars essentially of validation kind of missing from the market around software. And that just gives you a little bit of a feel for the magnitude of what we're going to have to build with tokens in the coming years. Maybe folks in this room will be doing it.

---

Okay, so that's validation. In my view, that's the hard part.

The rest of a software factory — I don't know. It's just a loop. A graph is also just a loop. A YAML file can also just be a loop. All of these, I think, are not all that different from a loop.

Where you measure your loop, how you implement your validator — these really are the art and the science of creating something that you're satisfied with, that can achieve expectations for your customers, that can exceed expectations for delivery speed and quality. And I'll say that validator piece — that is the technically hard piece. But going up to where you're measuring the loop, ideally you want these measurements to be as close to customer value as possible. Ideally, I would want to say revenue should be the outer loop measure. You might have to find a proxy for that. But the closer you get to the customer, the easier the conversation is around: have we designed the loop the right way? Are we validating it correctly? So the closer to customer value, the better. I think we've heard echoes of this previously in the agilism, about what is actually customer value that we're shipping.

The other words I would say about your software factory: you can't give up. There's a kind of technical stamina that's required. You will not succeed the first time. You will not succeed the second time. You just have to loop that — you just have to keep at it.

And you can test this for yourself. If you can simply take the output of any process and paste it into a frontier model, and if it still doesn't understand, then it's beyond the model's capabilities. But if you can find those tokens anywhere in your enterprise — anywhere in Slack, anywhere in SharePoint, anywhere in Drive, anywhere in a log file — and you can paste some composite of those pieces yourself, and you get the right answer, then you didn't need to do that hunting and copying and pasting yourself. An agent could have done that for you. And all you need is that single proof to yourself that this was possible, to sustain your conviction that you should keep at it.

So: design a loop, take validation extremely seriously, and just don't give up.

---

There are a number of points at which I could just walk offstage and that would be the recipe for a software factory. This is one.

There's another company — a fairly well-known company. I don't know if anybody saw this when it came by on X the other week, but this is essentially the software factory that's going on inside of OpenAI right now. Maybe the outer research is a little bit separate. Maybe the researcher process looks a little bit different from this pipeline. But for the average feature that's coming through in Codex or ChatGPT, it's subject to approximately this. And as we know, they're subject right now to a decent amount of load and scrutiny and customer expectations and so on. And they seem to be shipping a lot, and it's not bigger than this diagram. Apologies if it doesn't read well — you can find it on X, and you can find it in the slides after. And this is really working in a pretty high-consequence environment. I think we can all have loops that look like this, and they actually don't need to look particularly more embellished than this.

---

Okay. Another recent speech that made some waves — we heard that we should put down our pencils. And I think this was well-timed. We have to be doing everything differently.

One way that we think about it is to acknowledge that while there are many things that the models cannot do today — and I have my list of things they can't do, which is of course shrinking over time — if the model can do it, it's really important to break the habit and just stop doing it myself. So again, back to the copy-and-paste example. If I can copy and paste your Slack message or email, and a block of code, and a database record, and some output from a log file, and it solves the bug — then I need to build a loop to connect those sources, connect that context, and do the context engineering for me. That's no longer human work. That's agent work.

---

Okay. But thankfully, there's still a whole bunch of stuff that's super hard.

One early thing that's still hard is understanding what to build. And obviously all product designers, all product management teams are circling this question right now.

I'll just say there's also no answer. There are some disciplines that have the luxury of very specific, very concrete specifications. Up there is the molecular design or specification for GLP-1 — so if you can simply print this out on your 3D printer or whatever, you can make your own GLP-1s at home. The intent and the specification is all in there. That's not how software works.

The way software has worked historically is we put everything in a document and we called it a PRD, and that was the truth, maybe. Obviously, that is changing right now as well. But we still really do have to fix the faster horses problem — the problem where if we ask a customer what they want, they might say something that we should not build. This problem doesn't go away, and the way you ask questions and the way you listen — absolutely still out there. And the models aren't solving it on their own.

So this is a lot closer to where we should be working in our day-to-day, and let the production, the review, the operations loop.

---

Okay, so now onto the actual day two "now what" part of this.

If you accept that we can have loops, if you can accept that those loops can produce high nines of reliability and high quality, that they can produce according to your architectural standards, and that they can produce even according to your compliance obligations — I make all of those claims — then we still have a "now what."

How do we roll this out? So I'm just going to share the first — I think seven — patterns that I've observed, keeping in mind that this is just this year. The seven patterns that I've observed as teams begin rolling out software factories, as they begin industrializing the production of not just the lines of code but many of the loops and processes around the delivery of that code.

---

Pattern 1: Very new roles. None of them have names yet. But here are a few that are just barely forming in my experience.

There's some version of a cook, and if you have too many cooks in the kitchen, the kitchen is not going to work. If you have too many drivers of the vehicle, the vehicle is not going to go the right way. So reducing cooks and drivers to a ratio that is commensurate with the problem — so that one person can be driving and all the rest of us are helping in some way. There are the scouts that are somehow going out ahead, finding the context, finding the dependencies, and then bringing that context to those drivers. This is something that is absolutely an architecture that scales. But we don't have great names for these roles yet.

When you go back home, figure out how many drivers you have. If you have N folks on the team, do not have N drivers. Do not have N cooks. Have less than N. Find other things to support those drivers.

---

Pattern 2: Offense and defense. This is an extremely powerful one. We don't have a great name for this yet either.

Any of us — we pair up on a project. I'm going to take the first week and go on offense. So I'm going to expand the ambition of the product that we're building together. And on that week, you're going to play defense. I'm going to be throwing ideas at the product; you're going to be defending availability, integrity, brand against my onslaught. Next week we're switching. Then you have to have the creativity about how to expand ambition, about how to satisfy more customers. And I'm the one saying, "Uh-uh. I've got five nines of reliability. I'm not giving up any of that. I've got compliance obligations. I can't give up on any of that."

This develops empathy for both the creative process and the defensive process — the bull and the bear. You can have them on the same team on the same product, especially if you ask the same people to do it, but not on the same day. Every other week, it works.

Play offense, play defense against your product. Never ship slop, and never stop having ambition.

---

Pattern 3: An agentic estate. An agentic estate is one where the agents can move around and navigate. This very much applies to code, but of course it also applies to everything else. So please think about all of your surfaces — they have to be navigable by the agent. If you make it navigable for them, it might mean the code base needs to be shaped differently. It might mean your mono repo needs to be carved up differently. It might mean just when you got finished deleting all your microservices, you're going to start expanding microservices. I'm sorry, but it needs to be navigable.

---

Pattern 4: Shift way left. Another way to think of this: please, just get out of the building. Whatever your building is, get out of it. Go closer to the customer. I was having a conversation earlier in the hallway — get your product managers out of their chairs and make them sales engineers. Get your sales engineers out and make them sellers. Just shift way, way left. Get closer to the customer, ask more questions, and listen more.

---

Pattern 5: Have great examples. Through viral competence, you can have powerful examples of teams that are going faster than all the other teams. And that might actually be an easier problem to solve than trying to universally roll out a software factory across your entire enterprise. If you can get one thing working, and then through virality show "do it like this team" — you in fact don't even have to point at it. The team just has to be performing in a different way. Everyone else is going to notice, and they're going to seek to emulate that team.

---

Pattern 6: Denial, backsliding, and stasis. Unfortunately, the opposite of that is denial, backsliding, and stasis. This is something that is inevitable. But we all have to be aware that this is going to happen, and there are remedies for that as well.

---

Pattern 7: Don't wait. The final pattern — and this is like a refrain that we use constantly at Diffusion — is don't wait. That is a profound encouragement to not wait for permission, not wait for a vendor to deliver their deliverable, not wait for the API to be finalized or even the standard to be finalized. We can actually compute what we would do regardless of how the standard is finalized. We can compute what we would do regardless of what the data looks like when it's delivered from that dependency or that vendor. So just implement for all of those cases and be ready to go now. Be ready to ship today and just say, "Oh, as soon as that team plugs in the batteries, the machine will be on."

---

So just a small forecast.

I really like the refrain that we've been using at this conference the last few days of "I don't know." And I think it's good to acknowledge that none of us know. But there are at least a couple of things that we do know. We do know that this is the worst the models ever get. We do know that they have profound capabilities today.

And there's something else that we know: we can produce a lot more software. It is not a foregone conclusion that customers can, or want to, or ever wanted to digest 20% more, 50% more, 100 times more software. They're probably full of software. So we are right now in an overproduction of capability, and if we don't resolve the demand problem, we're going to end up just working on the cost side of the P&L — and that's boring, and it lacks creativity.

So everyone in this room has an obligation to figure out your demand factory, even if historically that wasn't your department. At Diffusion, there are a lot of folks from technical backgrounds. We refer to each other as members of commercial staff, not members of technical staff, because this is all of our problem. We just did this to the production function, and we need to do this to the demand factory. But there's no such thing as a demand factory yet. Nobody's coined the term before. Nobody knows what goes in a demand factory. We have to find out what goes in a demand factory, like now.

We need those signals to come in high quality. We need high signal-to-noise on demand. We need flow all the way through to deployment, and that means we have to have more customers. We need more share of wallet within those customers. We need more SKUs to sell. We need more lines of products. And then ultimately, if you can't get all of this digested, you need to just acquire your competitor. You need more of everything in order to rate-match our production to demand.

---

All right. So Gene encouraged us to do a help wanted. Anyone that's struggling with their transformation, please send them to me. Any aspiring factory architects, please come. Anyone in this room, please come to visit. Oracle Corporation, the database company, moved out of their headquarters in Silicon Valley — we moved in. And anyone with a base in Europe or India, please also come see me. This diffusion needs to happen globally, and we're very serious about expanding globally this year.

All right — 15 seconds left. Thanks, all.

Gene Kim

Thank you, Justin. In fact, we were talking to somebody yesterday in terms of demand factory who was saying that because of all the huge productivity increases in development, they're actually making outsized increased investments in marketing and sales — which I thought was exactly what Justin was suggesting in terms of, oh my gosh. There used to be a point in my life where I only believed that engineering wins only if marketing and sales loses, but apparently that is no longer true.

All right. Thank you, Justin.