Our Journey to Agentification

Host Intro (Gene Kim)

Welcome back. Okay, our next speaker is John Rauser, Senior Director of Software Engineering at Cisco Security.

In April, he presented the story of how last summer he convinced his SVP to require 100 of their top leaders to vibe code something and put it into production in the quarter that ended one year ago. And when I heard about this, I was like, "John, I want in." Can we survey each one of the 100 leaders to understand which leaders finished, who didn't? What were the factors that enabled them to succeed or not? But more importantly, for those who succeeded, how did it change their leadership behaviors?

And it was such a cool project for a whole bunch of reasons, but there's two findings I want to share with you.

One, the more senior the leader, the more fun they had. And the more fun they had, the more likely they were to succeed.

Number two, 85% of the leaders who finished and put something into production changed how they led. They became better at four things. One, they elevated the bar of performance — they learned how you can do things in days, not weeks. Two, it was just like what Dylan from Mastercard said yesterday: if I can do it, you can do it too. Number two, they shifted from being a problem solver to a direction setter. They knew that they were becoming the bottleneck, and they had to enable their reports to become more autonomous and make more decisions by themselves. One of the survey respondents said, "My thinking about leadership shifted from 'I'm the decision-maker' to 'my job is to set clear direction, set constraints, and set quality bars.'"

So John and I worked on this together using Claude Code at the time — this was back in January. And it took about an hour and a half to design the experiment, about two hours to analyze the data. And so if you want to clock back 12 years to the DORA days, this would've taken two to three weeks.

So John and I were pair programming this, and a week after that, John texted me showing me what he had done after that session together. And it blew my mind because it was performance review season at Cisco, and instead of staring at a blank page, John put more than 200 pages of material into a folder, had it open in VS Code. He put in all his team's status updates, his previous reviews, his notes, Cisco values, the job responsibilities at each level, and then he set up an `agent.md` file and showed me what he had done.

And so seeing what he had done, I actually felt anguish — because I remember decades ago writing my own self-assessment, spending days on this, knowing that no one's actually going to read it. It's just the idea of having an agent kind of do all this and do such a good job doing it just blew my mind. So it was mind-expanding just to see an application of coding agents for this totally different domain of management activity.

And here is John to tell you what he did after that.

John Rauser

For thousands of years, ships took to the high seas without really knowing where they were going.

You see, north and south are easy — you just look to the stars. But east and west, there was no good way to measure it. They used a method called dead reckoning, which is where you take your heading, your speed, and elapsed time and try to figure out how far you've gone based on what you've seen and where you think you need to go.

It worked. In fact, Magellan circumnavigated the globe for the first time in 1519 using these techniques. It worked, but the errors piled up, and many ships were lost because of this.

Now, I say to you that many of us lead our organizations in this way. We use last quarter's numbers, weekly status reports, and instinct and experience to try to figure out where our organizations are going based on where they've been and what we know. It works, but the errors pile up.

With agents, we have a better way. We have the ability to create navigational tools and techniques that allow us to guide our organizations through the stormy seas of our world. And that's what I'm here to talk to you about today.

That's what this talk is about. It's about how we use agents — how we can use agents to lead our organizations and be better managers. I call it agentic leadership.

So my name is John Rauser. As Gene said, I'm a Senior Director of Software Engineering at Cisco. I work in the security products group. I focus on agentic security and zero trust, and as I said, I work for Cisco. It's now one of the largest companies in the world — it's a Fortune 20 at this point. You know it as the networking giant, but there's a lot more pieces to it. There's the security group that I work in, there's the collaboration group led by Webex, there's the observability group led by Splunk, and of course there's the AI initiatives that are driving tremendous growth in our company.

Now, here's the thing. This growth has been fantastic, and I've been looking around my organization. I've been seeing coders getting 10x better. I've been seeing designers that are producing designs 10x faster and being able to iterate on them. I've been seeing our products become 10x more effective using agents underneath all of this. And I looked at that, and I love it, but I also think — what about me? What about us? What about us as leaders of these organizations? Am I getting 10x more effective? Is my organizational management getting 10x more effective?

So I went on a mission about nine months ago to try to figure out how we can use agents to do things better on our teams.

And I think that as leaders, this is part of our responsibility. This is maybe why you're here today. We have a responsibility to look for systems, look for techniques, find new methods and tools that can help our organizations be more effective. That's part of our job. In fact, that might be the main point of our job. So with the advent of agentic, I feel like that is the main point of my job now. I need to figure out how to use agents to lead and manage better and more effectively.

But to do that, I need to build. I need to use my agents to build things that help me. And that word is really important — it's what I want you to take away today, because it starts with us. We, as leaders, need to set the example and the tone and be the ones to blaze the trail in our organizations. This is a quote that I love. If you haven't heard it before, it's one of my favorite quotes, and it truly is how I lead my organization through inspiration and motivation:

"We want to teach them to yearn for the vast and endless sea."

You don't need to say it like a pirate, though.

When we start building and we start showing results and we start doing these things, they're going to yearn for the kind of efficiency that they can see us having — our peers, our people, our organization.

So what I'm hoping is that you'll feel inspired today. You'll see something that I've done, you'll go out, you'll build it yourself, you'll come back, you'll tell me about it, and we'll talk and we'll share ideas — because this is a new area. This is a new emerging area, the idea of using agents to lead and manage. So let's begin.

01The Agentic Management Operating System (AMOS)

Let's start with the main idea, which is an Agentic Management Operating System — a system of agents that are operating in our organization and helping us lead. I call it AMOS for short.

Each person, each leader, each individual might build their own AMOS and become extremely effective with it. But we also want to think about how we can build them together. So I'm going to talk about both of those things. I'm going to talk about what it looks like for me, and then how we've joined those things together in my organization.

Here's how I think about it. At the first layer, we want to get our knowledge base and our tools set up. We want to add to that agentic capabilities, and then we want to layer in integrations into our virtual environment. And here's the really fun part — weaving those integrations into our physical environment. And then finally, we weave it together in our organization. We connect our different systems, our AMOSes.

So let's start. Let's start with the basic AMOS. This is ground zero. We get our tools going, and we get our agent interaction services going. Remember, you can modify all these tools today. You want to add something to the right-click menu? You want to create a new plugin in Obsidian to do some tasks? That's all easy, you just do it. So creating those customized tools for your job, customized actions, building them into the tools, connecting into your knowledge base using your agents — that's the first step.

And then we want to set up our knowledge base. Here's how I think about the knowledge base. You've got your working memory — this is where I put my notes, my agent puts its notes, and it puts its logs and traces and state files and so on. This is sort of chronologically organized notes that we can go back and look at later. And then we've got our long-term memory, which is sort of durable entities, durable records — a story of a team, a person, a project, a customer, and different events that have happened with them over time — and we keep those up to date. Me and my agent keep those up to date.

I've got my inputs: digests of Webex meetings, digests of Slack channels, emails, meeting transcripts, of course. Those all come into my AMOS. They're feeding the agent and giving it the knowledge that it needs to update records and give me notes and tell me things to do. And then finally I've got output, which is the things I'm working on — it might even be this presentation right here. But it could also be a report or a status update and so on.

So this is sort of the ground zero, the foundation, the bedrock: the knowledge and the interaction services.

02Agentic Capabilities

Then we layer on the agentic capabilities. Agents have this sort of vast toolkit of things that you can do with them to operate. So you've got your prompt files — instructions that contain instructions and commands, things to do in certain situations. It starts with your `agent.md`, but many more get layered in there. I like to point out the `design.md`. I like to have a consistent design for my reports, my HTML files, my documents, so people look at them and they instantly recognize, "Oh, that's a Rauser document." It's kind of like a personal brand.

Then we bring in — where the rubber hits the road — the skills and the plugins. These are repeatable workflows and connections into tools that allow the agent to operate and tell the agent what to do in different circumstances. We've got triggers and automations that use those plugins and workflows based on different events. Maybe an email just came in from my boss, or a new Jira ticket got created. And finally, we've got agents and their hooks. So we might create custom agents for different jobs, we can have remote agents running in different places, and the hooks allow them to do deterministic capabilities — you know, at the beginning of every run, always do this; at the end of every run, always do that.

Now, I found very quickly that my AMOS needed a common architectural component — in most software systems, a control plane. It needs a programmatic interface, an API and an MCP, so it can talk to my knowledge base and other agents can talk to it. It needs a search capability — deterministic search, not generative search. I want it to be able to find things very quickly, instantly, in my knowledge base. So we built that as an API. I need a registry. I need my AMOS to know what capabilities it has. I need other AMOS systems to know what capabilities it has, so that becomes an API as well. And finally, I want self-improvement loops. If you're familiar with Hermes, there's a great self-improvement idea captured in that agent, and I've tried to bring that into my own system where the agents are looking for and learning from things that I'm doing, and from other agents as well. That gets interesting.

So this is the basics. Once you've got basic and advanced AMOS set up, I think at this point you can give it a name. You've now got something that you can interact with, something that helps you. I call mine Magellan — because I love the high seas, and I love the age of ships, and I don't know why, because I'm absolutely terrified of the ocean.

03Extreme AMOS: Virtual and Physical Integration

Okay, let's go to extreme AMOS. This is where the cool stuff happens. We need to weave the agent into our lives, into your virtual environment — your browser, your operating system — and you need to weave it into your physical environment. So I'm going to give you a few examples from each of these categories, virtual and hardware, and show you some of the things I built.

The first is a browser extension. I love this thing. It watches what pages I'm going to. It recognizes if one is important — maybe it's a Jira ticket, maybe it's a GitHub repository. It logs it, it summarizes it for me, and it keeps it for me. If I go to an article — like Steve's fantastic article on Wheelhouse, which you should check out — I can simply hit "note up." It's going to log it, summarize it, create a note for me. You can see I have the note open; you can't really read it, but I can type my notes in there, and that'll re-open next time I open the page. I love that capability. And then it also has an omnibar where I can launch workflows: "Go tell me all the things related to Wheelhouse, see if there's any GitHub repositories." It's going to fire off a headless Codex, and then it's going to go ahead and create a report for me. Things like that.

The browser extension is critical because it's where I do most of my work. And I don't know about you, but I often get lost in a sea of tabs and windows and spend way too much time trying to find things, poking around. Sometimes you just quit them all and start over. But Discovery — my browser extension — remembers.

Next, I've got something I call Endeavor. Endeavor is my operating system integration. It manages my clipboard, it manages my notifications, it checks my file system. It's basically an interface from my agent into my operating system. I found I needed this because I needed a way to instrument commands, instrument different notifications that it would provide for me. And it's been quite useful. This is just simply a dropdown — I have a little icon at the top, I click that, and there's various things in there that it can do, but mostly it's invisible to me and it's just helping my agent manage my operating system.

Next, I've got Astro Lab. Astro Lab is a dashboard. Every time I build a new capability, I build a dashboard for it. So, for example, I have my agent checking my email. I don't actually go to my email that often anymore. I go to Astro Lab and see what it summarizes for me, and what to-dos it found, and what things I need to follow up on. I just take it from there.

And finally, I've got something I call Traverse. Agents love to create Markdown and HTML files, but our file-sharing applications hate them, right? SharePoint doesn't like HTML files very much. So I had to find a way to share my files, and this quickly became sort of just the idea of a remote server where I could publish things and where I could also run my agents. The funny thing about this publishing is my friends saw me do this. They see I'm publishing these documents, and they said, "Well, what are you doing? What is that? Can I use that too?" And this quickly became almost a critical tool in our group where everybody's sharing their HTML files.

You'll also notice this little icon down here in the corner. That's not part of the page — that's a virtual dock, and my operating system running in Endeavor, it's a drop zone. I can drop any file or even a path to a file onto that. It'll instantly upload it and return the link to it, auto-generated, onto my clipboard. Bam — I just shoot it over to my friend. Really useful and super fun to build.

So these are the kinds of things that help AMOS be part of my environment, be part of my work life, be part of my teams, and sort of extend its reach and allow it to be integrated into my day-to-day. But like I said, it's not enough.

Have you ever adopted one of those systems like Getting Things Done, PARA, and so on? You do it for two weeks, then what happens? You forget about it. It goes away. I don't want that to happen here, and it's easy to let that happen.

04Weaving AMOS into the Physical Environment

So what I did, and what I started thinking about, was how I can build AMOS into my life. Now, I work remotely, so I sit at my desk in my house all day. I don't work in an office, but you can bring some of these things into the office too.

The first one, and maybe the one I'm most fond of, is something I call Semaphore — those flags that they use on ships. I have to redact a lot of stuff out of these screenshots, because these are real. Most of them. Some of them aren't — honestly, I've been traveling for two weeks and I had to auto-generate a couple of images. Don't blame me, okay? You'll see.

Okay, so this is Semaphore. It's a granular information radiator. It tells me a lot of different information about things that are happening, gives me to-dos. My AMOS can send me messages and tell me things on this little e-ink display — it's about this big. Have you ever seen those picture frames where you can upload, you can shoot pictures to? Well, this is like that. It's an e-ink display, and the agent can work with it through an API, and it publishes things to me that I might need to know.

So this is sort of like a granular view. But I also wanted a more coarse-grained view, and so I created these indicators. These are little dials — they actually have a rotary on the edge of them so you can switch between modes. These are little dials that just show me more coarse-grained signals. The one on the right I call the boss button. The boss button goes off if I got a message from my boss. Okay? That's what it is. Called the boss button.

And then I wanted an even more coarse-grained view — I wanted a vibe check. So I added a little vibe check. You can easily buy these lights, they sell them everywhere. You can use them through APIs — green and red, just a mood check based on overall health that my agent is seeing. And if anything's going really bad, it can turn to red and let me know.

Finally, a control surface. Simply a Stream Deck with my agentic app set up on it — and by the way, that's API-programmable too. So let's say my boss button goes off, and then I check my information radiator and I see there's an action item there. My agent's already updated the alert on my Stream Deck. I push the button, it opens the thing that I have to do immediately.

All that tension, all that searching, all that wasted energy from looking for things and constantly checking my emails, constantly checking my messages — I don't have to do that anymore. These things kind of bring it all together and weave it into my environment. They also don't let me forget about it, which is important. You don't want to put that notebook in the drawer, right? Never see it again.

So these are kind of the hardware devices that I've used to integrate Magellan into my life.

05The Organizational AMOS

Let's go to how we're integrating Magellan with other people's systems. I call this the organizational AMOS, and it starts with standards. It starts with setting some standards on: this is what a weekly update should look like, this is what a report should look like. It starts with integrating those systems together so people have developed APIs that are safe. We certainly don't want to leak any information out of different leaders' systems. I believe it was Steve Yegge that said that's part of the architecture — protecting from PII, and in this case, things like performance information or any kind of restricted information that shouldn't go out of an AMOS. That has to be built into these APIs, into these interfaces.

But once we get that going, we can also start to build systems that the agents use to connect and provide us with useful tasks.

I'll start with the first one. I call it Manifest. Manifest is a central record repository that all our agents can use to update durable records for different kinds of entities, starting with customers. So when my agent encounters, say, a meeting transcript that has some customer information in it, it's going to shoot that over to Manifest, and Manifest is going to parse it. It also, by the way, has an architecturally defined protection layer to make sure nothing's leaking in. But it's going to shoot that over to Manifest, and then Manifest is going to run a skill chain that's going to update these durable records and make them available to me and my agent. So when I have to jump on a call with that customer the next day, it's already updated. I get a meeting briefing from my agent telling me the background on this customer. I can go and look at Manifest and see what's happened. I can see that they had a change failure on one team the day before, maybe an incident a week before. Information that I wouldn't have known otherwise and would've been ill-prepared without when going into this meeting with a customer. So this is a very useful system that allows our agents to kind of work together to keep us all updated.

Next, we have something we call Pilot. We have to do a lot of reviews in our organization — status reviews, project reviews — but the one that slows things down a lot in engineering is a little review called the CAB. And if we can make the CAB better, I mean, that is just awesome. So we started there, creating a review surface where our agents could work together to do reviews using information from different teams and using a skill chain that we collaborate on that helps us understand what's happening when we're doing a deployment. I can come here, I can see what deployments have happened, I can see what the outcomes of those were, and I can assess them and maybe provide my judgment if needed.

The final one that I'm most excited about — but it's kind of most experimental — we call Forecast. This is the idea that we can use our agents to do predictive intelligence. We can combine signals, data, reports, things from around the organization to understand what's going to happen next week, two weeks from now, four weeks from now. Is this team going to slip on their target because their engineers are overloaded? Because we can see that they've got too many pull requests open, or this person's working on three or four things at the same time. Aggregating up all those signals into an overall, "Hey, this project might be a risk — you might want to look at it." It's really cool, but it's experimental, like I said.

06Outcomes

Okay, so we've got all this stuff in place. Here are my completely non-scientific data points — outcomes — of how this is affecting me and my organization.

First of all, I've just noticed a major decrease in meetings. I don't need to go jump on a meeting anymore to solve a problem. We just solve it almost instantly, async. Meeting length even is going down. We can end meetings early because we can get to decisions faster. We already have the information about what's happening before we got into the meeting, and everybody has the same information.

But most importantly: time spent thinking. I mean, this is just the outcome of agents in general. But for me, this is the 10x for me as a leader in my work. The coder got the 10x — here's my 10x. I get to spend 10x more time thinking because I don't have to go searching through that sea of tabs or trying to find that message or checking my messages for the tenth time today. I know what's going on. It's woven into my life and my environment and my workplace. I love that.

07The Help I Need

So here's the help I need. I want to know if you're doing stuff like this. What systems have you built to help lead your organization and manage your work? Have you been able to connect those into other people's systems? What are you building? And then I would really like to know — and I'm humble about this — what am I missing? What did I not see? What didn't make sense? Where should things have gone that you thought they would go? Or something you thought I was going to say when I introduced it — come and tell me. Let me know what that is.

And finally, we can build a community around this. We can build a set of people that are thinking and trying to solve this problem, and that's why we're here. That's why Gene has brought us together. So let's think about this together as a community and try to make this into something that truly helps our ships sail those stormy seas.

Okay, there's my contact information — pick your thing. Oh, and I also started a Substack. I'm going to blog on it too, so you can read that and keep up with what I'm doing. Thank you so much, folks. It's been great to be here. Thanks a lot, Gene.