Securing the AI-Powered Future: Identity, Access, and Detection in a Post-Human Workforce

[00:00:21.275] All right. I've mentioned a couple of times that the last year has been a super fun adventure finding kindred spirits who are passionate about how AI is transforming how we work.

[00:00:29.715] And one of those adventures happened in April when we had our IT Revolution Forum event, where we invited about 60 people to spend three days together to, believe it or not, write guidance papers. And so it was a fantastic three days.

[00:00:40.595] But surely one of the highlights was having Jason Clinton, Chief Information Security Officer for Anthropic, join us for a day. It was amazing having him give us a peek into the future and have him give us hints on where AI models are going and how it will affect our work.

[00:00:56.525] I've appreciated and learned so much from every interaction I've had with him over the last year, and I'm so delighted that he'll be sharing problems that he's passionate about, especially around the cybersecurity implications as AI models get more powerful. Here's Jason.

[00:01:15.285] All right, thank you very much. All right, how's the mic? Sounds pretty good.

[00:01:23.315] All right. So today I'm going to be talking about what the future holds. And I'm going to try and avoid talking about Anthropic and Claude as much as possible. You've heard lots of folks up on stage mentioning our models. I don't think I need to explain what we do and why it's relevant to this problem.

[00:01:41.955] So I'm going to stay away from that. And what I'm going to try and do today is do something called AGI-pilling you as the audience. Does anybody know what the phrase AGI-pilled means? Can you raise your hand if you know what that means? Just like maybe four or five people.

[00:01:56.275] So, AGI-pilled is the view that models will get smarter. That's all. And models getting smarter is the baseline assumption that we have at the frontier labs, and it's the reason that everything that's going on right now was predictable more than a decade ago.

[00:02:12.395] If you look at the history of model development, roughly, I think, 1957 is the perceptron. And if you take a line on a graph and you plot every single model that's been produced since 1957 on a plot, and you put the amount of compute that went into that model, and you track that up over the last 70 years, it's like roughly 4x year-over-year increase in the total amount of compute going into those models every year.

[00:02:38.745] There's been some ups and downs. There's been AI winters. But the models keep getting more and more compute going into them. And my perspective at a frontier lab is that the way that we are building the models assumes that the models get smarter the more compute that you put into them. This is called the Scaling Laws Hypothesis, which was first posited in 2017 and then later in 2020.

[00:03:03.625] So this viewpoint that the models will keep getting smarter is the background. So imagine, stay with me for a second. There's an analogy that I think is super important. Imagine you are a medieval blacksmith, and you're walking home from work one day at your forge, and a time traveler from today pops up on the path in front of you.

[00:03:30.665] Let's just assume for a second you don't think you're crazy and you're going to have a conversation with this time traveler. And the time traveler tells you, as the medieval blacksmith, in 500 years there will be factories that produce more horseshoes in a single day than everyone in your guild has produced in the last decade.

[00:03:51.345] Now, let's assume you're just going to believe this person. Your intuition might be, well, what about all of the jobs of blacksmiths? Or will the horseshoes that are produced by these blacksmiths be the quality of the ones that my guild produces? You might not immediately think: and so therefore, as a result of the technology that enables this, horses as a mode of transportation will be completely obsoleted and become a recreational curiosity.

[00:04:25.555] That is actually the conclusion and the thing that the blacksmith should be thinking about. So my proposal here today is we are effectively the blacksmith in IT, and for many reasons that have to do with the threat landscape, the pace of code innovation, and I'm going to try and hit on all of these and what we should be thinking about as we go into the next few years.

[00:04:46.705] So the reason that Anthropic has been spending so much energy on coding, and why this analogy that I just gave holds up from the blacksmith example: as a blacksmith, there was an initial blacksmith who made the parts that were necessary to build the first prototypes of some kinds of automation, manual automation at first, that made factories possible.

[00:05:16.105] In the same way, at Anthropic, our view has been that models getting better at coding enables models to make better models. This recursive self-improvement phenomenon is playing out right now inside of all of the frontier labs, not just Anthropic.

[00:05:30.905] And so what's happened, and what is happening actively right now as I speak, is that Claude Opus is training the next model, and we're using Claude Sonnet to train the environments that are used in reinforcement learning to train the next model. So this recursive self-improvement environment is a compounding effect.

[00:05:51.685] These models not only are getting smarter from the opportunity to learn more in the reinforcement learning environment, but also, of course, we're buying data centers and the compute and the chips are getting faster and all of those things. So you have this rolling compounding effect that's happening.

[00:06:07.625] And so Steve just mentioned that we believe, at this point, maybe 1% of the entire world of coders have even adopted agent coding systems. But if the models keep getting smarter, we're entering a world where we've democratized and lowered the cost of software engineering so dramatically that it radically alters the work that everyone in this room does.

[00:06:30.185] I'm the CISO at Anthropic. My job is to protect Anthropic. And so from my perspective, I'm thinking, okay, how does that change the threat landscape? But I also have IT reporting up through me. And so how does it change IT? How do I think about the shape of my teams as this revolution occurs?

[00:06:45.985] So we have inside of Anthropic an effort to use the models as much as we possibly can. This is internally just called Claude-ification. Everyone is encouraged to find ways to automate their work.

[00:06:56.945] And the pattern we see emerging is effectively that folks at the more junior end of the scale are now augmented in a way that allows them to move up the career ladder, and the trajectory of their career is now more senior as a result of using the models. And our desire for hiring junior and entry-level positions is sort of waning, and we're getting to a place where we don't need as many junior folks as before.

[00:07:21.665] So this pattern, this sort of like the models are getting better at coding, it's changing the shape of our teams. We're able to automate almost everything that we do at the junior end of the spectrum across every function, from IT to compliance to cybersecurity engineering to physical security even. All of that is happening in a way that I think will emanate in shock waves through the economy over the next 18 to 36 months.

[00:07:44.665] So if that's true, and maybe it's not, but this is my view, this is very similar to the moment that we went through with the Industrial Revolution or the Green Revolution. Technology is radically altering the cost and the effectiveness of teams. So what does that mean? What specifically, tactically, does that mean?

[00:08:03.345] So the first thing I would observe to all of you here is it is the case that anything that looks like an agent is getting really weird, really fast. I think the most simple version of an agent that you can imagine is one that sort of replaces an RPA process, where you have a very simple binary input-output. You have a model that says yes or no, or does a classification. It doesn't do anything else. You know what the safety guardrails on that thing are.

[00:08:29.085] In a world where models get memory and they have access to massive amounts of tools, and they feel out the problem space and decide what to do autonomously, that starts to look a lot more like a person's behavior.

[00:08:45.745] And we all know from experience in IT what happens: humans and our teams expand their access over time. You start with zero permissions on your day that you start at the company. But over time, you accumulate more and more permissions, more and more access, more and more freedom, more and more trust from your coworker and your manager who trusts you to do the right thing.

[00:09:10.505] And so the security model of using system service accounts or narrowly scoped, system-specific AI deployments doesn't really make sense anymore as the models get smarter. That is an urgent issue that is playing out in IT right now.

[00:09:29.605] Effectively, let's say, for example, that you want a model to use a virtual desktop interface, a VDI, to automate making a PowerPoint slide or something like that. That requires access to a VDI, which maybe needs access to a web browser because you use Office 365 or Google Workspace or whatever. And in that environment, you need a person account.

[00:09:51.345] And your permissions and the way that you treat and think about AI systems starts to collapse from one where you have a very narrowly scoped understanding of the risk to one that looks a lot more like insider threat.

[00:10:04.635] Insider threat in security teams is something that we've been dealing with, as a general rule, for the last decade. The idea is effectively, if you build an insider risk program from a security perspective, anything involving compromise of an endpoint or compromise of a person, you're building the defenses that imagine your entire infrastructure is this onion diagram and your attacker is moving from the outside perimeter to the crown jewels at the center, mixing metaphors.

[00:10:34.345] In that world, if you build an insider risk program, the same is true for agents. You've got an agentic system that looks a lot more like a person that's sort of playing out here.

[00:10:42.875] Another implication from what we're seeing happening is that the threat landscape of threat actors is collapsing. Democratization of access to coding plays out in the IT world, which I'll get to in a minute, but it also plays out in the threat actor ecosystem.

[00:10:58.385] So three weeks ago, four weeks ago, we published an example of a bunch of things that we found where threat actors are using AI models to automate, from the DPRK using it for the IT worker scams, to a state-associated actor using it for fully automating the malware and attack chain from a ransomware attack. All of these things are playing out in ways that are quite alarming.

[00:11:23.785] And that's only what we can see from those who are using our API endpoint and the threat response that our threat teams are using. If that is true, if a script kiddie can use a hundred dollars worth of API credits and can reproduce an attack as complex as, say, the Colonial Pipeline hack, that is a world where the threat ecosystem is altering in a way where every one of us as defenders have to think about, okay, how do I make sure that literally every way that my infrastructure might be exposed, might be attacked, is being autonomously defended against that kind of an attack?

[00:12:06.125] So I'll get into some examples how that plays out from an SDLC perspective. We've talked about SDLC this morning, but we'll come back to that in a minute.

[00:12:15.705] So if you are in a world where that democratization is true, where literally everyone in your company can be a programmer, and your existing IT team and your existing software engineers are much more effective, let's just take the models out 18, 36 months. Let's assume they keep getting better and better at writing code.

[00:12:39.575] I think at Anthropic now we're at like 90% of all of the code in Claude Code was written by Claude Code. I think somewhere 80 to 90% of all code at Anthropic is being written by Claude. If that keeps increasing, maybe 18 months from now we're at 98%. I think that world looks a lot like I ask for a day's worth of tasks in the morning, and I come back at the end of the day and do a couple checks and sort of check in the result.

[00:13:09.635] That is kind of the landscape that we're going towards. That changes the build-versus-buy equation in a very fundamental way.

[00:13:16.945] About a year ago, as Steve alluded to, we were starting to use Claude Code internally, and I had this moment where somebody on a recruiting team sent my security team a message and said, hi, I'm on the recruiting team. I have an English major background. I've never written code before. I wrote this Python script. What am I supposed to do with it?

[00:13:38.435] That's this emerging phenomenon of everyone now has access to coding. This enables organizations to decide, okay, well, I'm just not going to buy the recruiting tool anymore. If the recruiting team can write a Python script that does all the data aggregation and the analysis that they want, why would you buy the third-party SaaS tool that opens your organization up to even more attack surface?

[00:14:04.365] I think that's a really interesting question that we should be asking ourselves, and I think it fundamentally changes a lot of the decisions that we made. Of course, I will say with a caveat here, if a SaaS provider has data that makes them very unique, an example here would be EDR products that sort of do the endpoint defense, that's one where obviously you're not going to vibe-code your way to a replacement for something like that. So I think it just really depends on the application, and I'm not trying to oversell it here.

[00:14:31.425] So a couple things that sort of follow from this. You can imagine coding becomes ubiquitous. It's democratized. You have the threat landscape that's moving very quickly. So the question as an IT leader is: how do I prevent these vulnerabilities and the worst failure modes for data leakage and those kinds of things all playing out from both internal mistakes and from external attack surface?

[00:14:57.265] And I think a lot of the answer is that we will be turning to AI systems as part of what I think of as a virtual teammate.

[00:15:05.785] So everyone in Silicon Valley right now is working on something that looks a lot like memory. We announced this recently. Others have announced memory-related products. In roughly eight to nine months from now, we'll see the first startups show up, and it will be early, it will be rough, but I think the first startups will show up where they'll be offering as a service an AI that joins your team.

[00:15:33.185] And I mean that in the most literal way that I just said that: an AI model that has a name, it has a Slack account, it has an email account, it has a VDI, it joins your team meetings and listens to all of your team meetings. It goes through your onboarding process and is indoctrinated, or whatever word you want to use, for onboarding employees in your company culture.

[00:15:55.305] And it has all these memories that are associated with it, and people start working with these agents as a virtual coworker. So I think the first very early rough versions of this will be about eight to nine months from now. And then 18 to 36 months from now, we'll see this play out in a more fundamental way across the economy.

[00:16:13.585] In this world, we have to think of how do we turn IT operations in a way that allows scalable oversight of all of the things that I'm just talking about.

[00:16:23.065] An example here would be code review. Everyone in this room is familiar with code review. We think of it as the gold standard as a security control for mitigation of a single rogue actor inside of your organization. Now, I don't know about you, I'm an engineer by part. I've been in tech for 25 years. I think code review by humans is maybe 25% effective, even in a world where most people are, you know, that looks too complicated, I'm just going to click approve and let it go through.

[00:16:58.305] Yet this is what Sarbanes-Oxley calls for as the primary control for financial malfeasance, right, in our control maps from a compliance perspective. So the low bar here is 25% effectiveness. If you have a model that can get 50% effectiveness at keeping that bug from going out the door, that then is exploited by these newly capable threat actors, that is an incremental, obvious win for deployment.

[00:17:28.545] So from an SDLC perspective, you can imagine literally every stage in the SDLC. While the engineer is in the IDE, assuming they're using an IDE at all, can we detect the bugs before they even get checked in or uploaded for review? Once they're in review, can the code review bot look and find the defect? Once they're deployed to the staging environment, can we do automated pentesting?

[00:17:54.455] These models are very good at running fully automated offensive cyber operations. Can we automate a black-box pentest to look for these bugs before one of these threat actors finds it before it gets pushed abroad? So you can imagine and take what I just gave you as an example, the SDLC, and let that play out across literally everything that happens in an organization from a human review perspective. There's so many things that we do as control points for deployment that are obvious patterns to repeat here.

[00:18:24.995] All right, so what are we going to do as an industry? Everything that I just described is a call to action for us as leaders. We have to figure out how we're going to solve identity. We have to figure out how we're going to defend these systems that are running autonomously and use the systems to perform scalable oversight.

[00:18:50.325] I think the other thing that's sort of interesting here is I think all of us are getting pressure to deploy AI as fast as possible. I've heard this a lot. I don't know about you, but I am sometimes tired of hearing about AI. There are a lot of house parties in Silicon Valley now where there's a room where it says, no AI talk allowed. And so people will go to the room to get away from people talking about AI.

[00:19:18.665] But the situation at the moment is we're getting asked to move very quickly, in some cases with no idea what the right defenses are. So the way that we've handled this as security teams historically has been, with human insider threat, building effective detection response teams. So when all of your preventative measures fail, how do you provide scalable oversight to an organization from all of the threats? Well, we do that through detection and response.

[00:19:45.145] So D&R products in this space are well positioned to help with the agent deployment problem, but they're also, I think, potentially a place where some older and less favored technologies might be seeing a comeback.

[00:19:56.545] UEBA was a fad like five years ago, and then it turned out it was way too noisy and it just couldn't build. UEBA stands for user and entity behavior analytics. The idea is effectively, let's build a profile of what good looks like, and then if something goes outside of the norm of what that is, then we will alarm on it.

[00:20:17.145] So this gives us an opportunity to provide the scalable oversight that we will want from a security team's perspective and gain confidence that if a model goes off the rails, or if there's something that's outside of the norms of what the model should be doing based on what a human asked it to do, we're going to get alerted to security teams and be able to respond to that.

[00:20:32.725] I think we have a higher probability to succeed on that as an approach than we do on the preventative measure, just because we're being asked to deploy this at such a fast pace that we won't have an opportunity to be fully defensive.

[00:20:45.865] So I want to go back to the blacksmith analogy. If you look at a graph of all of the horses in existence in North America and you see this massive graph over time tracking the population of the U.S., and then we get to automating transportation with cars and the horse population drops precipitously, and it's still around. There are a lot of horses living in some ways their best lives, in ways that they were not able to in previous versions of our economy.

[00:21:19.065] This, I think, should be the framing that we should be thinking about from an IT management perspective. We will be fundamentally altering the landscape of what it means to do IT. And as we think about what that means for the future of our teams, we should be thinking about a world where much of what we think of as important work is changed and shifted to higher levels of abstraction.

[00:21:43.265] And there's a call to do something about this now because we're moving very, very quickly. Everything that I can see from a frontier lab perspective, from the data, to the chips, to the data center buildouts, to the emerging science, and the recursive self-improvement that I talked about earlier in this talk leads me to believe that at least the next two years, the pace that we've been seeing is going to keep going on.

[00:22:04.665] And with that, I'll end. Thank you so much.