Virtual US 2022
Transformed! A Musical DevOps Journey with Forrest Brazeal
Transformed! A Musical DevOps Journey with Forrest Brazeal
FB
Forrest Brazeal
Developer Communities, Google
Transcript
I know exactly the moment in 2021 when Richard Retter, whose work I've admired for many years, sent me something on Twitter. It was about this amazing song that the brilliant and talented Forrest Bra Brazil had composed and performed. I was just leaving the parking lot of a grocery store with all three of my kids. I pulled over, played it for them, and they loved it as much as I did. Obviously, I then tried to find everything I could that Forrest Brail had ever created. And in my conversations with him, I discovered that we both were interested in Hamilton. We were obsessed with Lin Manuel Miranda. And I must admit, in my grandest and wildest dreams, if there ever were to be a Phoenix Project musical, I'm hoping it would be as amazing as what Forest has created. But the other thing I learned is that this is only just one sliver of the amazing things that Forest has done.
In particular, I continue to be blown away by the success stories from an initiative you created called the Cloud Resume Challenge. So early last year, I asked Forrest if he would be willing to talk about his amazing work and share some of his amazing creations with us. And I'm so delighted that he said yes. And for me, one of the most amazing moments of DevOps Enterprise Summit in Las Vegas two months ago was seeing Forrest perform these amazing numbers on stage on this baby grand piano. We wanted to share that moment with you. So here's Forest.
Good evening.
It's
5:41 PM Do you know where your, uh, DevOps transformation is? We know where it started, don't we? It always starts the same way. It starts with high hopes. It starts with big dreams. It starts
With a vision. Picture us glorious rising up at last, the reborn with cloud big data and AI will bid our legacy ways. Goodbye. We're caterpillars now, but we're gonna fly. Yeah, we're gonna be transformed digitally. Never done it before, but how hard could it be? All these slide decks look so great. Set. The deadline won't be long to wait till we declare victory digitally. How hard could it be? Years pass a alas fighting still. The fires we fought before our POCs looked great. So why are we not seeing r o I end all the time. Our spend is climbing high Or will we ever be transformed digitally? Why didn't anyone warn? That's how hard it would be. Airport billboards had it wrong. Transformation's messy and it's long, and we're failing miserably. But wait,
Maybe transformation isn't just about technology. Maybe it starts
With us. The leaders, when we address the questions that have always plagued our teams, like when we ship code to production, what is causing all those screens? Have our results been feeble? Because we only pretend to train our people.
And do we know
Our systems are secure? Are we sure? And are we helping solve the problems that face our world today? If it's hybrid work or climate change, how must we change to lead the way? If you want to turn into a butterfly, you've got to break down into goop and die. It's painful and it's gross, but if we try, well, we just might be transformed. Digitally doesn't happen at once, but continuously. We may not feel it happening, but gradually we'll grow and flex our wings. We'll achieve maturity digitally. Oh wow. This time you'll see will be transformed probably.
Thank you very much. My name is Forrest Bra Brazil. I work for Google Cloud. I'm their head of developer media, which basically means I get to make all kinds of extremely unusual, uh, content that developers will enjoy. Uh, but before I worked at Google, I did cloud and DevOps kinds of things for a whole bunch of different companies. And this is what would happen to us. We would go on these multi-year DevOps transformation Odyssey. And at the end of that time, it sometimes felt like all we would've accomplished would've been to have refactored ourselves into slightly different silos. Uh, but can can I be real with you? Can I confess something to you tonight? Yeah. Okay, good. Good. Yeah. You're gonna have to give back tonight. Uh, I can't do this all by myself. Uh, so listen, I kind like silos. I do. I no, listen, hear me out. I like them. They're safe, they're comfortable. Uh, they're great at optimizing communication within functional groups. Sometimes they're full of corn.
Uh, there's
Really no downside to a silo until it's time to change the status quo, right? Time to change the way we do things. Uh, and then they become missile silos. And I had seen this so much at, at different places over the course of my career. Not Google. I'm not talking about Google. Don't come after me Google. But I've seen this in so many places over the course of my career. I that I felt like I had to write a song about it. And the song is called The Reorg Rag. Would you like to hear it? Yes. All right. So just like most reorgs, uh, this song is a little bit too difficult for me to pull off accurately in real time. Uh, so good luck to us all.
Well,
Our company used to have these big silos. We had Devon Ops in two different rows, and they'd rarely communicate except to fight. So we said we're gonna build ourselves a dedicated DevOps team because everybody loves a go-between. We're gonna break these silos down. We're gonna do it right. We're gonna do a little re yes Siry this time. We're gonna fix this company. We're gonna have this DevOps thing in the bag after one more round of the reorg rag. Well, the DevOps team started hot as heck, but they soon became kind of a bottleneck. And instead of two silos, now we have three. So we said we're gonna make dev do their own ops. Now it's all in the cloud. They can figure out how. And we believe the term for this is Ss r e, we're gonna do another E. You are, yes, e this time we're gonna fix this company. We're gonna storm the silos and capture the flag after one more round of the reorg rag. Well, it turns out Ops is incredibly tough. And Dar dev soon said they'd had enough. So we brought back ops, but now they needed more. They said, you're gonna need a platform team to guard and guide, and the DBAs wouldn't come along for the ride. We had silos everywhere, and they were all at war.
So here's where we're at. I'm
On the tooling team, which is owned by Dev, but
Functionally it rolls up under this new cloud, c o e, which was recently spun out a corp it with a dotted line, the security. And some of those folks still report to me. I think technically I'm my own vp. We're gonna need a bigger reorg. Yes, sir. Somebody put a sound of our misery, because my optimism may start to say, if we play another round, other play another round, other the play, another replay, another replay, another replay, another replay, another replay, another
Play, another
Play, another round of the reorg rag.
Thank you so much. But this is what we do, isn't it? And, and I think no matter what organizational structures or or strictures we adopt, we're always at risk of falling back into these kind of adversarial, uh, org patterns. Because fundamentally, we're human. We're, we're a tribal species. We like to to ingroup around things, whether it's, uh, an organizational thing, whether it's a technology fad. We do that for sure. And then once we form this, this new group, what do we do? We, we dig a deep mode around ourselves, uh, and we take up pitchforks and torches to, to ward off the bad ignorant people who are still using the paradigm from like four minutes ago. Uh, and, and we do that to, to distract ourselves from how small and insecure we really feel. Uh, speaking of things that are small and insecure, who here is using serverless functions, uh, in production? Anybody? Yeah. Some of you. Great. How about, uh, Kubernetes? Who's using Kubernetes in production today? Yeah, some of you are wonderful. Uh, who, who thinks it's all just a bunch of hype at this point? <laugh>, pathetic fence sitters. I see where you are. Well, listen, we are gonna settle this tonight once and for all. And we're gonna do it in the time honored fashion. We're gonna do it with a serverless versus containers wrap battle. Are you in <laugh>?
Alright, let's do it. Okay. Now listen, as you've noticed, I'm one person, so I cannot do this by myself. I'm going to need your help. So this whole side of the room, uh, everybody down here, whoever's up in the balcony, right down the middle, you're gonna be my serverless fans for tonight. Sorry if that hurts you. You're gonna be serverless fans for tonight. And when I have my serverless hat on, can you see that when I have my serverless hat on? I need you to make some noise. I need you to show some love. Let me hear all my serverless people. All right? All right, good. Okay. All right. We've got a robust fan club. Now, on the other hand, when I have my containers hat on, <laugh>, I need this whole half of the room to show me some love. So when containers gives out a sick burn, right? I need you all to go, Ooh, lemme hear y'all do that. Oh, oh, all right. Okay. I dunno. I, I don't know if any of us can win. It'll be all right. We'll, we'll get through it. Uh, okay, well, wonderful. Well, in that case, I think we're ready to go. Are we ready? Woo.
Okay. Drop the beat. Yeah, you can call me crazy. By the time my mama raised me, servers make me nervous, but serverless never phase me. Maybe I'm just lazy. But why? To play a box. When abstraction brings the action for a fraction of the cost, my code is in a zip file requirements in a PIP file, cram it in a function, and bam, all I do is ship while standing proud in the cloud, on the shoulders of the giants. What? You need a server for the value's in the clients.
There we go.
Yeah, but I wish there was something more portable to wrap my app in. Oh wait, there is, it's called containers. They make crap happen. Check that out. I can build the way I'm used to. Open source dope, of course. Run it where I choose to automate and orchestrate with Kubernetes at the helm hand on my tiller. Ima killer in the service realm. Plus all these hella world tutorials are slick. So someone called the docker 'cause the stack is looking sick. See, this is what you do. It's insane. You overcomplicate the name Kubernetes. So, or Ka, you spell it with an eight. I'm no cloud economist, but I'm sure you don't wanna miss the savings you could find if you put your mind to simplifying this are building Pixar is your name John Lasseter. Then why you need a service mesh? Why you need Ambassador Istio, miss eo, this kingdom isn't magic.
You throw it in production and the outage gonna be tragic. Hey, take a breath, man. You ran kinda long there. Functions have a timeout. You gotta save your song there. You got limitations. I run applications every enterprise as their eyes on containerizations. Why long jobs? Real time doesn't matter. I eat hard problems for mealtime. So pass the platter. So many industries left to disrupt. I know your cold start makes it hard, but try to keep up. Check it. 1, 2, 3. He's getting warmer. Wait and see the latency's no worse than your complacency. Face it. See, you're basically chasing a place you don't wanna be my services approval by themselves. They get better. Meanwhile, you're outta luck stuck, chucking out the cheddar. Hey, remember Specter and meltdown? You were up all night. Me, I slept. That's right. The cloud provider kept it tight. You can patch your run times. I'll have happy, fun times delivering value while pal you fight the same old fight. That's great. But wait, let's use our brains here. Yo, I got constraints here. I'm running Java eight here, digging in the brownfield, moving the ball. Downfield can't re-architect at all until we look respect the ball. I just wanna build more. That's what I get billed for. Functions are a blast. Our past those would've killed for, I know, I'm just saying we're in a different state of being. Yeah, but functions are amazing at wait, are we agreeing?
<laugh>,
Obviously both of us have the same destination. Get rid of heavy lifting with that differentiation. So whether your abstraction is a function or a node, you can get a lot of traction on the old cloud road. If your app goes down at 3:00 AM and it will, you gotta own that. It's your problem still. There's no silver bullet hocus pocus magic guarantee. But when businesses your focus, you'll be where you want to be. Thank you very much. Alright, well I have one more song to play for you tonight, but before I get there, I wanna introduce you to a fellow named Daniel Singletary. Because
Let's be real, serverless isn't the problem. Containers aren't the problem, right? There's no magic reorg that's gonna deliver value to our customers. Even the right vision isn't enough. We can attend all these talks. We can underline all the right parts of the Phoenix project. None of it means anything without what, without the people who have to execute. These things can't transform your organization unless you empower the people in it to transform themselves. Uh, so I wanna introduce you to Daniel Singletary. This is Daniel. I met Daniel a couple of years ago, right at the beginning of the pandemic. At the time he was a residential and commercial plumber working in metro Atlanta, doing all the things you would expect a plumber to do on his feet a lot long days. Shortly before I met him, he got called out to a shopping mall there in Atlanta because the owners were reporting a strange and horrible smell, pervading the entire building, maybe something you would expect a plumber to deal with.
So Daniel heads out there right away, the smell hits him in the face. He can, he can, uh, detect it. So he heads into one of the bathrooms. First troubleshooting step, takes one of the toilets off the floor. Instantly he's hit with a blast of noxious air. Now, I'm not a plumber, but Daniel assures me that not only is this not something that should happen, it shouldn't even be possible. You shouldn't have currents of air blowing through your, your sewer system. So how do you break down a problem like that? Well, in Daniel's case, he came back to the business with a colleague and they started at opposite ends of the shopping mall. And they started working their way systematically toward each other. Over the next couple of days, they systematically unscrewed tested and resewed every pipe fitting in the building trying to figure out where this horrible smell could be escaping.
They couldn't find it. Finally, they got down to two businesses in the center, a restaurant and a hair salon, and they had to try something else. So this is where they tried a smoke test. Who here has done a smoke test in the context of software engineering? Yeah, a lot of you have. Did you know that's a real term with a real smoke related meaning? I didn't until I met Daniel. Guess what a smoke test is in the context of plumbing. You get up on the roof, you find a vent, and you drop a smoke bomb down it. That's a smoke test in plumbing. Uh, and the reason they do that is, you know, anywhere that this gas could be getting out, smoke can get out too. But the difference is we can see smoke with a flashlight. So that eventually clears up the problem.
Turns out that someone has tied the vent hood for the restaurant's commercial stove into the sewer system. And that's what's, it's always something like that, isn't it? That's what's causing this, this error. Uh, it was around that time I believe that Daniel decided that he was ready to look for a new career <laugh>, maybe one more, where he could choose what to smell from time to time. Uh, and that is how Daniel and I met because his roommate worked in it, and his roommate pointed into something that I had created called the Cloud Resume Challenge. If you haven't seen the cloud resume challenge, you can check it out. It's at cloud resume challenge.dev. And it looks like this, it looks like a wall of text. It's basically a speck that tries to get people coming from non-traditional backgrounds. So people without a like four year college degree in computer science or whatever, uh, people who haven't worked with cloud before, it tries to get them hands-on and learning and building like an engineer.
So it's not a video course you watch, uh, it's not, you know, a certification you take, although we do encourage you to get a certification as you can see. But basically, uh, it's giving you a spec with some instructions to put your personal website in the cloud. Uh, but there's specific things you have to do, you've got to use. Well, here I'll show you, uh, an example. This is a, there we go. This is a recent challengers architecture diagram. And you can see they're having to use source control. There's some infrastructure as code built in here. They're using Terraform. Uh, they've got a cloud storage bucket going on. They built a small serverless, a p i. Uh, and if you can complete all this, which is not a given, many people try this and can't get through it, you've actually built something pretty impressive. You've built something not only that a lot of people can't do, coming out of a university computer science program, you built something that a lot of professional cloud engineers haven't done. In fact, every day we have people working on the cloud resume challenge. Who are, they've, they're three, five years into their career and they say, oh yeah, I've never strung all those skills together at once. I would definitely like to try that. So put yourself in Daniel's shoes and imagine how he felt looking at these kind of things for the first time, trying to decide if he wanted a career in technology. What do you think his reaction would've been?
I'll tell you exactly what his reaction was. He said, that looks familiar. So he went out and he bought a whiteboard. This is his real whiteboard. He sent me a picture of it. And this is what he drew to try to make sense of the project he was about to undertake. Daniel called what he was drawing an engineered print because that's what they call it in the trades. He did not realize he was drawing his very first cloud architecture diagram. So over the next few weeks, Daniel spent part of his time working on sewer pipelines and the rest of the time working on C I C D pipelines. Uh, again, a real picture, Daniel, let's not ask what's on his shirt, <laugh>. Uh, so he is working in the back of his work truck. He's getting home from these 11 hour days and he's crunching Python and JavaScript, learning about YAML for the first time.
We just pour one out for this man <laugh>. Uh, and so eventually he thinks about giving up multiple times, but he finally completes this thing. He, he has it live. He has his website in the cloud. He's built all these, you know, full stack frontend and backend things to make it work. You can check this out. It is totally live. It is@dssresume.com and it's maybe not the most beautiful like H T M L formatted thing you've ever seen in your life. But it doesn't matter. I call it the greatest resume I've ever seen because just look at the credentials that Daniel has on his resume. Check this out. This man is a licensed journeyman plumber. He's certified in backflow prevention and cross connection control inspection, and he's got four certifications on a w s. How awesome is that? I I, I just, I love that. I love that story.
So Daniel writes a blog post about his experiences. Actually, the first requirement for, or I should say the last requirement for any one of these projects is you have to write a blog post about it and share it. He called his blog A Plumber's Guide to Cloud. It ended up going viral on LinkedIn. It was viewed more than 200,000 times. And just a month or two later, Daniel started his new career as a DevOps engineer at a large enterprise. And as well, he should, right? Because when I talked to the hiring team that had brought Daniel on, they told me they, they liked that he had done some technical things. I mean, that was cool, but let's be real, he's still very green, right? He is got a lot to learn. But they really liked that story of the leaf blower sewer. Think about what's being demonstrated by that story.
Daniel is able to take a gigantic kind of diffuse problem. This entire building stinks and he's able to break it down into achievable steps. He's able to work with a colleague, basically to pair program, break the problem down, eliminate possibility step by step until they arrive at the only solution he gets tracing. He knows how to use tracing techniques. That's what that smoke testing thing was. And most importantly, this man has a great grasp of business continuity, right? He knew that he couldn't just kick out everybody in the restaurant while he meticulously took apart their entire plumbing system. That wasn't gonna work. He was gonna have to figure out how to troubleshoot this live. And so for that reason, in many others, he was a fantastic fit on his team and continues to kill it in his career today. In fact, that team, uh, at that enterprise has actually gone on to hire, I think, six more non-traditional career changers, uh, because of the success that they had with Daniel.
And Daniel's not the only person who's got a success story outta the cloud resume challenge. Uh, there's Jacob who was a, um, infectious disease researcher. Uh, he's got some great covid pandemic stories that are applied to tech. Uh, there's, uh, Chidi who is in hr. There's Brooke who has a worked at T-Mobile. Uh, there, we've had loggers go through. We've had professional poker players go through it and you can read a lot of their stories@cloudresumechallenge.dev. Uh, many of these people are now my friends and colleagues in the industry. But I wish I had even more of those stories to tell you because let's be real. You know, it's tough to get a job in this industry at the best of times. It's tough to break in. Um, I think a lot of us came up in an era where you could kind of just say, well, I scraped and scrambled around for a few years and eventually I turned out okay.
So the next generation turned out okay too. I don't think that was ever true. It's definitely not true. Now, Google Cloud is saying that we're gonna need something like 40 million more cloud builders in the industry by the end of this decade. A w s and others have put broadly similar numbers out there. Those people are not all going to come through that traditional four year golden path, uh, where they, they get a computer science degree and, and take advantage of some of the like green bean and new grad programs that you may have at your companies. They're not just gonna be floating around on the open job market. We have to think bigger. We have to get more creative with how we source and find this talent. And we have to be more creative and more intentional about how we onboard and create experiences for those people.
It's interesting, you know, this company that brought Daniel on, I told you they brought on several more people, uh, from the Cloud resume challenge. And what they ended up doing was kind of backing their way into creating an apprenticeship program at their company as opposed to a traditional developer onboarding process. And I think that's interesting because <laugh> guess who had lots of experience going through an apprenticeship program, it was Daniel, right? That's how he got into plumbing, you know, very familiar with that. And some of the things that he had to do in the plumbing apprenticeship world, like going out and working under different plumbers. I dunno if you know this, but in the trades, you don't just pin onto one, uh, practitioner. You, you get rotated around to different people through your local trade board. Um, some of those people were probably better to work with than others, but he had to do that.
And in the same way when Daniel came onto his, his team, uh, in tech, uh, they had to find senior engineers who were willing to basically pair with him. Um, and they had to set expectations with those people. And it wasn't all their senior engineers. They found some people that were willing to do this. Uh, and then they kind of grew from there. Uh, in the same way, you know, Daniel got paid in his plumbing apprenticeship. He, he didn't do it pro bono. He didn't get paid much much, but he did get paid for every hour that he worked. So this company that brought Daniel on had to figure out a way to make sure that he got paid a living wage. They had to set that expectation with, uh, their leadership team that hey, even though this person isn't going to be, you know, contributing in full capacity for a little while, we think that there is substantial value in investing in someone like this.
We think there's value in uh, making sure that they have what they need to succeed. And then finally, Daniel, in his plumbing apprentice world, when he completed a year or two, he was able to go on and get licensed. He was able to start out as a journeyman plumber on his own. So there had to be some clear progression and goals set as he was going through that process. It wasn't just nebulous and open-ended. Uh, and that's what ended up having to happen in tech as well. There had to be this clear understanding in the organization of this person is an apprentice. You have to have different expectations for them right now, uh, but they will get to a point where they're able to help. And here's what that's gonna look like. I want to talk to you much more about that, uh, later tonight.
So my ask to you, everybody's been asking for help with different things. My ask is for you to help me, help you help others. I want to talk to you after this about maybe apprenticeship programs you might be running at your company. I want to learn from you. I wanna share what I've learned. We've put thousands of people through the cloud resume challenge now over the past two and a half years. Um, they all have unique stories. They have unique stories of, of where they've landed and how they've gotten, uh, brought up in the industry. And we need to be sharing those practices more broadly. So please come find me. You can find me. Uh, you can see cloud resume challenge.dev there, you see my Twitter, you see my email address? Please reach out. I really wanna hear from you. This is not part of my regular job, it's just something I do, uh, because it needs to be done and I really want your help in doing it.
Let me leave you with this bottom line. Wait, wait, lemme leave you with this bottom line. Every one of us here in this industry, every one of us sitting in this room tonight got here because at some point in the past, somebody took a chance on you. Somebody looked at you when you had zero experience and said, that person has the aptitude and motivation to succeed in this field. And I think I can teach them the rest. And you prove them right? So how are you paying that forward? I have one more song to play for you tonight. I actually wrote this song a few months ago for my, uh, daughter who was born in May. A
So like her hello, lost this audio.
I wrote this song for my daughter, uh, who was born in May. So like her. This song is new. Uh, you're the first public audience actually ever to hear it. And I hope that as you go back and work with your teams, uh, mentor them, create cultures of belonging there, that this will resonate with you. Now, the song is called You Belong.
You won't always feel like a rock star.
Sometimes you'll
Struggle just to carry the tune in the mirror. You'll see an imposter and you'll wonder if the whole world smarter than you and stronger and faster. And you are a disaster. But baby, you are wrong. 'cause you belong. Even when you don't believe it, you
Belong.
Oh, I've got help here when you need it. Just hold
On.
Yeah, the road ahead looks steep. So take my hand. We'll make the leap together.
Oh, oh,
You belong. Sometimes you'll feel kind of burned out. You'll work with folks who seem diffuse and unfair, or you'll have a dream. And it turns out then no one else around you seems to care. And when you see what you're up against, it's find to fight, to feel and sense, or just to move on. 'cause you
Belong
In a place where you're supported, you
Belong
On where you're challenged and rewarded. Just hold
On.
As you feel your power growing, you can choose the way you're going. And it's oh,
Oh oh
Where you belong.
Oh, oh, oh.
And one of these days you are gonna reach the mountain top. Yeah, it's not if, but when and when you do, I hope you'll take a beat to stop and celebrate how far you've come, remember where you've been, and then reach back to those behind you. Shine a light so they can find you. Help the ones you can then sing to them again. Tell 'em you
Belong.
When you're new and nervous, then you still
Belong.
I know it 'cause of Ben, you just, hold on. I know the road ahead is steep, so take my hand. We'll make the leap together. Oh,
Oh, oh, oh. You
Belong. Oh, oh, oh. Come on. Sing it with me now.
Oh, oh, oh.
You belong.
Oh, oh,
Oh. That was good. Let's try it again.
Oh, oh, oh.
Where you belong.
Oh,
Thank you very much. Enjoy the rest of DevOps Enterprise Summit.