00:00:07

Hey, I'm Jeff McCaffrey. And I'm coming to you from sunny, Seattle, Washington. And I'm in my garage because you know, everybody's working at home. And while literally in my house, everybody's working from home. So this is my quiet spot. Uh, I'm a product manager at GitHub. I work on a bunch of different things, but one of my key focus points is helping, uh, folks do open source at scale producing open source consuming, open source, and just having a great open source program. And so that's what I'm going to talk to you about today. Now, until recently I was, uh, actually working at Microsoft and running their open source programs office, and that, uh, that's the set of people who help the company, you know, get good at doing open source who do it right, right. And, uh, engage in a meaningful way. And my move to GitHub was really about bringing a lot of that experience and understanding that we gained at Microsoft bringing that to get hub and thus enabling all of you who are doing open source to really have an open source program that, that rocks.

00:01:06

And today I'm going to share some of those ideas and approaches with you. Did, you know, you might be thinking this doesn't really apply to me. You know, Microsoft is such a big company, uh, and we're just a small company or organization, uh, that that's not really true in essence, a lot of what we learned and what we did applies to you, even at a much smaller scale, because once you get past the number of things you can manage on a spreadsheet, uh, you need some sort of automation. Once you have more than, uh, uh, you know, a few people you need to start to drive some culture change. All of those things are, are things I'm going to talk about in here. So this all starts with a basic premise, right? This, this premise that I like is, uh, aspiring to be world-class isn't enough when everybody else starts there.

00:01:47

So if you start out like, you know, we often did at Microsoft in the past saying, Hey, I want to build out some new cool application and it's going to need a database. So let's write a database and then, oh, it's going to need a UI. So let's write a UI framework. Uh, if you start off that way, you're really not going to progress very fast. And you're going to spend all your time building that infrastructure, uh, instead of building your real value and everybody else is going out and getting Mongo DB or my sequel or Ray act, or, you know, pick your favorite technologies and putting them together. And in fact, in some ways that's a hallmark of dev ops is, is taking components and assembling them into solutions quickly with confidence. And so essentially, uh, I that's, the way I see an open-source program is helping you do that, how enabling that, that fast, uh, assembly of components from open source and also that, you know, contribution and creation of those components.

00:02:38

So, you know, you have to start somewhere and it's already resource with, maybe you've got a small number of people, and this is certainly the way Microsoft was, you know, a few years ago, a small number of people using a small number of components. And then we sort of stick quickly advanced, uh, into having many, in fact, like 32,000 people from Microsoft are, uh, active in GitHub today. And there are many different components that we produce and projects that we use across the company. And so as that evolves quickly, you can imagine going from that very small number to a very large number of engagements, uh, that you really have to have some level of organization. I mean, this stuff doesn't just happen, right? And especially in like a dev ops world where you're running really fast, uh, you don't really want to have to stop and figure things out every time a new component shows up, for example, like, are there any vulnerabilities?

00:03:29

What's the license, all that sort of stuff you don't want to do that manually. You need some level of automation. You need understanding that that's what you should be doing. And there's a whole set of infrastructure and approaches that come and behind the scenes to make that all a friction-free so that you can do continuous integration, continuous delivery, and so forth. Now all of this is needed, of course, because once in a while, there's a component that, you know, isn't is a bit of a surprise. So my little tip of the hat to gene and, uh, and the unicorn project, uh, but you know, seriousness, you know, there might be a component that comes up, that's got a vulnerability or is malicious, or has a license. That's not compatible with the way you're using it. And you need to know all that. And you need to know that in an automated way, that's integrated into your workflows because otherwise you won't find it.

00:04:17

If it's manual, you won't figure it out. So that's been a lot of talk about tools and everything. And as devs, you know, I think a lot of us sort of devs and we'll imagine, well, there's an app for that. Or I can write a tool that'll solve that problem. And, and to a certain degree, you're right. Uh, there are many great tools and apps that can help here, but fundamentally, if you don't have that culture change in your organization that says, Hey, we're going to use open source. We're going to engage with open source. We're going to release open source. If you don't have that culture that drives that from a business need level, then you're, you're not, you know, those tools are not going to make, uh, be of any use to you because nobody's going to be doing it. They're not going to be engaging.

00:04:59

And again, here, you know, you've got to start somewhere, right? There's one or two people off in the corner, figuring it out for themselves, putting it together manually, uh, following the instructions from the book, you know, step-by-step, and that's cool. That's a great place to start. Uh, but from a culture change point of view, the real magic starts happening. When you get a lot of people together, a lot of helping hands coming in, collaborating, sharing, engaging with one another to see, like, how can we do this better? How are we doing it now? Where, you know, there's this other piece of technology that you could use. Uh, and, and that's where it really comes together the value of open source, but that also doesn't happen for free. You have to build a culture, build an understanding, a shared set of values, a shared set of expectations and understanding of what's desired.

00:05:46

What's safe. What's not desired how to behave because otherwise just like you, might've ended up with sort of tool and mechanical friction, you'll end up with societal or a cultural friction where people have different expectations and understandings, and it just won't go together. So you need to build that, uh, that understanding and that culture as well. Now you can imagine at a company, the size of Microsoft, both of those challenges, both the tooling and the culture or the process and the culture challenges are quite significant. And, uh, you know, fortunately we had a lot of help from, uh, on high. Uh, so Satya here, a loving, loving Lennox. Um, you know, this is really great. A lot of the senior leaders in the company understood at a deep level, the value of both what we sh you know, the value of the company, but also the value of open source in both, uh, uh, producing and consuming.

00:06:35

And so we have this guidance coming from top, uh, a lot of people coming into the company who've been doing open source for a long time. Uh, and so we needed to look at how we could evolve that now, as you imagine, going from sort of that zero or, or small number to a lot of, a lot of engagement, it doesn't just happen for free. Uh, there needs some sort of, uh, organization and coordination to make this happen. Uh, and that's where, uh, an open source programs office comes in. That's, that's their job. So looking back, uh, you know, around 2015 is when we created an explicit Microsoft, uh, programs office. And about that time, it was a pretty pivotal time, uh, looking before that we were doing open source here and there had some really serious efforts, but it wasn't widespread. And what happened in about, in about that timeframe was that we realized, okay, this is a good thing.

00:07:26

This is here to stay. This is the way we should go. And instead of Microsoft speak, we're all in a, and so we've got to make this real, we've got to systematize it, we've got to make it sustainable. We've got to coordinate it and really enable our teams to engage. Uh, and so we created the programs office, and that's what was, was our mission. And really, you know, if you look at where an open source programs office lives, it sort of lives between sort of that engineering need to engage with open source, the business drive to do so and move further. And those cultural shifts that need to happen. And the, the role or job of the program's office is to drive initiatives around culture, change, uh, understanding tooling, et cetera, uh, and, and iterate on policies and processes and tools that make that all happen, uh, for free.

00:08:12

And you remove the friction essentially, and organizationally a program's office can live anywhere in the org. I've seen them in HR. I've seen them in, um, marketing I've seen in the CTO's office, but I think the best place honestly, is, is nearest to the engineers. Often you'll have a central engineering organization who run the builds and the other infrastructure. That's a great place for it, where some engineering leadership, perhaps even the CTO's office might be good, but either way, you know, this is a sort of how we looked at it at Microsoft. Uh, we had the Oz pool in the middle, and we were taking executive guidance from, uh, from a committee or a council that would help us understand the business and the goals of the business, and then consulting with, uh, legal HR security, accessibility, all of the subject matter experts in the company who have an opinion and a stake in, uh, how we do open source at the company. And then our job was to bring those groups together, uh, to actually relate to those needs and desires and goals to the engineering teams and put it in terms that the engineering team can understand and can utilize fully. And then of course, drive that to, uh, value for our customers and communities that we're engaging with, and also collaborate with our industry peers, uh, so we can evolve the norms, et cetera, around open source.

00:09:40

So just in terms of looking, you know, at scale and back in time a little bit, uh, Jeff Wilcox was one of the folks on the team and he wrote a couple of blog posts. Uh, this one's in from 2015, and it really talks about the transition from 2011 to 2015, where they, you know, from 20 people doing open source on GitHub to 20,000 Azure, uh, folks doing open source on GitHub. And then he wrote another post about a year and a half, three and a half years later, where he talked about the scale of going from 22,000 to 25,000 as we made open source, uh, happen across GitHub, uh, straight across Microsoft. And even now we're at 32,000 folks engaged on, on GitHub. So I encourage you to go and check out the blog posts. There's a lot of great stuff in there. A lot of great detail about individual tools we built and individual initiatives we did.

00:10:30

Uh, so I, I strongly recommend you checking that out if you're in the process of, of building an open source program and then to put a little bit bigger picture on the scale of what we were trying to accomplish. Of course, you know, Microsoft loving a loving, open source on the use of GitHub side. So for producing and contributing to open source and currently have around a hundred orgs, uh, 32,000 people, 7,000 teams and 22,000 repos. So, you know, the numbers don't really matter that much as much as to say that it's, it's a big scale. Like this is serious. You have to take, you know, you need serious amount of effort and understanding of what's going on tooling and process around it. And the culture change has really happened because we've got all these things and all these people happening, uh, working on, get hub, and then looking at the Microsoft side of things of where we actually use open source in the products, uh, today we're tracking about 10 million uses of open source across 10,000 different products.

00:11:29

Uh, it's 200,000 different versions of 50,000 different components. And we discover half a million new users every month. So like, that's the kind of thing that, uh, that is going on now, again, back to that scale question, you might say, well, I don't have that many zeros in my system. Right. And that's true. But if you look at how we got there and how we're doing this, there's a ton of automation and that's there that you can use to solve your problem, even if you're at the hundreds or thousands scale on some of these things. So we'll take a look at that now and really talk about how did we do that and how did it unfold? So the open source programs office, uh, was roughly about 10 people, a and there's another 20 or so, uh, subject matter experts, lawyers, security folks, et cetera, and partner teams building helping us build tools.

00:12:21

And I would say that, you know, from 2015, probably for the next three years or so, our main focus was really, um, uh, building tools and helping to, to make our, to get the process to, uh, uh, proficient as what we would call it. And I mentioned earlier that, you know, culture eats process, and while it's true, like when we started out the program office, uh, we thought, well, Hey, you know what we've got to do. We've got to go around and evangelize. We've got to go and tell everybody that open source is good. You should get engaged. You shouldn't get involved, you should use, use and contribute more open source in your team. And it turned out that people didn't need convincing. They are ever right on it. They were okay, let's, let's go do that. Uh, but it turned out that our policies and processes were so hard that they didn't actually believe that that's what they should be doing, because there was just so much friction that even if they wanted to, it was really hard.

00:13:15

And just as an example, uh, we had every, every time you wanted to use a new piece of opensource, you had to manually identify that piece of open source. And then you had to go to a tool and manually enter an answer about 20 questions, but really 20 questions per open source, uh, piece that you wanted to use. And some of these questions, like, you didn't know how to answer. You didn't know what they meant. Uh, you didn't know where to get the information. And oftentimes we give that information to, uh, to somebody and they would have to go and revalidate it or whatever. And so what we discovered is essentially we had to evolve the process and tools and the culture to enable the culture change. And then as the culture got here, we had to advance the process and tools more and just keep walking up that ladder, those steps of doing a lot, uh, you know, step by step or a sequenced, you know, evolution of culture, of all the process and policies evolve the culture and do those in a, in a coordinated fashion.

00:14:10

So looking at some of the activities that we did in that space, um, from a culture change point of view, we did a bunch of consulting. So teams would show up every couple of weeks and say like, Hey, I want to open source this key key piece of infrastructure or this key part of our product, or this key technology. Can you tell us how to do it? Can you help us figure that out? And of course we would, we'd love that, but also love getting paid. So I want to make sure we're actually doing it in the, you know, the right way and for the right reasons. So we would, we would ask the five why's, you know, why are you doing this? And et cetera, how do you expect that to unfold? How are you going to create a community? What you trying to achieve, et cetera, and then come up with a, you know, essentially, yeah, this was a really great idea.

00:14:52

Or maybe you need to tweak it a little bit or yet no, don't do that. Or this isn't the right time. That's really not going to fit in. And we'd also devise playbooks that would help people to do that, essentially that discussion in a self-serve fashion. You know, if you're doing a cloud service and you have these goals, then do it this way, that kind of stuff, uh, social events like meetups, et cetera, to try to get people together and talking with one another and collaborating. And then on the other side of things, it turns out that people really want to understand what's going on. They don't want to just like pitch stuff over the wall or take things in from, uh, from other places. They need confidence. You need data and insights to understand what's going on in their engineering system, what's happening with their product, et cetera.

00:15:37

And so we did an enormous amount of effort, uh, gathering data, generating insights, and helping give people that confidence of what's going on, uh, as well as engaging with the open source community as a whole, uh, joining foundations, sponsoring projects, conferences, et cetera, just trying to get more engaged with the communities and understand what it is they're doing and why and what they need, uh, et cetera. So we also then, you know, sort of, you know, that was a culture part, as we were looking towards the policies and processes, uh, we sort of evolved this mantra of identify, eliminate automate delegate. So we'd try to identify a source of friction, and then we would try to eliminate that friction. And I'll give you an example of going back to the, the open source use case where, uh, you're renewed, needing to register the use of open-source.

00:16:28

Well, it turns out that, you know, there was, it was pretty obvious, there was a ton of friction there. So we were identifying that friction that was easy eliminating it. Was it going through those 20 questions and saying, okay, do we still need this question? Who's the right person to answer that question. Can we get that data from someplace automatically? Do we already have that information in the engineering system? Uh, and it turns out that a ton of questions could be eliminated or automated in that fashion, and then things that couldn't be eliminated or automated, we would then seek to delegate those to the right people. So for licensing questions, for an example, uh, you know, devs often don't understand licenses. They don't know what they're about, what they're for, where to find them, et cetera. And so, uh, it turns out that if you ask a developer about the license and they give an answer, oftentimes lawyers don't actually trust those answers.

00:17:16

So they go and figure it out for themselves anyway. Well, okay, let's let the legal team do that instead of asking the developers to figure it out and, you know, get the legal team to do it. So get the answers or the, the tasks closer to the people who have the expertise and the skin in that game, right? And then this is a continuous, uh, iteration where, uh, you know, you have to keep on going and, and, uh, cycling over. So every time you find and, uh, eliminate something, you'll get closer to a solution. And ultimately you just keep on cycling on this and every month or so, we would find another way to eliminate some friction or automate something. We would just roll out a new policy or a new tool and keep iterating on that. And eventually we got to the point where, for example, uh, you see those 500,000 new uses of open source being discovered every month.

00:18:06

Imagine if you had to do manual or review anything manual for any it, for any sizable chunk of those, that would be horrific. So through this iterative process, we got to the point where 99.5 ish percent of those new new uses were handled through some automated process that did it all followed encoded business rules, using high quality data. So the human never touched it. We automatically discovered it automatically processed it automatically validated it in a way we go. And actually you don't have to dive a little deeper on that. This is the sort of life cycle we put together around using open source. So I'm not going to go through every piece of this, but you can see that if you look all the way left, uh, you can see that, you know, when a dev is discovering open source, I need a new Jace on parser for my, uh, for my application.

00:18:56

They're going to go to NPM js.com or new get or Maven or wherever they're living in their ecosystem. And they're going to start looking for things. Well, we have a lot of understanding about what the rest of the company is doing, because we know we've we've know everything that's going on. We can help guide them to solutions or choices that are already being used in the company that are known to not have vulnerabilities that are, have licenses that are known to be compatible with their usage scenario. And we can do that all the way, you know, shifting all the way left into their browser. When they're on NPM js.com, we have a browser extension that will tell them right there, it gives them a Microsoft viewpoint on the components that they're looking at and help them decide, guide them to a good choice. And as you scan for the right, you know, we're doing detection, automated detection of the open source you're using.

00:19:45

So the engineering system knows that the engineering system knows where the components are coming from. If it doesn't, you shouldn't be using them like seriously. But so the, so the engineering system knows that it can also ingest that it can go and get the source code and bring that in and keep that, uh, for historical purposes and it to enable, uh, future servicing, uh, et cetera, uh, keeping track of the registration of who's using what and running policies like I was describing earlier, giving people notifications, when something's out of policy or breaking the build blocking deployment, if something's out of policy, all of these things allow you to run really quickly, like we're used to in a dev ops world, but also run with confidence that, you know, what's going on with, with respect to open source. Um, and when it gets to the point of actually shipping your, your system, you might have open source, uh, uh, compliance requirements like shipping, a notices file.

00:20:41

Most licenses require attribution, or you might have to disclose source code if you're using a copyleft license. So those things can all be automated because again, we know the, the information about the components you're using, and then once you've shipped, you know, two months later, you, a vulnerability might be discovered in a key component that you're using. And you need to know where you're using that. If you're using a component in several thousand places, and it's got suddenly has a vulnerability, you want to really be able to go quickly and find out what your data centers is that deployed to so that I can go figure out what up was in, rebuild it, fix it and, and deploy it. And then ultimately, you know, once you you're, you've shipped, uh, being able to understand the nature of the open source and your interactions and integrations in your code is through dashboards and reporting is super useful.

00:21:31

So that's just an idea of, from a use point of view, there's similar challenges around releasing open source and contributing to open source, but this from a dev ops point of view, I think that use is a, is most relevant here. And of course, in an open source way, you know, as we went through building all of those things, we built a bunch of open source tools, and we contributed to a bunch of existing open source tools. And I won't go through all of these, uh, but some key ones, the GitHub portal for example, was a tool that we built. And open-sourced, uh, it's a great way of managing your engagement on source, uh, all your teams and your organizations and your repos, uh, when people want to make a re uh, report public, you know, that's an event, oftentimes it needs a review. You can do all that kind of thing through the, uh, they could have portal, uh, GH crawler is another interesting one where again, monitoring your open source presence on GitHub and gathering all the data what's happening.

00:22:27

Who's committing, who's doing pull requests. How long are they taking? What's, uh, you know, what's going on with my teams. All of that information can be pulled in through GH crawler, and then you can build your insights on top of that data. And then one that's near and dear to my heart is clearly defined. Uh, that's a means by which we crowdsource the gathering of all of the compliance data that we need. It, it turns out that compliance data around source is really hard to get, uh, you know, team different teams use different approaches for, uh, identifying their licenses of their copyrights. Uh, and then the commercial products in this space are often incomplete. And so we actually started an open source community to gather and curate that information, uh, in a sort of systematic way and make it available. And now there's, I think 10 and a half million different, uh, components are characterized in clearly defined.

00:23:21

So you can look to integrate that into your process. So speaking of that, uh, that's been a lot about how we did it, and now I want to cover, uh, briefly, you know, how you can create your own open source project, uh, program. And, uh, you know, an important point here is that the very, there's a lot of variations. You know, you have a different tech stacks, different communities or cultures, different business goals, you're at a different point in, in sort of adoption or engagement. Uh, so you have to sort of tailor your own thing. Uh, but this next discussion is it's a lot of context that I wish I had when we started the open source program. And so we're passed that onto you and hopefully it's helpful. I think the first thing to look at here is what's your value from a business point of view?

00:24:07

What is it that makes people excited about your products? What makes them spend the big dollars to buy your, the things that you sell? What makes them, you know, line up to get the next release? Uh, that's your secret sauce? That's the thing that differentiates you from everybody else? The rest of the stuff, the databases and the, and stuff like that under, under the covers typically are not your secret sauce. They're the same thing that everybody else could use functionally. Uh, and so those things you can get for free, you can engage with open source and create them. You can use them, uh, and spend more of your time focusing on your value and delivering that to your customers. So understanding that value is great. Uh, and then the next thing you want to do is then focus on what's your engagement level today? What are you doing?

00:24:55

How are you engaging with open source? And then you can look to say, well, okay, I want to get these other values. I want to evolve to this next level of engagement. Uh, what, what is it that I should do now? Everybody's notions here are different. So don't take these all too seriously. It's more of a, uh, a set of key points to think about. And that act of thinking about is actually the real value. Uh, you're not going to score any sort of level here. Um, and so, you know, just the thinking about it is they, the real point, I'm going to skim over a bunch of these, uh, the blog. I did this all on a blog post that's listed here. I encourage you to check that out, uh, and get the details of it. And so we'll go over these and, and in part, uh, this is a good opportunity to have a lot of fun with Lego pictures as well.

00:25:41

Um, so denial, obviously Microsoft was in denial about open source for quite a while. Uh, you can, you know, saw as attacking and countering and preventing open source, uh, quite actively, uh, over the years. Uh, hopefully that's all gone now. Um, and you know, you all might be in there and there might be pockets of your organization that are in there, and it's legit. We just need to help people evolve past that if they, if they want to, if that's one of their goals that they've identified. So you can evolve from there into a sort of stage of hype. Uh, and this is where, you know, this is the next best thing the CTO has read about it on the airplane in wired magazine or something. And so there's, this, this is going to solve all of our problems. We're going to hire a bunch of high profile, open source people, and we're going to drive an whole new culture, uh, and, and change everything.

00:26:26

Uh, oftentimes, you know, those fall a little bit flat, uh, cause there's not enough of a business model, not enough of an impetus under there to make it a sustainable effort. Um, of course, you know, Microsoft has fallen into that trap as well. Uh, and we've also fallen into the sort of tolerant trap of saying, okay, you know, this is a natural evolution stage, but it's, you've got a few people off in the corner, you know, here, we've got a couple of folks hiding under the space bar, uh, you know, doing some great stuff, but it's not at all widespread, it's pretty ad hoc. That's certainly not rewarded. They might even be beginning penalized for doing it, not producing, you know, product that they can sell or not writing more code or whatever. Uh, and so this is, uh, you know, like I say, a legit place to be, but you, you, I think you want to evolve past this pretty quickly because this too is not super sustainable people aren't rewarded are just going to leave.

00:27:20

Uh, you know, it's not going to take hold. So where I think a lot of people and up, and, and where we kind of aspire to in many ways is being proficient. Uh, this is where, uh, you know, it's systematic, everybody understands how to do open source. It's well, tooled it's, uh, efficient and engaged. People are engaged in both internally and externally across the company. And it's part of a lot of different products and people understand the value of what's going on here. And I would say today, the vast majority of Microsoft is at this level, they're, they're proficient to not that's in part because it's a tooling that we created and we've sort of gotten to that point in a culture change. Uh, the, the next level is, is fluent, right? As you think of this in language terms, you know, you've moved from just being able to read the menu to actually being able to have an argument or a discussion, uh, with, with people on a regular basis.

00:28:14

So you can, you can manipulate the concepts. You can understand the value of open source and weave it into a fundamental part of your business, uh, in an open and sort of, uh, engaged way. And you've just, you know, you've recognized that value. So you're rewarding. It, people who do open source, engaging open source are rewarded for doing so, and that means more people want to do it. And, and it just evolves from there. Think if, if you look@microsoftorquiteafewteamsatthatlevelvscode.net TypeScript, a bunch of the Azure folks, a few other teams around the company, uh, they're really at this sort of fluent level of, of engagement. And then you get to mastery, right? Uh, these folks are, there's relatively few of these, but these folks have got a fundamental core understanding of where open source can help them, what they can do, uh, et cetera. And, and they're using it to disrupt markets, uh, disrupt incumbents, uh, sharing they're sharing control of the open source projects that they engage with.

00:29:12

They're really getting proactive and integrating open source into the fundamental nature of their business model. So the question for you really is where do you want to be and how do you want to get there, right? And these sort of ideas that I've just outlined can maybe help you understand both of those questions and make some progressions. Uh, and then there's also a bunch of material out there from the to-do group, which is a collection of open source programs, offices that can come, that have come together, produced a bunch of case studies, uh, how to guides, even things like job descriptions for hiring and hospo, uh, staff member. Uh, so I encourage you to go and check out to do group.org, lots of great information. And, uh, that's about it for me. I hope this helped. I hope that you are excited about creating an open source program. And I look forward to seeing, seeing what you do in this great world called opensource. Thanks a lot.