00:00:02

Hey, everyone. Welcome to an encores. If you're here to learn about Anton's you're in the right place. Thanks for sticking around to the very end one last, I think the last breakout session of the day, what we like to do to start our stand-ups at Accela for some of our teams is giving each other a fist bump and a smile. So we're going to do that right now. We're gonna start this off with high energy. So look around you, give somebody a fist bump and a smile we're from DC. So I'm excited that the gnats are playing in game seven tonight. So there we go. I got an ass fan back there. Cool. So my name is Zach airs. I'm a scrum master at Accela. We're an it consulting firm based out of the DC area where we're in Arlington.

00:00:47

I'm Josh Cohen. I'm a senior developer at Accela, and Zach's going to go over our agenda for today.

00:00:53

So we're going to go over a couple of different things. We're going to kick off talking about our culture of experimentation and how we constantly look for ways to be learning organization, a learning project we're going to talk about and ons what they are, how we use it on our team. And we're going to go into a culture of psychological safety and why that was important to us. There we go. Cool. So it's important that we give you guys some context of our project. Uh, everything that we're talking about is in the context of us working with a, within the federal sector. So our project was with ISIS and we were a scrum team of about six to eight folks. Back when the project started March, 2016, we had a product owner, scrum master, and we were having a lot of success. And so they, they, they staffed us up a little bit more.

00:01:52

We doubled in size and it was at this time that we split into two different scrum teams. We were working within a single product backlog, if you're familiar with less, large scale scrum, that was the scaling model that we were working in, uh, after about a year and a half from when we started the project, we grew even more. So we're at three teams now, three different scrum masters, still working within the same product backlog. We're not going to go too much into like the structure of our team, but it's good for you guys to know this. As we talked about one of the experiments that we ran at the product level.

00:02:26

So that was a little bit about our history and, but we also wanted to go over some of our accomplishments during that time. So the first one is our ability to deploy frequently. We deployed over 10 times a day and this allowed us to incrementally release features. And that way we could get feedback from our product owner or stakeholders. And we could do that as soon as the feature was done, we didn't have to wait two weeks a month or sometimes even longer for a release to happen. Next is our ability to get code into production quickly. So if there was a bug that we needed to put in a fix a, we could take a code commit, have it go through a full suite of tests and have it live in prod in 20 to 30 minutes. And then the last thing I want to highlight is their ability to acknowledge incidents within one minute. So by being able to acknowledge incidents in such a short time span, we could come to a resolution faster.

00:03:20

So why am I mentioning these things? And the reason for that is because we think the reason we were able to accomplish a lot of those golden metrics is because of our culture of experimentation. So here you have a celebration grid that I'm going to run you through a little bit, so on the right you have practices. So these are things that your team has decided you should be doing and generally have successful outcomes. And so what's an example of that. So this isn't development related, but say you have some savings and you decide to invest that generally considered a good practice and hopefully we'll have successful outcomes, but with any investment there's risks. So maybe it might end up being a failure because the market crashes or something. So then on the flip side of the grid, we have mistakes. Things that we've decided are things we shouldn't be doing and tend to have negative outcomes.

00:04:16

So instead of investing that money, you go downstairs and the casino, and you decide to bet all of those savings away. So I'm not going to go into the odds of the different games, but generally we consider that something you'd want to avoid, but there is a chance it could be very, very successful. But the point of this is unfair practices and mistakes. These are things that we've already decided we should and should not be doing. So there isn't a lot of learning that happens there. So where we try to focus, our experiments is in the middle and that's where we have just as much of an opportunity to succeed as we do fail. And so that's where we also maximize learning, obviously.

00:05:00

So during one of our team retrospectives, we noticed that our cycle times were starting to climb. And we had a case of what we call the almost dones. Tell me if you've ever heard this before, but during standup, our developers would give an update on the feature. They were working the previous day and they said, Hey, I made a lot of progress. I'm almost done. And the next morning they say, Hey, I ran into some issues, but I worked through them. I just have a few more tests, right? I'm almost done. And this would happen way too frequently. And we decided this is an area we wanted to improve on because our teammates were only bringing these issues up at certain times, like stand up for example. And we wanted us, our team to collaborate when these issues were realized, not at ceremonies.

00:05:47

So this team formed in September of 2017. And it was at that time, one of the first retros that this team ran, where they said, Hey, we want to be a highly collaborative team. We want to have great communication. And they started working on different experiments that they could, they could do to foster a good culture of collaboration and communication. And as Josh said, we started noticing our team had the case of the almost dones and none of the experiments that they were working on really seemed to land and resonate with them. So as their observer and as their scrum master, I'd heard about this and Encore and I felt it was important to share the knowledge I had with the team. And what I'm going to take you through is an abridged version of a presentation that I gave this scrum team on an encores.

00:06:35

And I really want to highlight that I didn't tell them what to do. All I did was presented the information, the knowledge that I had at the end of my talk to them just said, Hey, what do you guys want to do next? So what we do is we started what, what is an, an Dawn before we talk about when an on-court is it's important that we understand the an add-on and what an Anton is, is it goes back to the Japanese paper lantern. It literally means a light. The end on cord was made famous in car manufacturing specifically at Toyota, there we go. Car driving fun. So at Toyota, they had a, they have a string along their assembly line. And when a frontline employee noticed the defect in the process, the Anton cord would be pooled. And the, and on the light, there would be a light that would light up on a light board.

00:07:27

What that does is it signals a manager to come, come check things out, a group of individuals we'll we'll swarm together. And that conversation will, will lead to a path forward where they work together. Uh, what made this culture unique at Toyota, where, whereas it someplaces, when there's a defect notice, you might have to write a longer report about a path to resolution at Toyota, there was a sense of gratitude where the manager, one of the first things they would say is, Hey, thanks for pulling the and Encore. What can I do to help you? And so they would work together to identify a path forward. So the team thought this was really cool. One to try this out and what did we do? Well, we decided we were going to have our own and Encore. We weren't going to have a string along our desk to pull, but we were going to create some kind of metaphorical and on-court for our team. We wanted to make sure that when the cord was pooled, everyone on the team would stop what they were doing and then work together to identify a path forward. Why would we pull the end on cord? Well, yes, if we noticed any defects, but also we wanted to make sure that we were improving our collaboration or communication channels. And so if somebody felt stuck or needed the help of the team, they would pull the Amnon cord. Then.

00:08:42

So with our experiments, we want to make sure that we're measuring success. And so things we're looking for is an improvement. In cycle time. We wanted to make sure that we were improving our collaboration. And also as Josh referenced, we didn't want to have the case of the done. And so instead of planning our collaboration, where we would wait for meetings or stand-ups or retros, we wanted to make sure that we were talking about things as they came up. So here's what we did. I don't know if the video is going to work or not. If not, I can talk through it, give us a thumbs up or thumbs down if it's going to work down. Okay. So I'll talk through this. So this is our slack channel. What would happen here is somebody would type and on. And when, and on his site, you get an asset here in the channel.

00:09:31

So you can see that here and Encore pooled, people will get notified. And so that was how we notified folks in slack, but we didn't just end there. This is a picture of our room. I'm bummed. The video's not working because we have a dancing two man at the end of this, but we had a, if this, then that integration with slack. And so what would happen is when, and on, is typed into slack, it would trigger if this, then that to some Amazon smart plugs that would turn on this rotating red light string, Christmas lights would be flashing on and off. And then we had one of those like dancing car salesman to man, uh, just going crazy in the office. And so it was fun when somebody pulled the cord, it was a, it was a fun experience, especially early on. There's a lot of joy that came out of that.

00:10:17

So I talked about how we wanted to make sure that we were measuring our success and visualizing our data. This is, uh, when we began the experiment on May 25th, 2018, this is what our cycle time looked like for the three weeks after we began our experiment. So hovering right around three days in progress. This right here, visualizes R and encores pools per day. So after about the second week we were running this experiment, we were averaging about 0.4 and Encore pools per day. So if you want to take a picture of this, I got, I promise I got a cooler graph to show you, this is the same data. Just, we're about to look at it at scale. So we were pulling the cord a lot early on, but what happened? And July that summer, about two months later, we stopped pulling the Amnon cord. But we noticed that because we've looked at our cycle time, our cycle time had an all time high.

00:11:11

So we were looking at 11 days and progress. So scrums and empirical process, we inspect, we adapt. So we asked ourselves what was going wrong? Why weren't we pulling the and non coordin? So we realized we liked point the end on cord. It was fun. We had a two man go, go nuts in our office that, that made us laugh. And it was great, but we weren't playing it out often. And it was because one I'm gonna pick on Josh. Cause cause he's here. If Josh was working on a story that he felt should be simple and everyone else thought it might be simple, but Josh felt stuck. Didn't know the answer. He didn't want to necessarily reveal that he didn't know what the, what the issue, the problem, the solution to the problem was. Another instance would be Josh is working really hard and, and sees it.

00:11:56

Everyone else on the team is heads down. They're either pair programming, momming, and doesn't want to interrupt and say, Hey, what I got going on is more important than anything you guys got going on. So I'm not going to interrupt what your workflow is. So we, we talked about that and decided we were going to change when we would pull the Anton cord to be whenever we need the opinion of the team. One of the things I love about this group is that we really enjoyed spending time together. We, uh, would go get lunch. We would go to impromptu happy hours every week. We would, we would talk about what our high for the week was, what our low for the week was. And so we started pulling the Amnon chord for non-development related things, in addition to what we had identified early on. So here's what happened. And again, before you see the next part of the story, remember we're pulling, we began to pull the, an Encore for non development related things. So, and Encore pulls go up and there's correlation there. And here's the rest of the story. So you can see how the Anton cord pools in the cycle time are kind of a leading indicator for each other. You guys can take a picture of that if you want to.

00:13:06

So it's a little bit of a heartbeat. Uh, we didn't get it perfect all the time. We had to continue to remind ourselves the importance of pulling the Anton coordinate and the importance of collaboration you can see in, in late December, early January, this past year, we hit zero. Again, we moved offices that was around the holidays. And so we said, Hey, let's incentivize ourselves for pulling the Amnon cord. And so Joe, one of the devs on our team, we decided we were going to pick on him a little bit and create a team currency. It looked like a monopoly dollar. When somebody pulled the, an Encore, they would get a Joe Buck, the person with the most Joe bucks at the end of the month got a traveling trophy. So we did that. And you see, we started pulling in on court a little bit more again. And, and so it was fun. Same, uh, uh, same data just shows in a table form. You know, how we, we started out with three days in progress with our, or our cycle time. And, you know, went through about 11 days when we stopped pulling the Anton cordon. And then the happy ending when we started pulling the and Encore more often and cycle time, it went to two days.

00:14:12

Uh, one of the big things for me as, as scrum master is looking at, um, morale for the team. And before we started the experiment, uh, with the Anton cord, our happiness as at a team level, started to decrease a little bit. So I was opening that by doing the Amnon cord, our happiness would increase. And you can see when we, when we kicked off the experiment, it was a fun project for the team. And they seem just to be a happier group of individuals. So not only did it improve our work productivity, but it also improved the team's happiness at work.

00:14:45

So Josh talked about the celebration grid. We decided as a team, this was actually a good practice. It was a good thing for us to do. And so we moved it from the experiments column over to the practices call and it was a good practice. And why did we do that? We'll improve our cycle time. We saw a greater psychological safety with our collaboration and we continued to inspect and adapt our process. And so I talked about how we didn't get it perfect right off the bat. We had the Joe books that we were doing, and we had a couple other experiments we did on top of the non cord just to continue reminding ourselves of its importance.

00:15:23

So Zach just mentioned psychological safety. And so if you went to Dustin and Nicole's talk on Monday from Google, they talked about how psychological safety was the number one element for high performing teams. And so we feel like the end on experiment helped bring psychological safety to our scrum team. And there was no longer a feeling that Zach mentioned earlier. Like if we have new team members, they weren't worried about bothering the rest of the dev team. Or if we had some teammates who liked working more independently, this helped encourage them to collaborate more. And so I think what's so great about psychological safety though, in addition to this is not only does it help improve our development work, but I think it applies to everything we do. So we found that, uh, teammates started coming up with creative solutions or improvements to our scrum ceremonies. And I think because we had psychological safety and they knew that this was a safe space to present their ideas to the team, that they were starting to think about those things in the back of their head, whereas opposed to the opposite, if this didn't exist, they wouldn't even try to think of solutions.

00:16:32

So up until now, we've told you the background of what an Anton cord is and how we used it at the team level, but there's other ways you could adopt it on your teams. And so we want to present how we used it at the product wide level. And when we use it at the product wide level, that was reserved for what we called code red incidents. So that was when production had an outage or something that was critical. That was similar to that. And so when the hands-on cord was pulled at the product wide level, everyone stopped what they were doing and they tried to figure out how they could help. And this was really beneficial for us because this gave us the key personnel we needed to resolve issues. So somebody was familiar with an area of the code base that was causing problems or had certain subject matter expertise that we needed. They were then available for these critical issues. And then once we assessed what the problem was and started working towards the resolution, if you aren't necessarily needed, you could go back to what you were doing.

00:17:31

So we're here at, uh, Gene's conference. And so we're going to talk a little bit about the three ways of dev ops and how we able to tie that back to our team and sort of, as we were discussing what we learned from our experiment, we talked about three ways of dev ops. And so for us, we looked at how we were able to improve our flow, um, specifically the cycle time, but more importantly, how you're able to visualize that with your teams and with your organizations, um, is what made it really powerful for our team. And so, you know, we have a physical task board where you see stickies going across the board each day. We, we visualize our cycle time and our shared office space. And we also visualize our, an Encore pools per day. And at every single retrospective, I would have a printout for the team of that graph that I showed you guys earlier of, you know, what was our correlation between an Encore pools versus cycle time on a sprint level, on a monthly level, so on and so forth.

00:18:27

And they, and on experiment helped shorten and amplify our feedback loops. So instead of waiting for daily standup to present your issue, you were reaching out to the team and getting help. Once those issues were realized

00:18:40

What's most important to us is the culture that we have of continual experimentation and learning. Uh, like I said, we, when this team started in September of 2017, we started trying to run experiments to address collaboration, communication. And when nothing seemed to land, we didn't give up and wave the white flag. We continue trying to run different experiments and chip away and finally land on something that was impactful and meaningful to our team. So let's give you guys some advice on, on bringing this back to your teams and your organizations. Um, so five, five pointers that we have, uh, the first is coach, uh, use the celebration grid. It's you can find it@managementthirty.com. Um, that link is also in our slides if you check those out, but, um, that celebration grid is from Juergen, apple, and you can check that out, but, but coach your teams on the importance of the celebration grid and the, you know, experimenting and that equal chance of success and failure and the power behind that,

00:19:46

And then teach your team, educate them on what an Anton cord is and what are the different ways that it could potentially use them on their scrum teams. But just as importantly, ask them if this is something they're interested in, we thought this was a really cool idea. So we decided to implement it, but if it was forced upon us, it wouldn't have been as successful.

00:20:07

So you saw after two months of us running this experiment that it crashed to the ground. I mean, we weren't pulling the Amnon cord at all. Our cycle time is at 11 days. And by all standards, it felt like a failure. Um, but our leadership, um, at our product level and, and at Accella has given our team the freedom to feel empowered, to fail and fail often. Um, and so having that freedom allowed us to inspect and adapt and iterate on our experiment to create something that was more powerful and more meaningful for this team that, that had an amazing impact. Finally, we, we had a lot of fun with this. We start single day with a fist bumps and a smile. We have a dancing tube man in our office with a police light and Christmas lights that go on and off. And so this is yes, it's meant to kind of call the attention that somebody needs help, or somebody just wants to talk about something, but it was also supposed to be fun for me. I was my vision of the Amnon quarter was to have some dealings with as obnoxious as possible. Um, I would love to have, you know, an Amazon Alexa that starts saying and on unfold, um, something ridiculous and it's supposed to be silly, supposed to be fun. Um, so just trying to create joyful moments and you're in your office place.

00:21:28

So we haven't figured this out perfectly. We still have some, some problems, Josh and I are on a new project now, but some of the issues that we were facing at the end of this project was one pulling the Anton cord on a teammate. We felt like that was sort of the next tier of psychological safety that we were striving for. So if Josh hadn't pulled the and he's been sort of, I get the sense that he's stuck on a story for a couple of days as his teammate, I should be empowered to pull the cord and say, Hey, I think Josh has stuck. Let's talk and let's make sure Josh gets to help because it, what that speaks to is a level of team driven accountability instead of individual or top-down accountability.

00:22:11

Uh, we, we would onboard teammates. You can see that we continue to scale. And so we were constantly bringing on new teammates to our team. And so introducing them to the unique culture of the Amnon chord and just our culture of experimentation, getting that buy-in and getting them to accept it as something that was just as important to them as it is to the greater team. That was a challenge for us at times, but, you know, ultimately it worked out and, and finally Josh and I were on new teams now. And so we're going through the grind of that five step process that we laid out of coaching, teaching, asking, and not telling, providing freedom and having fun. We're trying to sell that to a new group, different personalities, different individuals, different different backgrounds. And so getting them to see the importance of experimentation and maybe even the an on-court is something that we're struggling with right now.

00:23:02

So we're really proud of this experiment and want to share some team insights with you. These are actual members of our team. Jen is actually one of the leaders on our project. This is a slack that she sent us before we came here. She, uh, she kind of spoke to the happiest metric that I referenced earlier, where she saw the morale boost with, from the team working together. Uh, she said that it offered an outlet for all types. The inspiration of the Anton cord was just what the team needed to become more reinvested in the project overall. So the code red and on that Josh spoke about would not have happened if it wasn't for the leaders that emerged from this team and not was there the code red at the product level, but the other two teams within the product adopted their own type of Anton coordin that happened because the same members of this team that were their happiness was declining. When we began this experiment, you know, it gave, it, gave them an opportunity to show some leadership at the project level.

00:24:05

And so now let you guys hear from the members of our team, Andrew talked about how he was surprised that the team embraced the idea of reacting positively to all the and Encore pools and not being bothered. I'm sure it's a question that you all are thinking right now. If we introduce the, and quarter, people can get ticked off that they're being interrupted, but you'll see in some of this feedback that they actually, the team actually wasn't bothered by it in wind to help. Cause Brian talked about how using the non core to allow the team to identify and resolve critical issues in blockers, much faster. And Andrew spoke about how the lights were just distracting enough to get his attention. We're excited to hear from Matt, that the team members, again, didn't mind being interrupted in order to help a fellow teammate Nick's on increase in collaboration. And Andy agreed. He saw a higher level of engagement from communication. There was more pairing and swarming that happened as we pulled the cord and we respond to incidents much more quickly.

00:25:06

So we reached out to everyone through slack, hoping to get some feedback about how they felt about the Amnon experiment. And we love Joe, but Joe's not very good at checking slack. So we actually pulled the Amazon cord and Joe noticed all the lights going off. So he asked, Hey, he stopped what he was doing and then asked how he could help. And so we told him, Joe, you're not checking slack, but we're actually looking for feedback on the, and on experiments. And so he told us that he knew this would never work, but then he admitted that he was wrong. It did, we had data to back it up. He found that the team was more engaged, problems were addressed sooner and productivity was higher.

00:25:48

So at the end of the day, this is just our experience with a cool experiment. Uh, there's probably a million different ways of doing this. Um, we we've know use slack as our communication when we're not, when it's not face to face. And so this was something that worked for us. We would love to hear other stories of, of folks adopting this experiment within their own teams and, and continue the conversation with you all about, you know, issues with collaboration, communication you guys have within your teams and organizations. And if this is something that you guys want to try out, we would love to hear your stories. Um, I think we have a couple minutes left if anyone has questions. Yeah.

00:26:29

You know, you, you, you saw in three-year experimented cycle, timeless produced. Did you also find an increase in quality?

00:26:38

Yeah. I saw your cycle time reduced, did quality increase

00:26:42

As well as to production.

00:26:45

So I would say it, the quality didn't decrease. Um, and so it wasn't like we had, because we were working through things more quickly, we saw, uh, more bugs coming through our system, the nature of the work evolved as the project went along. And so that would be, uh, for us, that would have been a hard metric to look at. Cause we, you know, sometimes we weren't always deploying or we weren't always working on something that was going into production and being integrated with interacting with users. So yeah,

00:27:23

Hello, I'm your code red. And then how it was the product team,

00:27:30

Uh, the product team, uh, I would say we had at any given point in time, we could have, you know, 25 to 30 people at the product level. Um, what would happen when there was a code red and on polled is everyone would read the slack and stop what they were doing. But the code red conversation would offer context and slack as to what the incident was. And so as Josh talked about, the Don helped us, you know, deploy the right personnel to figure out the issue. It just really help you kickstart those conversations.

00:28:03

I noticed in your data set. So there were a lot of ups and downs in the pulling of the end on cord. And you said it might've had something to do with fear sort of re-establishing itself. Did it ever actually stabilize and just become a regular behavior? And if so, if not, why not?

00:28:18

Yeah. So what you're seeing is from the beginning of the experiment in may of 2018, this team was no more in may of this past year. And so we scaled down the project and you know, are now on two different projects. Yeah. And so it was, it was up and down the entire time. And so that's kind of like that next level of psychological safety that we were talking about, that we felt that if we were able to start pulling the on cord on each other, that maybe it was going to stabilize, but we never got to figure that out.

00:28:47

And even if we stop pulling it, we always, we knew it was a good thing. And we decided it was a good practice to have. So we try to find ways to bring it back.

00:29:01

Awesome. Well, thanks so much. Uh, I think we have a break after this, so yeah.