Virtual US 2022

Endless DevOps (and DataOps) for Data Ecosystems at National Bank of Canada

Sharing takeaways (success and failures) about:

• leadership and software engineering throughout a 5+ years journey in 2 different ‘data’ departments at NBC (National Bank of Canada).

• the complexity to kickstart a software engineering community in an existing IT organization (100+ teams, 3000+ employees & consultants) and its coexistence with established DevOps communities.


Also in the mix, the importance and role of organizational culture and diversity, specifically at the department level, in management (and technical) IT teams.


Context and use cases will be based on those departments:

• Corporate Data -- IT only -- 50+ employees & consultants -- prioritization done internally.

• Master Data Management -- Business + IT -- 150+ employees & consultants -- prioritization done by Business PO (Product Owner).


Some insights, reflections, and updates from my previous DOES presentation in 2019 will also be shared.

MC

Maxime Clerk-Lamalice

Senior Director, National Bank of Canada

Transcript

00:00:05

<silence> Hello, Bonjour everyone. My name is Maxim Clerk Lames. Thank you to the organizing committee for the opportunity to speak. Again. I'm a software engineer by training based in Montreal. I have been helping software teams for many years now in the startup world for around 10 years. Then N B C, national Bank of Canada since, since 2016. So here are some facts about N B C, which was created in 1859 and grew organically and through many acquisition. As you can see on the timeline, we are now serving 2.7 million clients. The bank's mission is to have a positive impact and to do so, it operates for a line of business, personal and commercial banking, wealth, financial market, US specialty, finance, and international.

00:01:11

So let's start with some word about what's new. Since we last talked back in 2019, I had the opportunity to present in Vegas in person. What a great experience to share and learn from peers. In the last three years, I have worked with many groups at the bank and also with many providers and third parties, but most of my time was spent with those two teams, the data warehouse team, and a data platform team. So basically a lot of software and data. My talk will be about both teams, but also shared services or enabler groups at the bank.

00:02:00

Before jumping into my experience report, let's do a quick recap about my 2019 talk. It was about a three year transformation journey from a classic model dev versus ops to a full DevOps model. It was about a fun and performing team and how to measure what matters. It was also about converting challenges into opportunity. Not an easy task, but a required one in 2021. During covid, I joined an existing team to help put the predictability of the, of the delivery, you know, a predictable scope, timeline and budget for all squads perfect asset. I have a plan. So with my colleagues using my 2019 DevOps transformation roadmap, we planned and

00:02:59

Compress a timeline to a quick six month thinking it'll be done and fun. Now let's talk about what we learned before. Going further. A short disclaimer. This story is not about design pattern, service, mess, data, mesh platform migration on-prem versus cloud. It's our reality and we deal with those topic on a daily basis. Let's start with some context about my current team. M C P stands for master client profile. It's a program and a platform integrated by the bank. Here are some facts. It is an M D M, which mean mastery data management platform, and it is in production since more than two years. For the technology stack, we are running on hybrid and front mix of on-prem and cloud, A lot of Java code, some classic platform applicative providers like Oracle I B M with cloud deployments. For the newest component that we're adding team size, it's around 75 on the IT side and the equivalent on the business side dependencies. So this is where it gets, uh, interesting. More than 70 project or line of business are involved at all time ranging from integrating new data to adding actually new features. Were vi very popular. So basically we're surviving only the correct information, the golden record about customer profile and making it available through modern patterns. Fun fact, we are exposing many, probably the most endpoint in the N B C ecosystem.

00:05:01

So why am I speaking again? Well, it's about taking and sharing a snapshot of our culture and of course brainstorming with leaders. With all of you, we'll still transforming and we will never stop. So we are sharing ideas and transformation stories. The topics are not necessarily coming from major failures, but where we put some extra effort. Here are the topics that we want to share with you today. Onboarding employees, category of work, governments, committee, experimentation. So those are a mix of ideas, observation, use cases, topic or simply experience that we find interesting to share with all of you. And especially we want some feedback. Note that I'll navigate between the team, the program, and the global IT organization.

00:06:06

We often say that it's all about the quality of the integration. Since it's true for our system, our team member onboarding process is following the same role. So let's review for a specific role. Typically an individual contributor. We are of course doing the classic welcome breakfast with the squad, sharing documentation, weekly links, recorded knowledge session, and having a mentor to help for the first two weeks. And we keep focusing on the experience a bit like DevX developer experience, but not only for developers for all our roles. Also helping for the general onboarding process. We have team meetings and community of practice all within our program spanning our program. All roles have community of practices at the N B C level, making it more fluid to exchange ideas and be exposed to new use cases. So what about metrics? Well, we are measuring our onboarding with the following time to first deliverable.

00:07:16

It could be a commit, a design, a requirement, a use case quality, so bug number of rework. It's very specific to a role and also the fun factor through surveys and feedback. Then we wanted to simplify the workflow, onboarding workflow in the sense of how people actually work across role within a squad. So to bridge the gap between roles within the same squad, we created a platform specific automation framework allowing for traceability from the business requirement up until the integration test, bridging the gap between the business requirements to the software deliverable and then measuring it once it's placed. The feedback loop did increase naturally and it was a collaborative effort within the squad. So in summary, starting with a tool that then helped processes and ultimately was about bringing, bringing the squad together, that was a great side effect. Now you think before talking about the second topic, it's interesting to know that at the program level we have long lasting squad with fixed capacity evolving towards you build it onic, super classic DevOps evolution. Until a year ago, we had a challenge to progress at the right pace in our transformation initiative. We needed a way to

00:08:54

Be more systematic about working on the evolution of our platform and to protect a certain capacity. It was very variable for Sprint. And so by PI product increment, the solution was to add a notion of category of work. It could be business integration ops, SS D L C measured in ratio of capacity. Then we spent some time to categorize our initiative and of course explain and promote this approach with the full team taking the time to change management. Then the fund began, we negotiated the actual ratios by by category for each print with our pos, having the right and sometimes hard conversation about squad level capacity and also program level expected progress in all those categories of initiative. It helped us drive the conversation with the business scope, budget and prioritization. And we are now reviewing those metrics at the beginning and the end of each iteration. So in summary, there are newly added categories. Help everyone to have the same mental model about our capacity and scope. Plus it helps set the expectation on the planning level.

00:10:25

For this third topic, we are talking about a governances committee. The scope of this committee was for all it at the N B C level. The word governments is important here because the goal was not to replace the natural or organically existing communities, chapters, uh, group, but to align and possibly rationalize, uh, tools and processes having a positive impact on devex developer experience. So here is some context. Since many years, many communities of practice existed and a lot was created at different level of the organization. For example, DevOps, uh, chapters by delivery towers, community by role, et cetera. Starting in 2019, I was involved in the first iteration of this government's group. The typical pushback that we got was not sure why we need that kind of committee. We tried many variation of tool, content and scope for this committee. For example, invite only voluntary participation, o n I broadcast Yammer, channel polls and surveys, contest, technical publication, self-assessment, expert

00:11:52

Audit. Long story short, after many iteration, the committee is now in place and having an impact focusing on the mindset processes and tooling. So here are some lessons learned while creating this community, this committee, going back to the basics, mission and terminology to ensure common understanding, it needed a dedicated permanent core group because ad hoc and best effort is not enough. Use industry standard to fast track and then personalize. Avoiding the reinventing the wheel and endless debate executives sponsorship was required at least at the beginning to get representation from all departments. And then lastly and most importantly, focus on common and most popular blocking issues to rally everyone around it. And remember, it takes time to create a momentum for that kind of governments committee, but it'll get easier. Think about the flywheel effect.

00:13:09

The fourth topic is about experimentation. As a leader, like many of you, I had the opportunity to help different group within the same organization. The help comes from sometimes in the form of past experience, good call, prior success, knowledge of the organization context, but other time it's in the form of let's try this as a team and then end up based on the results. When I did present in 2019, I did not realize it at the time that I was experimenting a lot because of multiple factors. Here are some of them. Size of the team around 75, the low coupling of our asset in the bank ecosystem. It was a backend of a backend executive trust.

00:14:04

And in this period there was a mix of experiment. For example, creating a new pipeline to spin up data environment, new squad structure, merging and redefining roles, new feedback processes, taking, uh, the time to talk about failures with the wall of fail. And in retrospect, it was quick to implement even with the famous corporate taxes. Of course, team size, employee consultant, ratio, prioritization, process, line of business impact and team maturity greatly impact the speed and quantity of those fermentation. Here are some recent use cases in my current group. And remember, I'm not alone in those use cases. So creating a vision and a roadmap for new tools and then putting it in action with a P O C or p o v point of value. Reviewing the format for individual and team follow-ups, the frequency structure plan versus ad hoc versus check-in. Diversity of the team through recruitment.

00:15:19

Important not as a buzzword, but as an accelerator to help gen drive the change and also be challenged. That's very important. Hackathon our team tournament to develop the team's mindset. Learn to uh, to solve unplanned challenges as a team. Quickly reviewing with the team expected target, ss, l o, ss l a, KPIs and flow time adding or removing to simplify measuring and reviewing data quality. Same as code quality for software component. Because code and data are it coupled. Communicating the teams accomplishment and creativity outside of our department. Also in the context of experimentation. Being an early adopter of new product out of the box or internally developed requiring often inner sourcing or entrepreneurship is still hard and still high cost and still a good deal because of the impact and team motivation. Remember to fail fast and learn fast. So here's my summary. As a leader, we have a responsibility to experiment in those three class spheres, role and structure, processes and technology. Please do so while mentoring your team. So I'm ending with one simple question, very specific to a data platform team, but could be applied to your context. Have you ever integrated merge data quality teams and software development teams? If so, why and how? What was the result? So let's continue the conversation. Thank you again for your time.