Supporting the Delivery of COVID Vaccine in Wales: The Welsh Immunisation System

Congratulations again to Sinead and Richard.

I love that they have a video from their CEO, Lutz Schüler, congratulating them on the work they've done, showing that their work matters to people who matter.

Incidentally, they are only the third talk to have achieved this, joining American Airlines and eBay. And by the way, if any of you have a technology story that your CEO appreciates and is willing to talk about it, please tell me about it.

I have a conference that I'd love for you to tell your story at.

Okay, here is a story behind the next talk.

In January, I saw someone post something on Twitter who mentioned how the flow metrics described in Dr. Mik Kersten's book, "Project to Product," explained how hugely valuable it was to minimize the separation between the, quote, "technology and vaccination teams." So you can imagine that this caught my attention. So I reached out to Dr. Anne Marie Cunningham to learn more about what she was referring to. And what I learned was absolutely amazing because she and her team of 15 people were instrumental in helping get Wales to be one of the very fastest countries to get vaccinated, as well as having one of the highest percentage of the population being vaccinated.

We all know how important this is for every society, providing a critical capability for the most dire medical emergency and dire economic crisis in 100 years.

Regaining any degree of normalcy is simply impossible without it.

I thought of how incredible it would be to share her story here at DevOps Enterprise, and I was so grateful that she said yes. So Dr. Cunningham is a medical doctor, a general practitioner, but she also has been an academic researcher.

As Associate Medical Director for Primary Care for Digital Health and Care Wales, she helps build software, which includes the scheduling system that supports all child vaccination, which it turns out can be a huge part of the puzzle when you're trying to vaccinate entire population. So here to tell her amazing story of how this helped nearly get every citizen of Wales vaccinated is Dr. Cunningham.

Hello.

I'm here today to talk to you about supporting the delivery of COVID vaccine in Wales through the development of the Welsh Immunisation System.

I grew up here in the Mourne Mountains in Northern Ireland and studied medicine in Belfast and trained as a GP.

You might know that as a family medicine doctor.

I then moved to Cardiff in Wales via Bristol because I was considering retraining as a public health doctor.

I changed my mind, but I am glad that I completed my master's in public health because I've always enjoyed working in health at a systems level, as well as working with individual patients.

For 20 years, I have worked as a GP in this village in the South Wales valleys, Gelligaer, and alongside that, I have held various other roles.

For 13 years, I worked in the medical school as a researcher and an educator.

I was lead for the use of technology in the undergraduate curriculum and then at one of our statutory health boards as a primary care clinical director, it was named after Aneurin Bevan, the founder of the NHS, before I joined Digital Health and Care Wales as the primary care clinical lead five years ago.

In this post, I would call myself a clinical product owner for several products that we work on. And by that I mean that I work with our users, our stakeholders, and our development team alongside the product owners. But before this, I cut my teeth on digital health by attending and then running NHS Hack Days.

These are two-day hackathons which bring together technologists, designers, clinicians, patients, and the public. And these coincided with the start of the Government Digital Service 10 years ago.

It was a time whenever expectations were really being raised off public sector IT and digital delivery, and that applied in the health sector as well. So one of our strap lines for NHS Hackday is Geeks Who Love the NHS, but the other is Making NHS IT Less Bad. So given that, when I took my role in Digital Health and Care Wales, one of my friends said I had moved from being a maverick to mainstream, but there's still some of the maverick in there. So now I'm going to take you back to the summer of 2020.

Springtime had been very hard.

Everyone was exhausted with the changes that COVID had brought, particularly those working in health and care.

I had been leading our primary care response in the organization, doing what we could to support staff who were changing the way that they worked at an exponential rate and supporting data initiatives such as our Shielded Patient program, where we used data, clinical data about the population to try and identify those who were most likely to be vulnerable so that they could get added support. But unfortunately, thousands of people across the UK and in Wales had died from COVID, including the most vulnerable living in care homes, and the elderly. But by the end of June in Wales, we had had four days where there were no COVID deaths.

We had survived the first wave of COVID, but we did expect a second wave to come.

We just didn't know when.

Still, there was a sense of measured optimism about.

The UK government was starting to purchase millions of doses of vaccines, which were yet to be approved for use, but there was a belief that sooner or later they would be available. And the Welsh government had asked the seven health boards in Wales, who have a statutory obligation to plan for their population's health, to start getting ready for the biggest vaccination program we were ever going to see against COVID. So what we did know was that COVID was a devastating illness and that vaccinations were going to be one of the ways that we were going to be able to stop people dying and getting very ill. But there were many things that we did not know.

When was the first vaccine going to be available?

Which vaccine would be approved first?

How many doses of it would be needed?

How long would the interval be between doses?

Could vaccines be mixed and matched?

How could different regimens be managed together?

Who were the people who were going to be prioritized for the first available vaccines?

We knew it was going to be those with the highest risk of death, but at that time, some of the research hadn't been done to work out actually who those people were. So there was a lot that was really unknown, including where the vaccine would be delivered. And that was really important because the usual vaccine campaigns that we have, say, for example, getting your flu shot, go to your GP practice. So we thought that it was unlikely that these settings were actually going to work for some of these vaccines because we knew that the properties of the vaccine did not lend themselves to that kind of delivery. So everything that we were taking for granted about our infrastructure for health was maybe not going to be available to us for this. So there was much uncertainty, but we knew that we needed to be able to maximize the flexibility of any software that was to be used to support the vaccination campaign.

We would need to be able to identify and prioritize those cohorts.

We would need to be able to call people for vaccination and then recall them whenever they needed a second dose or they needed a booster. And it was really imperative that the software could be used in any setting. So this meant that the good thing was that we had quite a lot of experience about all of this in our organization.

For more than 20 years, we had been working and responsible for developing a child health system, to be able to make sure that your children got vaccinated. So whenever they hit a certain age, you'd get an invite in the post.

When you needed your next one, it arrived. And that was a tremendous strength for us.

It meant that when we were thinking, what were we going to do?

Were we going to go out and buy some software even though we didn't know all of the requirements for it?

Or could we actually build software?

It meant we really had a very big head start, and a lot of that head start came down to my colleague, Gill, who I'm going to mention now. And it meant our confidence, which I'm going to explain to you why it was there.

It meant that we were the only part of the UK to develop an in-house solution using an existing software team. And we knew that our time scales around that were less at that time than- It actually was delivered within 16 weeks. And we thought that it was actually going to take less, that we were going to have less time than that. So it was a tremendous kind of very brave effort to actually commit ourselves to that. And the reason we did it, and the reason that there was that support was because when the subgroup of the COVID Vaccination Board was considering the options, they knew about our background in this, and they knew Gill. And Gill is my colleague, who unfortunately can't be here today to co-present with me, who was a product owner for the product.

She has 25 years or more experience in actually working in supporting mass immunization.

She had worked on that child health system, CYPrIS, and its forerunners back to near the very start of all that work. And when the data and everybody sort of thought about this, they knew that Gill would be able to deliver.

Gill had built, and her teams had built many services, run them, operated them, fixed them.

They were a stable team.

Gill knew all of the team inside out.

She knew what their strengths were, what their weaknesses were.

There was very high trust amongst her team, and they were trusted by our organization and by our partners.

They had empathy for their users because they supported them, and they knew the challenges that they faced. And they could work in this truly agile way because Gill was comfortable with taking on the responsibility along- I did help her with it, with of actually making the decisions and prioritizing the backlog along with the stakeholders. So all of that meant that Gill and the team were not just bringing technical knowledge to the vaccination program, but also knowledge of the business.

In some ways, I would say that Gill knew more about how to run a mass vaccination campaign than the teams that were planning the campaign locally.

For example, Gill foresaw that one of the really key elements to this was going to be these, what we called system-generated appointments. So that meant that you set up your clinic sessions, you had your cohort of thousands of people, and you pressed the button, and you had populated all of those sessions.

Sometimes the health boards were not really thinking that they were going to run the program that way, but Gill knew, and she was right.

Because of the 7.25 million vaccines that we have now given out in Wales to a population of 3.1 million, over three-quarters of those were delivered in our mass vaccination centers. And they were delivered and scheduled, a lot of them, apart from some at the very start, through our application.

I remember one of the days, because we were going along to these tabletop exercises, virtual tabletop exercises, and that was where we were learning about what the health boards were planning to do with the program. So sometimes as Gill said, it was to glean what the requirements were, but actually, a lot of the requirements were known to Gill and to the team. And so in a way, we were just checking in to make sure that what we were thinking was going to be needed was not going to break anything for the plans that the health boards were coming up for. And after one of those sessions, Gill was saying, "Do you think it would be useful if we built in a service where we could send people an SMS to remind them of their appointment?" Because at that stage, we were thinking, "Well, we'll generate all these appointments, and then the health boards might do a mail merge or something, and they'll send out these letters." And I said to Gill, "Look at the gov.uk notify service." And she came back the next day and said, "It's brilliant.

We can integrate with this really easily.

Get this set up, get all the SMS sent." And she said, "But also, they do a service where they send letters." So the old snail mail. And that was important because many people, certainly at the start, were going to need information about the vaccination campaign in a way to reassure them. And getting your letter through in the post, landing on your doorstep, telling you your vaccine's here and it's ready for you, at a time also when there was lots of worry, I guess, about SMSs and things, and were things a scam, and were people going to charge you?

It was really reassuring for people to have that. So we went from actually scheduling all of these appointments, and sending out altogether, I think about 4 million letters have gone out through our service to people.

That was at our last count last month.

I think it's even up higher. And 15 million text reminders have gone out through the service. And the team, as I said, Gill knew the strengths of her team, so she was able to put some of the team actually on making sure that back-end service worked for all of the health boards. So they would sometimes pick up on things like, "Did you realize that you had set up that clinic and you hadn't actually scheduled anybody into it?" So it was a full service that was being delivered along with the more technical aspects of actually having software that worked in any location. And we feel that that has been a tremendous strength for us, and I think it shows the vision of whenever you have people with really deep-setted knowledge who are able to contribute and see sometimes the things that other people can't even see. So the Welsh Immunisation System was born, and it went into use.

We didn't have time to do all of the testing and everything, and some of the elements.

Well, when we did some of the testing just before we went live, we picked up on some issues from some of our health boards. And to be honest, some of our health boards did not think that we were going to have this ready in time.

They were setting in place other contingency measures to make sure that they would have ways of recording and managing. And they just really couldn't believe it, that we could start and build something to be able to do this. But on that first day, on the 8th of December, I was out in one of our vaccine centers in Aneurin Bevan Health Board, where I work as a GP, in a leisure center that was now given over to working as a vaccination center. So you can see alongside there, some of the staff getting ready.

This was just before they started.

We started with a bit of a slow start, very sensibly. And they were there all ready to start recording the vaccines that they were administering. And that was one of the health boards that was a little bit unsure about it, but they became one of our biggest supporters.

This is how it looked.

I don't know why I've taken this photograph at the time, but it was Cwmbran Stadium there on that day.

It was a web-based application.

We were using 365 for authentication.

It could be used anywhere.

A few days later, I went up to one of our smaller centers in a different health board where the staff were being inducted into it.

There were some people who were a bit upset.

They were thinking, "Well, maybe we should just record the vaccines on paper." And I was actually going in and out with some of the other team, just sort of showing them how easy it was to record live into the application, and they all became converts and were there going and managing to show others how to do it. And when I say every setting was used, this was a few weeks later, and this is outside Cardiff City football stadium, where one of our clusters of GP practices had decided to try...

I don't want to say they tried.

They actually did use it, having a drive-in clinic. So here is, over the web, the team getting set up to be able to record this vaccine.

It was a very cold day.

It was just above freezing. And people were coming up in their cars and being jabbed, and the vaccine recorded there and then. So we felt tremendously glad and happy that we had succeeded in our plan of integrating scheduling with the recording of vaccines. And a key element that came up as a requirement, not at the very start, was actually the tracking of the vaccine stock. And that application, as I said, was used in every part and every setting in Wales. So to give you a little bit more information about the tracking of the vaccine stock, we in Wales, at the start, everywhere in the UK, the constraints around how fast you could go with the vaccination program were what staff you had, what buildings you had, what capacity you had, but more than anything, it was your vaccine stock. And in Wales, we managed to have it that as soon as the vaccine arrived in Wales, it was only three days until it was into people's arms. And across the rest of the UK, that timeframe was about 10 days or more. And the other thing that happened was that we had no, or next to no vaccine wastage.

There was a tremendous tracking of the vaccine stock.

It was being recorded into the system.

It was being made available to the pharmacists across.

They had complete confidence of where the stock was and how it was being used.

It was such a great success that they really do want to think about doing something similar for other vaccine programs as we're going forwards. And that flexibility, because we were building it ourselves, it had never been a requirement before.

You're doing a childhood set of immunizations.

You don't need to know exactly where the vaccine stock is.

It's a planned program.

You don't suddenly get a big rush of needing a lot of vaccines in one day to deliver to people. But here we were getting short life stock coming in that maybe needed to be used within a period of a few weeks, and there was complete confidence that we would be able to do that in Wales. And we did. So it wasn't just of course the development team and the support team that were there.

There were other teams that were really involved in this within our organization and across. So our information teams were centrally identifying the cohorts to be vaccinated.

They built dashboards which let the NHS and others know how they were getting on, also made available the data, which could be then reused by others to make up their own ways of looking and slicing and dicing at the data. And we're still doing that.

I was just working earlier on with our team, where we were looking at people that are immunosuppressed and are due to get an extra spring booster vaccine. So we're still working through all of that.

We know a little bit better what we're doing now, but all of that work is still going on.

One of the key things that we did was to build vaccine payment for primary care. So I've mentioned already that well over three quarters of our vaccines were given in the mass vaccination centers.

That was for all kinds of reasons around the kind of vaccines that needed to be used and the suitability for smaller centers and stock and the volume that you needed to get through in a day to really make it sensible. But when the AstraZeneca vaccine came on and was licensed in January 2021, we were able to use the GP practices, and we knew that that was going to allow us to start going a lot faster. So we needed to make it really work for the GP practices because GP IT is very mature in the UK.

People use the software for many years.

They know it inside out, and they don't really like to use something else. But it was really important to us that everybody used the same application because we felt that was the way to be most robust over being able to know what was going on. So we built a little bit of a sweetener or a user need.

We met the user need that the claims for the vaccination were built directly into the software.

You could set it up at the start of your session and say, "I am claiming for every vaccine that I give in this session." And then we pass the information over to the people who paid them, and it all happened seamlessly so that they got paid for the vaccines within a few weeks.

It was something that we hadn't done before in our organization, but it worked really well. And I think it's a really good example of when Mike Bracken, again from GDS, talked about how the old way of doing things used to be to start with a policy, then go out and procure something, and it comes back in again, and does it meet your need?

No, not really. So instead, we were able to identify what the user need was here, that we really needed to make this easy for people to use, and be able to claim it was going to be important.

We then technically checked out how would we make that work.

We found out that we could, and then we managed to get agreement at the time this legislation was going through very fast to say that the policy was that it would happen, and it would be recorded that way. And so it actually went into the enabling legislation to allow the primary care practice to do the delivery. So we managed to get it in the right order, starting with user need, then the technical prowess to be able to get that delivered, and then the policy as a sign-off at the end. And along with that, we were able to use our infrastructure across primary care, where we set up secure platforms to allow the GP practices even to see the people that they had not claimed a vaccine payment on. And we made it very easy to go in to claim if you had missed a payment.

You could just go back into a record and change it.

It was all the one system.

We did a roll and reconciliation, job done.

The systems used in some other parts of the UK were a little bit more complicated than that. And when I was talking to one of my colleagues about it in another part of the UK, he said, because I was mentioning that this was really quite high trust.

You could actually go in and edit a vaccine, the payment for a vaccine maybe that you had not actually delivered. But these are professionals, they're not going to do that, and if they did, we would know.

We'd have audit trails, we could have picked up something odd. But he said, "Well, if it all goes wrong, you might end up in Siberia." But luckily, I'm still here in Cardiff, and it didn't go wrong. And trusting your users and the service and making things streamlined for them really did pay off for us. So as I was saying that we built in the claims to be able to do that and to be instantly corrected, a real strength for us.

We had the dashboards.

We also had our business change team involved in with our service desk and our primary care teams, supporting the onboarding of thousands of users.

Thousands of them.

Because we also made available elements of the GP record, so within the software, there was an integration so you could see if somebody had a flu vaccination recently, if they had any allergies.

It meant that we needed to go through ensuring that people had completed their IG training.

They also were going to be able to get access to the vaccination record of everybody in Wales. So we went through all of that and really had tremendous support from our teams in actually the onboard of the users and also the support with some of their needs as they were going on.

I'm making it sound like GPs are a particularly tricky bunch, but anybody that's from the UK and has worked in the health sector would know that that is sometimes perceived as being a particular issue.

It's not really, it's just that they are very proficient in their use of IT, and they kind of know what they like and what works for them. So I'd identified that when we were doing this, bringing on board all of the GP users, we were going to need to- Suddenly we're moving from working with seven organizations, the health boards, to now 400 GP practices who are going to be involved in the delivery. And although we were going to run webinars, we had to bring them on board very quickly because, again, this was day to day.

You did not know when the vaccine was going to come live, that it could be used in that setting and go live with it. So over one weekend, myself and one of our business change team organized, we did just like a recording, a screen share of how you could use the system.

Very short, to the point.

This is what your receptionist will do.

This is what you will do as a vaccinator.

This is how you view and edit a record.

One of the things, and it did take us, what, a few weeks until we'd identified this, was that at the time when we built it originally, you were going through and you had to select yourself as a vaccinator at nearly every time, whenever you were giving the vaccine.

We should have got that fixed earlier. But I knew that it was going to be a big issue with the GP users. So I was recording this video, and I'd just done it, and we were getting ready to go out live with it, whenever Gill messaged me, and she said, "Oh, we've just fixed that now.

We've just done another release.

We've fixed it so that actually you can default and choose yourself as a vaccinator.

It'll persist for the user per user per session, so that will speed things up." So I just had to add a little note onto my video before it was picked up by practices the next morning.

This, as you can see, is actually a Facebook group.

I had started this Facebook group when I started in Digital Health and Care Wales.

By the time we were at the start of COVID, I had 100 members in it. But because I was doing so much interaction with people and letting them know about things that were happening and what was going on across different elements of all the programs that we're involved in and answering queries, although there were many other channels that people could use, we're up now to having over 650 people in the group and still more people coming in and joining all the time. So that's been a real success for us as well. But I want to tell you about one of the challenges that's there.

This is a slide that my friend Matt commissioned for a different talk that he was giving.

You can't buy a box of interoperability, as you can't buy a bottle of or a jar of DevOps either.

We do know that integrations within healthcare can be difficult and can be challenging. And I use this as an example of how that is very clearly evident.

We needed to send back the vaccine record to the GP system.

The GP system is perceived as being the lifelong record for a citizen. And although we had a permanent record within our system, which was accessible by anybody that needed to use it across all of Wales, we also wanted to have a record within the GP system as well. So with one of our suppliers, we had got as far as February, so we'd been vaccinating now for two months, and we wanted to get data back into the GP record. So we talked to them and they said, "In three weeks, you will be able to send a message through a new standard, FHIR, Fast Healthcare Interoperability Resources.

We will be able to receive that message, and it will be deliverable within three weeks." So with the other alternative, which was to kind of bung them over a CSV file every evening, which is actually what we did with the other supplier, they said, "No, don't do that.

Let's wait three weeks, and this will be ready for you." Now, given this was one of the top priorities for everyone, it was a top priority for those suppliers, it was a top priority for us, for everybody to make that work. But that service did not actually go live until August, so it was many months longer than we expected it to be. And it shows, I think, the challenges of whenever we- We had everything working well within what we'd built and what we were working with. But when we started working with others, the challenges increased, but you still have to work through it. But you just have to take that on board and give the time for that and keep the commitment to keep it going and don't give up.

Some of the other interoperability we did, which was more successful, has been sending our vaccine data over to NHS Digital, to the NHS COVID pass, and that's been really successful and has helped people be able to prove that they've been vaccinated so that they were able to keep using services when they were going out.

Not so much now, thank goodness, because we've got the vaccines and we have treatments as well, so the COVID pass has less, some utility, but it was a tremendous success for helping people feel that things were getting back to normal. So overall, what did this mean for us and for Wales?

Well, we were at a point in time in May 2020, although this was reported there in 2021. But in May 2020, we were at- No, sorry, it's May '21.

In May 2021, we were at the top in the world for having the most of our population had actually been vaccinated.

10% or more people in Wales had been vaccinated than in other parts of the UK. And Israel, which had got off to a flying start and was one of the fastest vaccinators we were, even had managed to surpass and get past them. And it was not just us that delivered that, but we supported it.

We were part of it.

We had, and we continue to talk about having a real Team Wales approach to this.

When I think of Team Wales and I think of the vaccination program, it's important to point out how really we managed to be so central to the delivery and how we managed to stay so tightly aligned with the vaccination program's needs. And that was because, I'll be honest, at the very start, there were a few meetings that took place without having a digital team there, but we made our voices heard, and we were so instrumental to the delivery that Gill and myself, for well over a year, were attending every single operational meeting of the team that were delivering the vaccine program. So there, as the digital contributors, we were there with all the health boards that were doing the delivery, with the pharmacist teams that were managing the stock, with all of the operational elements. And that was where my original tweet had came, that Gene picked up on, about how narrowing, I feel very truly, that the narrowing between the business and the IT, and that allowing us to get to fast flow was really key. And that was because of the colleagues and the teams that we work with and because I'm very sure of Gill, our product owner, who was able to inspire that confidence and speak authoritatively and confidently and take responsibility with her team, who she knew so well as we were going through this. So Gene said to me, "You should tell people how they should reach out to you if they want to contact you and be able to give some support." Now, there are lots of challenges that we have going on around my work.

One of our challenges that I'm really looking at a lot and trying to think about is how we think about demand for primary care and how we manage that and look at it, analyze it, be able to pick it up in really seamless ways. So I'm doing some thinking about that if you want to reach out to me.

The other two things I'm going to ask you for are that I'm still involved in doing NHS Hack Days and these events.

We're hoping to get going again very soon. And I would love if you would reach out to me if you're interested in sponsoring us or helping us with this because I think it's a fantastic way of allowing people to understand more about the health tech space, to meet people with real problems and work on them, and to give them the confidence to actually go and move into the sector and learn, and for clinicians and people like me to feel confident enough to go and take on more roles as well. And alongside that, I am also in our professional membership organization, the Faculty of Clinical Informatics. And within that, we started this body a few years ago, and it's to really support people like me who have a clinical registration who are taking on a digital health role and being able to contribute to that. And we are interested and keen to think about how we can work with partners that are interested in health tech space in actually supporting what we do. So if you're interested in any of those, do reach out and contact me.

You can.

That's my Twitter handle, @AMCunningham.

I've opened DMs, please let me know. But thank you so much for letting me come here and speak to you about this.

I feel that for myself, this is going to be the thing that I am going to be most proud of in my professional career.

I'm 36 years now qualified as a doctor, but I think being able to contribute to the vaccination campaign in such a way where we know that when we got faster and we vaccinated people faster, as soon as those people received a vaccination, their chance of dying from COVID was reduced by 80%. And that's something that is always going to stay with me and with our team.

Thank you.