Guest blog: Robin Cook, lead delivery manager, UC Working Age Digital at DWP Digital, describes how two key services were migrated to AWS
For the past few months we have worked tirelessly to ensure the safe migration of two of our digital services to the Amazon Web Services’ (AWS) public cloud. This change will enable greater flexibility to deploy service enhancements more quickly while maintaining the highest levels of resilience, security and cost effectiveness.
I want to say how proud I am of my team. The fruits of our labours were realised when both services went live in AWS within a week of each other. To top it off, we were able to fully decommission the old services a few days later. It was a busy month!
A tale of two services
The two migrated services are quite different and therefore presented different challenges.
Budgeting Loans is a citizen facing service that enables eligible customers to apply online for a loan to help with certain types of expenses that cannot be budgeted as part of their benefit payments. The service is used by an average of 100,000 customers per month.
The Digital Claimant Activity Messaging Service (DCAMS) is used by around 20,000 Jobcentre colleagues across the UK. It provides caseload management, appointment scheduling and text message reminders to customers, and enables referrals to be carried out under the Work Health Programme.
Debunking lift and shift
When I was asked to take on the delivery manager role for these two services, the term ‘lift and shift’ was used a lot. The phrase would conjure up expectations that the migration of these services would be quick, simple and painless…
A large part of the migration could be considered predictable to a degree. For example, we could determine a significant portion of the infrastructure required and we knew we would have to change our applications so they could be deployed using container technology (which packages an application so it can be run, with its dependencies, isolated from other processes). However, as we progressed we identified a number of challenges which required significant application changes.
The Budgeting Loans service loosely integrates with a number of back end services, which then ultimately store and process a loan application and make the associated payment. But the integration route used by the original hosting environment would not work from AWS.
To solve this, we had to re-design the whole message queuing architecture and connectivity mechanism to our back end services.
The DCAMS service was equally challenging, as it is host to a significant amount of data which we had to ensure was safely and securely handled before, during and after migration to AWS.
Encryption of data was key, but we also had to ensure that the service remained performant despite the additional overhead introduced when processing encrypted data. I’m delighted to say that we were successful in overcoming both challenges!
Changing ways of working
The unpredictable nature of these technical challenges led us to adapt our ways of working. We started by using the scrum framework, but I found that our sprint goal was often difficult to achieve when unpredictability would unfold.
For example, the DCAMS application was developed some years ago and designed for the environment in which it was hosted at that time. In order to host the service in AWS we had to introduce a significant amount of encryption while also maintaining useable levels of performance.
To do this we would analyse the original code, refactor it to handle encryption, check how the service performed, then continue to iterate until we achieved an acceptable outcome. If that was not enough, at the same time that we unpicked the code, we would identify unforeseen issues that would have to be redesigned before we could complete the original task.
An analogy would be moving a pile of rocks from one side of the garden to the other. You pick up the first rock and place it on the opposite of the garden. OK, great, so we just need to repeat this task a few more times. However, the next rock is a dirty great big boulder and beneath it is hiding a great crested newt. So now we have to rehome the newts before we can move any more rocks!
To adapt our ways of working to this unpredictable scenario we started to adopt Kanban principles and operate a continuous flow approach. This allowed us to react quickly to unforeseeable issues, increasing freedom to reprioritise our work on a continual basis.
We still maintained a focus on the outcomes we needed to achieve and prioritised the tasks according to those that would enable us to make the greatest progress.
Personal learning
It’s been a really rewarding piece of work and I’m surprised at how much I have learned. It’s been very technically focused, which has enabled me to develop a completely new set of skills as a delivery manager – I even completed the AWS Cloud Practitioner Essentials course.
I’ve also consolidated a lot of my understanding and knowledge of different agile approaches and firmly believe that a team should be empowered to find the right way to work, which isn’t always the prescribed way to work!
DWP Digital are hiring - If you’re interested in working with us, take a look at our current vacancies.
