Skip to the content

Making a success of cloud migration and data lakes



It is possible to take an incremental approach to managing a shift to the cloud and making better use of data, writes James Norman, UK public sector CIO for Dell EMC

Migration to cloud computing is a key element of public service transformation. The cloud has gone beyond being an ‘interesting option’ for IT teams to becoming an underpinning necessity in the overall strategy for any organisation looking to develop new, more agile and cost-effective ways of managing their operations and exploiting their data.

Finding savings and improving efficiency are significant drivers today across the public sector. Cloud is attractive for the savings to be made from moving data and applications onto cloud platforms – mainly in removing the big capex spend needed for domestic infrastructure and the ability to pay only for capacity that is used. Government’s Cloud Native policy also advocates the organisation of processes to take advantage of cloud technologies.

Another major benefit is cloud’s inherent ability to empower both organisation and employee alike by enabling creation of more person-centred services, giving employees more flexibility in how they work, and collaborating and sharing information with different organisations. A rapidly growing market of applications run in the cloud – usually offered as software-as-a-service (SaaS) – opens up many possibilities for public sector leaders, and a considered approach can harness immense potential.

But migration from in-house to cloud computing can be a daunting prospect: it is still a significant investment, and there are risks around transferring data, maintaining good governance, compatibility with legacy systems, and how new applications fit into the broader picture.

Hybrid approach

In many cases the best approach is to mitigate the risk by opting for a hybrid cloud – a combination of on-premise, private and public cloud services tailored to meet the specific demands of the organisation. It can give an organisation access to a modern infrastructure without taking on excessive risk, and provide for an incremental, carefully managed approach in extending their cloud capability.

Dell EMC has focused on this approach with services in its enterprise hybrid cloud range to help organisations through the crucial passages of a cloud migration, notably in identifying possible weaknesses in advance. These include tools to assess which applications can be moved to the cloud, the impact of doing so, the databases to which they are linked, and how easily they could be broken.

Others make it possible to use platforms from different cloud providers within the hybrid model. An ‘out of the box’ portal can be configured to monitor applications across platforms, moving data from one to another to take advantage of different levels of security, access times for critical and non-critical data, and lower costs for information that needs to be archived but is not required for immediate use. It is even possible to set policies within the systems for the management of data, and to obtain an across-the-board view for better control.

Much of this is relevant to one of the major issues now facing public authorities – harnessing ever growing supplies of data to support operations and provide the insights needed to take their services to a new level.

Lake concept

Dell EMC has been working on the concept of the data lake, a repository for vast amounts of raw data in its native format that can then be exploited. Data is stored in a ‘flat architecture’, rather than in hierarchical files or folders, with each data element having a unique identifier and a set of metadata tags. When an organisation is examining an issue it can launch a query related to those tags and pull up only relevant data for analysis.

This is an increasingly important business tool, and although there are challenges in managing a data lake across a hybrid cloud, Dell EMC can again provide the tools to make it possible.

Its portal enables a data lake to be created across a number of databases, whether they are on-premise, on a private or public cloud, and create the security and governance rules that control how data is taken in, managed and used. This can extend to audit and archiving policies, and determining who can access and share the data on the platform.

Scalability factor

Importantly, doing this on the cloud makes it possible to scale up and down without big investments in storage based on unknown requirements. This applies equally to data, that can be moved into a cloud platform or on-premise repository in the short term for analysis. 

Having the lake itself in the cloud means that, even with activities that demand high levels of computing power, the cost comes only when it is used – ensuring that money is not spent on wasted capacity.

This flexible approach can underpin collaborative research and working between organisations. For example, data can be shared on a common platform, but policies can be set to acknowledge when, and who by, new intellectual property has been created. This factor is likely to grow in importance as more public sector organisations look to exploit the commercial opportunities in developing solutions by pooling resources.

All this places the hybrid cloud and data lakes firmly within the toolbox for creating a new generation of public services that are cost-effective and responsive to citizens’ needs. Cloud Native is on the way to becoming a prevailing requirement for public authorities, and they will need the support of partners with both the technical expertise and knowledge of the public sector to manage their migrations successfully.


Image by Jenny Jimenez, CC BY 2.0 through flickr

Register For Alerts

Keep informed - Get the latest news about the use of technology, digital & data for the public good in your inbox from UKAuthority.