Cloud Computing Archives - Digital Scotland
Agile Cloud Migration – Modernizing for Digital Transformation
Category: Best Practices,Digital Transformation Author: Neil McEvoy Date: 2 weeks ago Comments: 0

With the Scottish Government about to embark on a Cloud First policy to migrate their estate of legacy data centres and applications to the Cloud, a critical skill set needed for the local market is ‘Agile Cloud Migration’ – Extending the scope of Agile development teams to include the migration of legacy applications.

Problem Statement

Building new Cloud services from scratch is a relatively simple proposition and is why startups can take to Cloud adoption quickly and easily.

For large enterprise organizations like government their primary challenge is their existing legacy systems, they operate huge data centres that over time have accumulated multiple eras of hardware platforms, operating systems, databases and applications, stretching as far back as the original mainframes.

There is also a long tail of thousands of Microsoft Access databases and Excel spreadsheets littered across departmental PCs, and broader still, the use of paper-based forms that must be completed by hand to execute some processes, or at best downloaded and submitted via email.

In many cases the original skills required to maintain these systems are long gone and the systems have hardened to become black boxes, the organization dare not try and adapt them lest they break and no one knows how to fix them, but they still work and so are kept in place.

For the Government sector elderly systems like COBOL are still prevalent, indeed in the USA they account for 70% of IT spend, and cost the government nearly $40 billion a year to maintain. This is why they aren’t investing as much in new innovation-enabling technologies like Cloud as they might.

In Oct 15 the UK Authority web site reported that the National Audit Office said the public sector is still struggling to master and realize the potential of digital transformation, despite the citizen and cost benefits it’s known to deliver.

They also identified legacy applications as the root cause of this lack of progress in all of these areas, reporting that over £480 billion of government revenues were reliant on them highlighting the many risks this presents, most notably resistance to the new digital innovations governments are required to adopt to achieve new online services:

“The government’s ICT strategy, published in March 2011, recognized legacy ICT as a barrier to the rapid introduction of new policies and particularly the move to ‘digital by default’.

Legacy ICT reduces the flexibility to improve public services, makes it harder to protect against evolving cyber threats and increases government’s reliance on long-term contracts with large ICT companies. It is also likely to increase the cost of operating public services by preventing higher levels of automation and hinder data sharing intended to prevent fraud and error.”

In their audit they review a sample of government department situations and their legacy application challenges – the DWP Pension Service, HMRC VAT Collection, NHS Prescription Payment Service and the OFTs Consumer Credit Licencing Service.

These scenarios feature a variety of aged technologies, some originating as far back as 1973 running on a mainframe computer. The HMRC identified in 2009 that their 600 systems were “complex, ageing and costly”, and the report highlights how expensive a burden this is: The VAT collection service costs £430 million per annum to operate, and the DWP’s Pension Payment service £385 million per annum. That’s almost a billion pounds a year just for two applications.

Platform Modernization

Simply ‘lifting and shifting‘ these apps to the Cloud, ie virtualizing and deploying them to IaaS, won’t address the bulk of this challenge. Yes it will tackle the issue of aged hardware but the code remains as is.

The critical consequence of this is that the code is therefore still as difficult to modify as before, no transformational benefits have been achieved, it is still an inhibitor to digital transformation.

Therefore the first aspect and defining goal of Agile Cloud Migration is to migrate these systems into new DevOps environments so that this becomes possible, and new innovations can be developed and deployed at the fast pace these techniques make possible.

In short it must also be combined with application modernization best practices, as begins to touch on, achieving a full transformation of the application stack.

Legacy modernization best practices can address these issues, delivering business benefits including:

  • Untangle and map legacy application complexities – Build a basis of understanding of existing application and data architectures to establish more intelligent IT planning concepts in line with business and technical demands. Developers with no experience of the legacy software can be enabled to implement changes in line with business needs.
  • Extend the life of legacy applications without the risks of greenfield COTS projects – Numerous reports highlight how a COTS (Commercial Off The Shelf) approach to modernization is very high risk with expensive failure rates.
  • Align user interfaces and back-end application and data models with modern business processes – Modernization can be used to achieve IT objectives such as SOA, Cloud migration and Web-enablement of applications.
  • Leverage new technologies and tools – The overarching benefit is the transformation of software that is now resistant to change and thus innovation, as the required skills have long since retired and/or the suppliers are no longer in business. By moving it to a modern software platform new tools and techniques like ‘DevOps’ can be implemented to speed the rates of innovation.

A couple of case studies illustrate the basic principles for AWS and Azure respectively:

Netflix – Netflix is the poster child for ‘Cloud Native’ development but at one time they too operated a traditional enterprise IT approach. To achieve the massively disruptive digital services platform they now operate they underwent a holistic migration and transformation, to the AWS Cloud.

In this blog they focus on the migration of the core Netflix billing systems from their own data centre to AWS, and from Oracle to a Cassandra / MySQL combination, emphasizing in particular the scale and complexity of this database migration part of the Cloud Migration journey.

They also reference a previous blog also describing this overall AWS journey, again quickly making the most incisive point – this time describing the primary inflection point in CIO decision making that this shift represents, a move to ‘Web Scale IT‘:

That is when we realized that we had to move away from vertically scaled single points of failure, like relational databases in our datacenter, towards highly reliable, horizontally scalable, distributed systems in the cloud.

Microsoft MS Sales – Microsoft’s core Revenue Reporting system had reached the limits of both the underlying infrastructure and also the application functionality essential to agile competitiveness.

It was identified that Lift and Shift only would not address the latter challenge, and so additionally the apps were modernized for Azure PaaS to leverage a Microservices and Big Data architecture.

Monolith to Microservices

The Microsoft case study highlights the essential dimension to this transformative Cloud migration approach – Modernizing the core architecture of the enterprise software, from a monolith to a microservices model.

A microservices software architecture is the pinnacle of Cloud Native computing, and is relatively simple to understand when considering greenfield projects, but for most enterprise organizations it quickly brings them back around to the topic of legacy modernization, requiring a much more complex challenge of how to adapt their existing systems to this new approach. InfoQ offers a great series of articles on the topic. That poor old monolith, you can migrate it, transform it, decompose it, break it, smash it, or just skip it.

This presentation from Linkedin offers a detailed case study, describing their approach for exactly this scenario – From a Monolith to Microservices + REST:

This describes:

  • A legacy estate of Java, Servlets, JSP and Oracle databases.
  • A need to support fast release iterations as far back as 2010, which ran into the core challenges associated with monolith software: Test failures, rollback difficulties and complex orchestration and dependencies between services.
  • So they broke apart the codebase, adopted Continuous Delivery practices and devolved controls, implementing a decentralized code base.
  • The use of Java RPC meant a proliferation of APIs made backwards compatibility a big problem, a situation they addressed by moving to , a REST + JSON framework, key components from the Netflix suite – Apache Zookeeper for dynamic service discovery, and DECO for URN resolution to explore data graphs.

This combination formed their particular ‘Microservices Recipe’, and when you consider the role social graphs play across the Linkedin environment, how our business contacts are inter-connected and we dynamically explore our way through them, you can see how it would be an ideal design for this type of web site.

Others offer very practical permutations. For example in this article Flickr describe how you can utilize Github to operate a ‘Microservices Store’.

“Some of the products that we work with at Yahoo have a very granular architecture with hundreds of micro-services working together. For scenarios like this, it’s convenient to store configurations for all services in a single repository. It greatly reduces the overhead of maintaining multiple repositories. We support this use case by having multiple top-level directories, each holding configurations for one service only.”


This is a great idea when you consider Github can provide the foundation for a complete DevOps toolchain, augmented in many ways such as adding apps to support Agile practices.

Similarly Sensedia propose a recipe for Legacy Modernization that defines how microservices can be utilized as an API enablement strategy.

Chandra Rajasekharaiah, Enterprise Solutions Architect at Macy’s, published this excellent deep dive analysis of the Monolith to Microservices transformation and the software engineering challenges it presents, and Anil Madan, VP of Engineering at Intuit also describes the same journey encompassing a broader perspective of platforms and organizations.

Agile Cloud Migration

AWS offers a wealth of insights developed from their experience of having now migrated hundreds of enterprise customers to their Cloud.

For example this presentation describes an ‘Agile Approach to Mass Migrations‘, providing a comprehensive primer on a wholesale transformation framework that is based on and can integrate with existing Agile practices, achieved through building a Cloud Centre of Excellence and referencing thought leaders such as Gene Kim’s Phoenix Project and Jez Humble’s Lean Enterprise principles.

Best practice resources

They also offer an extensive supporting library of further resources:

Rating : 0 / 0 votes
  • Currently 0/5 Stars.
  • 1
  • 2
  • 3
  • 4
  • 5

Scottish Government at Scale – Harnessing the Cloud
Category: Best Practices Author: Neil McEvoy Date: 2 weeks ago Comments: 0

The Cloud offers a platform for accelerating digital innovations in Scotland.

The Herald described how it presents Scottish businesses with the innovation computing power of NASA, and the ScotGov Digital Economy plan defines a number of specific action points for accelerating adoption:

Mandate the use of common platforms and infrastructure, including cloud hosting, as appropriate across the Scottish Government.

Make better use of cloud-based solutions as a source of both cost reduction and service innovation, and move public sector data hosting to a cloud environment wherever this is appropriate in terms of security and efficiency.

As early as 2013 it was recognised that government procurement of Cloud was an essential dynamic for stimulating the Scottish tech sector, and Holyrood Magazine ran a roundtable asking what would it take to grow adoption. The Scottish Government has published this Cloud Computing Guidance.

Government at Scale – Moving out of the basement

Niall Creech, Head of Cloud Engineering at the Ministry of Justice, provides this excellent case study of the type of benefits Scottish agencies can seek from doing so, documenting their move to the AWS Cloud as shifting to Government at Scale.

Other agencies like the Home Office are also seeking to replicate the move.

Agile Infrastructure

Niall makes the point that in today’s IT world there is little value to be had in operating traditional data centres any more, and moving to the Cloud represents “moving out of the basement”.

It’s not simply a process of outsourcing, transfering the same technology paradign from in-house to an external supplier, but of harnessing an entirely new paradigm all together.

The case study offers a very articulate definition of how Cloud provides an ‘Agile Infrastructure’. Instead of just migrating the same virtual servers to IaaS, Niall describes how they have embraced Cloud Native building blocks, such as containerized applications, serverless functions and elastic storage, to make possible more dynamic and agile provisioning and management of IT infrastructure. For example all live services have the ability to have any of their servers destroyed without notice, with no alerts and no user impact.

By creating apps through composing together AWS services as building blocks, and automating deployment through nested stack templates the team are able to abstract themselves away from low level administrative work, enabling a focus on value generating digital services, not IT operations:

We were also quick to understand the value of freeing people from maintenance, data security, and disaster recovery that key managed services like Amazon RDS gives.

Even with high degrees of automation, maintaining a growing infrastructure places a burden on an organisation that can hold it back from achieving its core aim, developing and providing people with the digital services they need from modern government.


Cost Savings

Other case studies include Derby City Council, who have similarly moved out of the basement to drive cost reduction of on-premise software licencing, among other benefits including:

  • achieved significant reductions in total ICT spend
  • improved server monitoring and optimisation
  • the ability to respond more flexibly to changing business needs
  • transferred repetitive tasks, such as server patching, to a supplier
  • reduced the time, effort and cost to procure and manage new services
  • accessed suppliers with niche skills that traditional outsourcing would exclude

Similarly the DWP saved £20m a year by migrating their ‘Tell Us Once’ digital service away from a single, non-extendable managed contract. This case study also highlights the agile nature of the G-Cloud procurement process too; under very tight timescales they were able to utilize the fast-moving process to secure the deal in only 8 weeks.

Nottinghamshire Fire and Rescue Service faced a challenge of a complex legacy telephony equipment causing excessive costs and becoming entirely obsolete with no support. Again leveraging the G-Cloud procurement marketplace they sourced a modern Skype for Business based solution, one that provided all of the same core features they needed plus a host of new ones, such as multimedia collaboration, intelligent call routing and integration with fire station PA systems to announce emergency messages.

Rating : 0 / 0 votes
  • Currently 0/5 Stars.
  • 1
  • 2
  • 3
  • 4
  • 5

Cloud RPA as a Foundation for Transformational Digital Government
Category: Best Practices,Digital Transformation Author: Neil McEvoy Date: 2 weeks ago Comments: 0

The theory is there is no silver bullet in IT, but actually there are common requirements that manifest repeatedly across all types of organizations and that can be addressed that way.

CRM isn’t this approach, it’s a specific functionality for a specific business scenario. In contrast ‘RPA‘ – Robotic Process Automation, is such a general purpose solution, as the term suggests it automates any and all kinds of processes.

RPA – Robotic Process Automation

FinTech Futures explores the potential for automation financial compliance procedures for example.

The huge potential it offers can be fully realized when considered within a context of outsourcing, from Cloud through traditional, large scale BPM outsources.

Furthermore this context and these deals can be further scrutinized through a microscope of Digital Transformation – How did they enable the organization to bring new digital services and capabilities to market? For the public sector today this is Goal #1.

However these deals are notorious for expensive and poor delivery success stories (e.g. Edinburgh City) and one of the primary causes of this is a failure to achieve a more advanced, better optimized IT platform. In short they simply transfer the entire estate often including all the people, to the outsourcer, with the theory being the outsourcer will then realized multiple customer economies of scale and deliver better IT for all of them at a shared cost rate.

This logic ultimately leads you to Cloud computing, it’s the full conclusion to that principle on a global scale, but it also highlights the root challenge: Legacy IT. For example simply “lifting and shifting” your app as is to an IaaS environment won’t deliver transformational benefit either. If you don’t also modernize the app itself and the tools and methods used to enhance it (DevOps et al), then the business benefits of Cloud Migration will be minimal and all technical in nature.

The same principle applies to outsourcing. If you ‘lift and shift’ your IBM mainframe to be run entirely by IBM, then yes it’s in a safe pair of hands and off your plate, but the root problem itself has simply been moved from one place to another. Indeed what customers often find is that once the services team are behind change control procedures, technology dynamism is entirely lost, their IT has become a black box that works fine but can’t easily be altered or extended, the fundamental activity of Digital Transformation.

Hence why many senior executives often find themselves surprised, that this partnering with a tech leader doesn’t result in cool new tech advances for their business, but rather cements them in place with the legacy platform they already had, and with the added complexity of outsourcing procedures has become more cumbersome not less, and critically, fails to realize any efficiency and cost savings.

Cloud RPA – The Cloud as Smart Middleware

Actually outsourcing is somewhat of a red herring. As this UKA news highlights the same issues cause the same effect even when IT is run in-house, with this locked in nature also being the reason for not adopting new, innovative technologies. Organizations are hard coded to their data centres, whether they run them in-house or outsource them, and this resistance is the main reason why Cloud adoption on a large wholescale has yet to happen.

However the outsourcing dynamic is key, because how you adopt the Cloud is the critical factor. Thinking about Cloud only in Lift and Shift terms is the wrong perspective, because you’re only considering a scope of technology that you already have and a migration that won’t expand its’ capability in any meaningful business way.

In contrast sourcing a new technology capability is a very powerful advance, and is especially so when it is used in such a way as to augment and enhance the value of the existing IT. Think of adopting the Cloud as applying an “overlay” across your existing IT, a new surrounding layer not just a place to host what you already have.

RPA is a perfect example – An advanced technology that is best adopted As A Service and that can automate the integration between multiple existing legacy systems. This type of system integration to enable streamlined BPM is again one of those scenarios that is a major pain point for every and all IT implementations, hence the point about it being a generalized solution.

Digital Government Transformation

Vendors like Thoughtonomy offer an entirely new paradigm for solving this thorn in the foot issue – A ‘Virtual Worker’ approach.

Some organizations solve the challenge through ‘swivel chair integration’ – They simply hire staff to re-key data from one system to another, offering easy insights into how legacy IT costs balloon. Virtual Workers are simply the automation of that manual labour, what technology is ideal for and how it delivers cost efficiency benefits.

It also highlights the realities of Digital Government projects today. The ‘GDS Cult’ has somewhat positioned Agile software development as the hammer for every nail, with a consequence being they are excellent at digitizing a few big processes, but the reality of every government agency is that they are a maze of these swivel chair integrations and manual procedures.

You might be able to complete one part of a government procedure via a super whizzy web front end app, but then later to complete it in full you have to print out and post back a hard copy form. Or you have to send an email to someone who is currently on vacation, or phone a call centre who knows nothing about this first digital service, and so on.

It’s difficult to automate these types of lots of little interactions via software development, but it’s what RPA is ideal for. No need to reinvent the wheel each time, just automate it appropriately using pre-developed and proven modules for that task.

Building this capability into outsourcing/Cloud deals is therefore what designs in the ability to deliver what senior executives want from them – Cost savings resulting from efficiencies that are passed on to the customer, and improve digital capabilities. Identifying areas where they incur excessive manual bureaucracy and automating those areas, as a standard activity of the outsource, will realize both.


The main tools of government procurement today: In-house, outsourcing or G-Cloud, bring some but not all pieces of the jigsaw for what’s needed to truly achieve Digital Transformation in the public sector, given the scale and complexity of this manual procedure landscape.

Software dev can digitize key processes, Cloud migration and outsourcing provides the core IT infrastructure capacities, but neither addresses this long tail of manual process, meaning by definition it’s not end to end and thus not fully digitally transformed.

RPA is the category of technology that maps entirely to this category of unaddressed scope, hence it’s silver bullet nature. It’s an example of how adopting Cloud today can deliver transformational impact and in a way that fixes rather than is stymied by the issue of legacy app complexity.

Rating : 0 / 0 votes
  • Currently 0/5 Stars.
  • 1
  • 2
  • 3
  • 4
  • 5

Serverless Startups – Inside @Skyscanner Engineering HQ, with Gareth Williams, CEO/Co-Founder
Category: Best Practices Author: Digital Scotland Date: 4 weeks ago Comments: 0

Serverless Computing

Serverless Cloud Computing is a great example of both the dynamism of Cloud services and also how this infrastructure evolution is changing the very nature of software engineering itself.

Serverless computing is a cloud computing execution model in which the cloud provider dynamically manages the allocation of machine resources. Pricing is based on the actual amount of resources consumed by an application, rather than on pre-purchased units of capacity.[1] It is a form of utility computing.

For example AWS offers this beginners guide for setting up your first serverless application.


It is an ideal technology and service model for startups: Literally, pay only when your service is active, billed only for what you use. Not only that but a development paradigm that emphasizes rapid delivery of new, modular digital services.

Skyscanner is one key poster child for the Scottish tech sector, marking the first ‘unicorn’ through becoming the nation’s first billion dollar startup, harnessing the power of the Cloud to do so. Listen to CEO / Co-founder of Skyscanner Gareth Williams explain how

fundamentally they are a software company living in the age of continuous deployment.

As they describe in their tech blog they’ve sought to emulate the global digital pioneers of these trends notably Netflix, synonomous in the tech sector with cutting edge ‘Cloud Native’ approaches that best exploit the power of massive providers like Amazon, referring to a software architecture that utilizes ‘microservices’, breaking large unwieldy single ‘monoliths’ of code into small, interlinked modules.

As Phil Dalbeck Principal Engineer for Skyscanner describes in this Slideshare presentation they too have leveraged AWS to make this dramatic growth possible and scalable, also implementing best practices to ensure their information security. Also highlighting another key trend, ‘multi-cloud’, making use of multiple Cloud providers to fulfil all your business needs, they’ve also adopted for their sales team to use.

The industry has since made another evolutionary leap forward, one that further lowers the barrier for new digital entrepreneurs, what is called ‘Serverless’ computing, implemented through services including AWS Lamda.

As the name suggests, and as Professor Bill Buchanan of Napier University describes in his Linkedin blog, this represents the ultimate pinnacle of the Cloud computing trend, a complete abstraction away from the underlying mechanics of IT, freeing developers to focus entirely on the value add software they’re creating. They never have to meddle with any aspects of the servers or other infrastructure components, the parts that don’t add new value.

Critically for entrepreneurs is the point that it’s implemented entirely On Demand. Other AWS services offer a utility pricing model yes, linking pricing to usage but it’s not entirely fluid, requiring reservation of resources for example. In contrast Lamda is entirely ‘event-driven’, it activates and operates only for the time the software is called and executed, and customers are billed only for that exact usage.

This Silicon Angle article captures the essence of the opportunity – Entrepreneur Ryan Kroonenburg has built the world’s first ‘Serverless Startup’, a business model based entirely on leveraging this capability.

On Demand is the ideal model for startups, you incur costs only at the time you generate the associated sale, every transaction is profitable, but to date it’s been a utopian ideal not a realistic possibility. Serverless now brings that utopia to the real world, and for Scottish startups who don’t have access to the same investment capital as their USA counterparts but compensate for this through a wealth of ingenuity and creativity, it offers a hyper-accelerating rocket fuel for our innovation economy.

Rating : 0 / 0 votes
  • Currently 0/5 Stars.
  • 1
  • 2
  • 3
  • 4
  • 5

Forgot your password?

Enter your account data and we will send you a link to reset your password.

Your password reset link appears to be invalid or expired.


Processing files…