How the UK government can continue to innovate while reducing tech costs

As UK government departments brace for budget cuts, new programs for innovation or re-imagined services will no doubt see a decrease in funding. And who will feel the impact of this drop in tech investment the most? The people of the UK.

I have been working with government organisations for the last 20 years, evangelising the use of open source software, open principles, and open government, with the sole purpose of helping solve some gnarly government challenges. Oh, and I like to write down my thoughts from time to time…

As digital technology accelerates at pace, the tools that support this progress proliferate. Projects, teams, and partners are building their own “Frankenstein monsters” made up of completely different sets of tools for metrics, logs and traces, cyber, data search, and data retention. In its wake are troves of disparate data sprawl that are hard to decipher, share, and sometimes (actually, a lot of times) secure.

These tool silos prevent cross-team collaboration and re-use, don’t meet user needs, and produce spiralling costs. They also create blind spots for bad actors to infiltrate, or for fraud to go unnoticed. To compound the issue, these conflicting data perspectives often result in months or years passing before realising a problem exists. Remediation can then take an eternity, due to skills shortages and limited access to these silos. In the meantime, a department can suffer lost developer productivity, reputational damage, and again ultimately higher costs.

So how do we reduce technology costs while maintaining effectiveness, security, and innovation as a priority? 

Scrap the multiplex of tools and create a ‘unified data platform’

The good news is there are ways to reduce government technology costs while maintaining effectiveness, security, and innovation as priorities. But how? By leveraging a unified data platform.

Data is one of the government’s most strategic assets — and to be fair, you could argue we have too much and it's only getting bigger. By 2025, IDC estimates that the world will produce more than 180 zettabytes of data per day.

Now, government data is more complex, sensitive, and wide reaching than ever before. The collection, presentation, and security of this data are absolutely vital in delivering citizen-centric outcomes for the digital age. Data collection, data insights, and data security require individuals to work at different classifications while embracing COVID-induced ways of flexible work.

It starts with a unified data platform. One that is secure, can scale infinitely, and produces results that unite developers, engineers, data analysts, security, operations, and real users, which brings them together with the agility, adaptability, and cohesion of a small team.

When your data is rich, reliable, and easy to access, utilise, and share, you should be able to:

• Eliminate data silos to increase collaboration
• Create a culture where everyone understands the value of data
• Ensure data meets users’ needs
• Accelerate innovation through meaningful collaboration
• Become a truly data-led government organisation
• Be proactive at pace in times of national or regional crisis
• Save costs by avoiding duplication, removing technical debt, and being frictionless
• Assured, joined-up and interoperable infrastructure to support this platform

Implementing an open data platform

The data platform you implement needs to be open, not lock you in, and not fuel worries of spiralling costs. Open code base technology platforms that are built on open standards and open architecture — with a strong engineering community behind them — can provide an easy starting point for government departments, where the cost barrier to entry is zero.

The latest open platforms can allow app/dev squad teams to develop in high code for scale and performance, and data scientists and business analysts can re-imagine the data at speed using low code techniques, making it possible for even non-technical users to get the information they need.

An open data platform that is ubiquitous and can run anywhere puts the control back in the hands of government employees, whether you choose to operate in the cloud or on-premise or at the edge. Of concern is the inherent cloud lock-in when using the cloud vendors' native tools, where bill shock comes in when the data grows exponentially, and more ingestion lanes open up.

Elastic has well-established deployment patterns where developers can deploy in low classification tiers (or “Official/Dirty”) and exploit the flexibility of the cloud and then copy to the high classification side (Secret) where data is too sensitive to exist on the public cloud. These hybrid patterns built with a common schema are game-changing.

So many times, I have seen teams develop at low classification and then the solution becomes shelfware as they cannot get it assured at high. Even better, as it’s one platform, it makes the security team’s lives easier when they’re threat hunting (or as I like to call it, playing “whack-a-mole”) within the cyber realm across all these zones without impediment.

Break free from punitive licensing models

Per-agent, per-host, or per-event, per-ingestion-of-data licensing models don't scale when migrating to dynamic, cloud-native patterns and microservices. With Elastic’s resource-based pricing model, you can scale your unified data platform along with your business and have full control of the data you keep, resulting in lower — guess what — costs!

Unlike proprietary solutions, our heritage is free and open source, and we must continually provide you with value by delivering secure enterprise-grade solutions. And as our code is free and open-based and revolves around a community, there’s no risk. You can break free if you’re not happy with us, and the community code will remain available indefinitely for use and improvement.

Leverage one platform to apply search-powered solutions to any kind of data and unlimited use cases. Want to explore your application logs in SIEM? Go for it. Want to monitor your environment? Go ahead. Pay only for the increase in resource consumption.

Wrap people and expertise around your data

Say goodbye to swivel chairing. With fewer tools to learn, maintain, and stay up to date on, your team can extract huge amounts of additional value from the data and share information with each other faster — with the common goal of using insights to better serve constituents. 

In tandem with accelerating innovation, this dramatically lowers technical debt through reduced license cost, lowered cost of ownership, and multiple teams working together using the same skills.

Avoid duplication of effort by focusing on the reuse of search, observability, and security patterns for citizens and stakeholders with communities of interest groups. Co-create on common data and security challenges. Accelerate innovation through collaboration.

When you have a unified foundation, you then put focus on expanded technical growth. Enabling individuals to acquire new skills as part of their daily DNA will help them continue evolving their careers and growth within your organisation.

In summary, don’t let austerity slow you down. A unified data platform can serve as a solid foundation for limitless innovation at pace, putting data and talent at the forefront of your strategy while reducing the burden of multiple data silo tech costs.

Next steps 

Read more in the white paper "How the UK government can continue to innovate while reducing tech costs."