Why Database Patching Best Practice Just Doesn’t Work and How to Fix It
Patching really, really matters – patching is what keeps technology solutions from becoming like big blocks of Swiss cheese, with endless security vulnerabilities punching hole after hole into critical solutions.
But anyone who’s spent any amount of time maintaining systems will know that patching is often easier said than done.
Yes, in some instances, you can just run a command line to install that patch, and that’s it. These instances are increasingly rare though – given the complexity of the technology environment, you’re more likely faced with a complex process to achieve patching best practice.
In this article, we’ll outline why database patching matters (yes, databases are vulnerable too!), explain what the problem is with patching databases, and point to a novel solution that takes the pain out of database patching.
Watch out – your database services are vulnerable too
We know that database services are critical – databases underpin IT operations in countless ways, working away in the background. Yet databases just aren’t the most interesting parts of the technology stack, which is one of the reasons database patching can get neglected. In a recent survey by Imperva, the company found that nearly 50% of on-premise databases were vulnerable to a known exploit.
Cybercriminals, however, are not ignoring databases. Just like any other element of the technology stack, databases are full of vulnerabilities. Just one database service alone has over a thousand related vulnerabilities.
Consider a few examples. In September 2016, CVE-2016-6662 was reported, a vulnerability that allows attackers to inject malicious MySQL configuration settings into a victim’s database service. It affected MySQL clones too – including MariaDB, which was forced to publish detailed mitigating steps here.
Another example: in 2020, a database vulnerability was identified where attackers could mount a privilege escalation attack because of how certain versions of MariaDB handled “setuid” at the installation stage.
In both our examples, patching – or upgrading to a newer version of the database service – would close the vulnerability. But herein lies the problem: patching doesn’t happen as consistently as it should, and not just because tech teams are lazy – or because database services are forgotten about.
Just get on patch management, right…?
Not quite. There’s a second reason why database patching gets neglected – patching a database can be incredibly hard, with conflicting and ambiguous instructions. This problem is particularly prevalent where database implementations are quite complex.
Take MySQL clusters, for example. The open-source database MySQL has an official article outlining how users need to patch a MySQL cluster – but the instructions are intricate, and it only considers one particular setup of MySQL cluster, InnoDB, when there are other MySQL cluster strategies.
The above MySQL instructions also leave out a few important aspects of patching. It doesn’t cover how the patching process may affect other applications – or how it may affect other systems in your technology solution. It can’t offer this advice, of course, because every environment is different, and the writers don’t know what your environment looks like.
And therein lies a major issue with patching best practice, and database best practice in general: it’s almost impossible to account for the infinite practical variations – from differences in database configuration to different levels of technical knowledge.
Patching best practice just fit for purpose
The net result can be that implementing published patching best practice is a very ambiguous and uncertain exercise. Sysadmins can easily decide the risks and implications of patching going wrong is much more significant than the risk of a cyberattack on the database. So, while in theory, it’s easy to just “get on with patching”, the reality is very different.
Even where teams have the technical knowledge and the practical certainty to make a success of database patching, there is still the reality that a database service must go offline for some time to perform the patching.
Without high availability, downtime is the most disruptive side effect as tech services go offline, disrupting work.
High availability configurations can ensure that there’s no downtime, but even these are likely to experience service degradation as some servers in a cluster are offline and unable to support demand or provide adequate protection while some nodes are down for maintenance.
Complex patching procedures also consume a significant amount of time which takes resources away from other important tasks – and in some cases, the resources might simply not be there to ensure consistent patching.
Finally, taking databases offline for patching and managing complex migration processes always carries a risk of something going wrong. Data corruption could creep in during migration, or some servers may fail to come back to life after patching. These risks can’t be ignored – and are intrinsic to current database patching practices that require restarts.
Live database patching as an alternative
Until recently patching a technology service almost always required a restart, but live patching is becoming increasingly prevalent. With live patching, the patching tools perform an in-place swap of the patched code: the service being patched keeps running while patching takes place, with no restart required.
That’s exactly the role of DatabaseCare, the new solution from TuxCare. Thanks to DatabaseCare, you can perform comprehensive patching as often as you like because DatabaseCare patches your database while it is actively running and serving data.
How does this work? It’s simple in practice. Your server connects to the on-premises DatabaseCare ePortal where patches are securely stored. As soon as a new vulnerability is logged, an agent communicates with the ePortal, which then pulls the update from DatabaseCare. The agent then momentarily freezes your database service in a safe mode, and transparently applies the patch in memory. This “freeze” is so fast that it doesn’t even disrupt network connections to the database service or running queries.
The result: your databases are automatically updated with the latest security patches, without downtime, with minimal disruption and risk – and with zero ongoing effort from your technical teams.
How does DatabaseCare benefit you?
Let’s take a clearer look at the benefits of live patching – compared to patching best practice as it stands.
We’ve already pointed to the complexity of database patching, particularly for high availability, distributed databases. DatabaseCare replaces a complex set of steps involving tricky migration procedures with a single, one-time, simple step – that’s easily automated too.
It removes the ambiguity from patching your databases. Are you following the right instructions? Will it work, even if you do it perfectly? All those questions are now gone – patching happens automatically and in the background. And so yes, the risk involved in patching databases is now significantly mitigated, which reduces the hesitation to patch.
At the same time, automated patching also means that you don’t need to try and fit patching in amongst another long list of draining IT tasks. And, when patching doesn’t compete for resources, it happens more regularly. Other business units inside the organization will appreciate how you no longer require long maintenance windows for your patching operations.
We all know what regular patching means: tighter security. Reduce the window between the release date of a patch, and when that patch is applied, and you reduce the window of opportunity for attackers to exploit a vulnerability.
Best practice matters – but DatabaseCare unlocks consistent security
Patching database services isn’t new – best practice instructions have been around for some time. But there are practical difficulties to patching best practices as it stands, and these practical difficulties leave a window for cyber attackers.
DatabaseCare plugs the gap – it doesn’t disrupt your operations, it doesn’t pose a risk of failure, and you don’t need resources to make it work. In turn, your security becomes much more solid. Installing DatabaseCare is simple too. To find out more, just review the DatabaseCare page on the TuxCare website.