The Challenges of Managing Web Infrastructure

You may think it's easy to set up a website, quickly add everything you want and need in, and be done with it. It's what places like Google, Amazon, or GoDaddy want you to think - it brings in business. After all, you think it's so easy so you throw your hand at it; and your wallet. But what you don't see is that it's a vicious cycle of "buy this, buy that, now buy this so it can actually work with that". It's deceptive marketing at best, founded in the very core idea that it's so easy to do all these things they promise. As someone who has been a part of this industry for years, doing the things they claim are "oh-so-easy", I can quite comfortably tell you very simply and easily it isn't. 

First off, building a website can be easy. If you're using a premium tool such as Squarespace or WordPress, it's more of a matter of using the mouse to drag-and-drop and typing in the occasional thing. But even the more "traditional" way of building a website, by creating files and typing in everything yourself, can be easy - provided you've got the right tools. I could spin up a professional-looking website in a day and not break a sweat. Sure, it may not have fancy features I love to seed in just for the wow factor and usability, but it will get the message across and most importantly work. But alas, I've been building sites for years for fun, and I have all the knowledge and tools already at my disposal that makes it child's play. And this isn't about just making a website - it's about the whole infrastructure its contained on itself.

You might be wondering at this stage what the fuss really is about? Surely a couple hundred files that have pretty looking text on a screen, move about occasionally and maybe have an image or two don't really need that much? If my Word document can open on a low-end PC, why on gods earth would a website need more?! The fuss is that there is so much more at play than that.

Servers have a certain messy-beauty to them

For a website to be a... well... website, it needs to run on something: A server. Servers are basically dumbed down computers that serve other computers. Say for example you want to check your online bank statement - the banks' server will hand you the website. But servers are for more than just websites. Even a simple site may have a database or two running in the background, busy filing away data that it needs to keep; you then might have a hidden portal that's for employees or yourself only; or you may even run a radio from it as we at Ceasefire do. The possibilities are endless. And that's one of the things that makes it painful. It isn't just about managing a website, but absolutely everything. Even the operating system running on the damn server itself needs managing and upkeeping. 

A bit of backstory to set the scene...

Years ago, I used to run and manage gameservers for a well-known game named Garry's Mod. They were fairly popular gameservers and I was proud of them, but I had most of my work cut-out for me: I was using a dedicated provider. I simply needed to manage the gameserver itself and the host would upkeep everything a step back from that - la server. I also had bought webservers from that provider, and once again, I just had to manage the website aspect of it, and they'd take care of the rest for me just like that. Now I realise that the hypocrisy is strong with this, having denounced providers like the one I used earlier on, but this is different. The provider I used never told me or claimed that setting up a gameserver or a webserver was easy, rather quite the reverse: they claim it's difficult, so they handle the hard part and you go ham with the easy part. There are literally thousands of providers like that. However, the providers I singled out do none of that. They're business-grade but also advertising to the average consumer. Even when you throw your wallet at them and then some, the learning curve is vertical. Yes, it's literally vertical. Yet they claim it's easy. The only thing harder with a learning curve more diabolically insane is that step-back that you'll end up using with them anyway. Does the step-back have a name? Indeed it does: Bare-metal servers, which I've simply been and will continue to call servers. 

The learning curve of bare-metal servers

I've already mentioned that with servers you manage absolutely everything, and that's where all the difficulty lies. That's why there are providers out there that take care of it. They are what can make developing websites be easy. That day I could spend designing a website? Simply upload the files and hey presto! Well, it isn't as easy as that. You've also got to manage a domain name. But a lot of providers do actually even give you a free domain these days, so not all is lost. It's when you want absolute, full control that things take a turn for the worst, but there's a catch: absolute power and full control are both, combined, equal to unlimited freedom and creativity. Servers give you that absolute, full control. These business-grade services the providers I single out give you have that absolute full control, and still claim they're easy to use. But you can only have one or the other, and at the end of the day, a business wants the former if it means they can get the job done even if the latter is sacrificed. And that's where deceptive marketing comes in. But I once again digress.

With great power comes great headaches...

Tangent after tangent, we're starting to get to the punchline if you will. There's another set of providers I want to touch on, and that is the bare-metal providers. What do they do? They simply give you a server and disappear like a father going out for a pint of milk. Though that last one may just be me. But back to the point: the server they give you has nothing but an operating system on it, and even then it's a special kind of operating system - what's known as a headless operating system. No, it hasn't been decapitated, rather the part where you use a mouse for anything has. It's MS-DOS for the modern age. And.... that's it. Everything else is left up to you to do as you will. Want to make a website on it? You'll have to install all the software and set it up yourself before it even dares to etch a word into the screen. It's much, much harder than it sounds. But it's a price to pay for absolute power, because you have full control.

When I was setting up this very website you are reading this blog on, there are so so many things that went into it. I knew exactly what I wanted, how I wanted it, and where I wanted it. Installing and setting up all the software, as someone that works with this kind of stuff all the time, was a breeze really. Could all be done in a week. Even when I had the extra difficulty of combining 2 servers onto this 1 domain (Editors Note: It's now 3 in 1) it wasn't all that hard. You just had to know where everything goes and how to mangle it together so that it's a recognisable lump. The really difficult part was how can it all be done so that it's maintainable? And that is the biggest challenge of all. Because on servers like these, it isn't just a website running -  you've got countless databases shifting data around, there's a gameserver running on it, even things like Docker have recently joined the foray. Then on another server, you've got even more databases and a radio service which all do have a minor dependency on the first, and we haven't even mentioned the domain itself has to be managed to keep everything ticking along.

This computer is headless. See that screen? No need for mice. 

Every so often, I'll take a service down for maintenance for around half an hour. Most of the time, it's to update it or clear some temporary files, or just do some diagnostic checks to make sure it's all working fine. And by every so often I mean at least once a week. And this is for one service. Now multiply that half-hour by every service running, and you can easily reach 5 hours. And this isn't including system packages, which do need updating from time to time - and when they require a system reboot, you can very easily spend the whole day doing nothing but ensuring the system is working fine and that nothing will break while you leave it unattended until the next week for when it's next check is due. It's kind of like a doctors appointment, that can last a day (I've had issues before that required me to stay up for a whole 24hrs during maintenance day. Not exaggeration.). AND THEN, to pile on top of that, certain things, like SSL certificates (that little green padlock in the browser), need renewing every few months. That is an indescribable pain - trust me.

We're now starting to get into the territory of "ramblings of a lunatic", which is probably quite accurate, but one must wonder if all of this is what made me a rambling loony. I've been doing this kind of work for the better part of 6 years. As flabbergasted as I am to realise just how much of my life I've dedicated (Wasted) to creating, managing, and maintaining servers, I don't regret it. It's something I'm passionate about; something that gives me great joy. There are no words in any language for the feeling you get when you see the project you where working on not just function, but be used and loved by people all over the world. It almost makes the pain worth it.... almost.

But managing it all is still a pain in the ass.

Note: We make numerous references to different companies, services and software in this article, however, they are not meant as endorsements nor are we sponsored, affiliated nor endorsed by any of the aforementioned companies or software.