Maciej Mróz Personal Blog

Because why not

Jan 1, 2015 - 7 minute read - Technology Server Side

Beyond virtualization

I guess it’s time to sum up recent trends in how we build, deploy, and operate complex server side software. Virtualization and cloud computing have been with us for quite a while but right now the era of containers is coming, with entire Docker ecosystem paving the way. You might want to ask a very valid question: “What’s in it for me?” :) I will focus on business implications of the technology. But let’s set the context first.

The explosion of cloud computing in recent years literally changed the way we think. Well, you probably could treat Amazon EC2 instances as if they were ordinary machines, but “in the cloud”. That of course would be totally missing the point. The key advantages of cloud computing are on the intersection of operations, software architecture and development - these cannot be realized without moving to true DevOps. Trying to slap “cloud computing” sticker on top of existing organization with IT/Operations/Development (add more if needed to fit your case) silos will not give anyone much value. Once you change your approach, you suddenly get the true benefits:

  • Shorter cycle time from concept to delivering customer value
  • Ability to to a lot more ad-hoc without unnecessary risk
  • Resilient and scalable application architecture - one that’s flexible and evolving, not set in stone
  • Reduced TCO thanks to automation, completely new usage patterns (i.e. instance that runs once a day for an hour) and services that cloud computing provider can operate way more efficiently than everyone else
  • If everything else fails, ability to brute force your way out of many problems :) The real cloud computing usage scenarios could look like this:
  • The developer spins up new game instance because the team anticipates that the changes deployed just minutes ago will require more processing power. It happens in less than five minutes, instance is automatically put in proper security group, attached to load balancer (if necessary), tagged, monitoring is set up so that any problems with the instance are escalated to the team that launched it, and serving traffic! By the way, instance is based on a machine image that was generated automatically by Jenkins job triggered in the morning by change to the Git repository - few hours earlier someone found and fixed race condition in the gaming platform. All with single click or call of a command line.
  • An analyst spins up an database instance with the last snapshot of production data, runs some very expensive SQL query, then shuts it down. All happening without any disruption to production service, at very low infrastructure cost, without bothering anyone. CEO gets the report he ordered a lot quicker than expected :)
  • We get huge traffic spike on some of our smaller apps - this time somebody on the team actually makes decision to move to bigger instance type and spins up 3 larger instances, waits till they start serving traffic, then stops 4 smaller ones. Because of underlying automation, the new instances are automatically spread across availability zones. All this happens without any disruption to the service.

All of these scenarios, and a lot more, could happen at my company. In fact, they happen all the time, sometimes in parallel. To us these are just events. Not issues, emergencies, or disasters, but events. Extensive planning, overprovisioning, delays, service disruptions … with the right mindset, you no longer have to worry about these and can focus on solving other problems. At least most of the time, obviously it’s not always perfect and rosy :) Bottom line: once you truly embrace cloud computing, the value is huge. If we can do all this stuff, why are people so exctited about using containers instead of just spinning up VMs like we do now? Here we come to the hard part, because if you are a software developer who used a public or private cloud the way it should be used, you just know that Docker is a huge deal. If you didn’t, you might have trouble here but I’ll try to explain anyway.

What public IaaS/PaaS services have really given us was a glimpse into the future. At the same time, they only took us half the way. With AWS etc, people have started to build resilient and scalable systems, but many wanted to take it further. Microservices are an obvious step but even cheapest and smallest VMs out there are just too powerful for that approach. If you imagine an app that’s deployed on 100+ micro instances for architectural reasons, well, that’s not pretty. We went to the cloud to reduce operational overhead, yet we found ourselves under pressure to actually increase it and to waste computing resources in the process. Containers enable reasonable isolation without overhead of using a VM. Starting a container is not really different from starting a process (in terms of both resource usage and speed!). Docker makes all that easy to use, and easy to build further automation on top of. All the missing pieces are suddenly in place. We get to:

  • Architecture described in code, possibly changing from version to version, deployed and scaled automatically
  • Heterogeneous environment that doesn’t dissolve into complete mess
  • Continuous delivery of user value with even lower risk - deployment at service level, very easy side by side deployments
  • Modularization enabling everyone to better control technology debt, and giving the option of rewriting stuff if absolutely necessary
  • Standardized workflow: developer machine, test, staging, production - all running the same code, just configured differently
  • All that without locking ourselves into using single IaaS/PaaS vendor (I don’t think of it as a big deal, but some people do)
  • With close to optimal resource usage because we can pack many containers per VM!

What does it all mean if I’m just starting? When I was finishing my studies, early into professional career, the cheapest way to set up a server was to use bare metal machine and pay $300/mo or something like that, very often on 12-months+ contract. And these servers weren’t really that powerful, iirc it was something like dual socket single core Xeon (several generations behind!) + 1 GB of memory and small SCSI disk. You want RAID? Load balancer and more machines for better availability or to scale performance? Dedicated DB machine? Costs were going up very, very quickly. Count in operational overhead of bare metal hw and you ended up with pretty high numbers (account for the fact that back then there was no Puppet/Chef/Ansible etc - we used bash and Perl to automate operations!).

Today you can get a fully featured VM for as little as $5/mo, so you can easily host replicated db + redundant web instances for $20/mo. You can scale it up when the needs arise, with provisioning within minutes. Thanks to Docker, you can use this setup to build using microservices from the very beginning, so there will be no need to go beyond these cheap VMs before you are actually resource limited. Realistically, we are talking 30-50x reduction in initial infrastructure costs. On operational side everything is also a lot easier thanks to APIs and much more mature tools. Setting up/tearing down test environments can be fully automated. If you go with AWS, for a slightly higher price you can get DB and storage services that require almost zero work on your side (for that reason AWS probably is the sweet spot when you are limited by time). What has also changed is the efficiency of our infrastructure usage - LAMP stack is pretty much dead now (except maybe for the DB part) and people moved towards things like node.js, Go, or some new JVM based stuff which bring further cost savings. Not only is the infrastructure cheaper, we can also get more from it, and all that without sacrificing application architecture. Obviously, you still have to know how to build distributed applications, but having that knowledge things are a lot easier now :)