Docker and Kubernetes

Up to now, we’ve talked about microservices at a higher level, covering organizational agility, designing with dependency thinking, domain-driven design, and promise theory. Then we took a deep dive into the weeds with three popular Java frameworks for developing microservices: Spring Boot, Dropwizard, and WildFly Swarm. We can leverage powerful out-of-the-box capabilities easily by exposing and consuming REST endpoints, utilizing environment configuration options, packaging as all-in-one executable JAR files, and exposing metrics. These concepts all revolve around a single instance of a microservice. But what happens when you need to manage dependencies, get consistent startup or shutdown, do health checks, and load balance your microservices at scale? In this chapter, we’re going to discuss those high-level concepts to understand more about the challenges of deploying microservices, regardless of language, at scale.

When we start to break out applications and services into microservices, we end up with more moving pieces by definition: we have more services, more binaries, more configuration, more interaction points, etc. We’ve traditionally dealt with deploying Java applications by building binary artifacts (JARs, WARs, and EARs), staging them somewhere (shared disks, JIRAs, and artifact repositories), opening a ticket, and hoping the operations team deploys them into an application server as we intended, with the correct permissions and environment variables and configurations. We also deploy our application servers in clusters with redundant hardware, load balancers, and shared disks and try to keep things from failing as much as possible. We may have built some automation around the infrastructure that supports this with great tools like Chef or Ansible, but somehow deploying applications still tends to be fraught with mistakes, configuration drift, and unexpected behaviors.

With this model, we do a lot of hoping, which tends to break down quickly in current environments, nevermind at scale. Is the application server configured in Dev/QA/Prod like it is on our machine? If it’s not, have we completely captured the changes that need to be made and expressed to the operations folks? Do any of our changes impact other applications also running in the same application server(s)? Are the runtime components like the operating system, JVM, and associated dependencies exactly the same as on our development machine? The JVM on which you run your application is very much a highly coupled implementation detail of our application in terms of how we configure, tune, and run, so these variations across environments can wreak havoc. When you start to deliver microservices, do you run them in separate processes on traditional servers? Is process isolation enough? What happens if one JVM goes berserk and takes over 100% of the CPU? Or the network IO? Or a shared disk? What if all of the services running on that host crash? Are your applications designed to accommodate that? As we split our applications into smaller pieces, these issues become magnified.