Local Development Clusters

Why Local Clusters? A local cluster is a great way to get started with Kubernetes. You’ll be able to experiment and make all the mistakes you want. But when you're looking around for local clusters, there's tons of options! Too many! Luckily, we’re here to help.

Why Local Clusters? A local cluster is a great way to get started with Kubernetes.

You’ll be able to experiment and make all the mistakes you want. And afterwards it’s very easy to just reset the cluster and start anew. So it’s a great way for learning and trying new things.

But when you're looking around for local clusters, there's tons of options! Too many! There's Kind, there's Microk8s, there's K3s, Minikube, Docker for Desktop...? They all kind of copy each other, and there's tons of configurations for each.

So how do you choose? Are they all the same? Are there any differences? Does it really matter?!

Luckily, we’re here to help.

How do I tell them apart?

Every cluster needs to solve 3 problems:

Docker for Desktop is one big component that solves all 3 problems. The others let you mix and match.

The advantage of being a single component is that it’s easier to set up. But it comes with limitations, and when it breaks, the whole things breaks.

For the others, you can mix and match components, and optimize the different parts to your workflow—this way they can be more powerful, but it's also more complicated.

So what are the components? What you should pay attention to? What are the parts, and can you replace & optimize them?

What should I pay attention to?

Builder

This is stuff like buildkit, kaniko, img, and so on. What matters here? For example: When building, you don’t wanna build everything from scratch. So how good is the caching in the builder that you're using?

Something else to consider is that builders are getting better all the time, getting new features and functionality. In your cluster of choice, will you be stuck with the default builder, or can you try new ones as you please?

Registry

When developing locally, you shouldn't have to push your container images to DockerHub every single time. Ideally you'll be able to keep them locally, in a local registry.

But here's the thing: When using a local cluster, will resetting it (e.g. when it gets wedged) going to delete all your storage and all the cached container images you've already built?

Runtime

Trust me, you don’t want to have to debug virtual machine drivers, or decide by youself which VM is the mostest optimalest for the hardware you're using.

But what VM your cluster chooses, and how lightweight the container runner in that VM is, can have a big impact on the speed and stability of your app.

For reference, commonly used container runtimes are Docker, containerd, and CRI-O.

What are the choices?

To help us make sense of all the options we have, let’s classify them into three categories: all integrated, customizable, and “kitchen sink.”

Docker for Desktop

Everything is one component. It has an amazing virtual machine experience—you often don’t even know you’re running a VM. Even the other clusters below often use Docker to manage the VM part. On the other extreme, we have...

Minikube

Minikube can be anything you want it to be! There’s a way to run it like Docker for Desktop, where the builder, registry, and runtime are all a single component.

And there’s a way to do the absolute opposite, and customize it to your hearts content—beware though that not all the options really make sense, which can make things quite complicated.

Kind

The name stands for Kubernetes in Docker. It uses Docker to run the virtual machine, and for runtime it uses the generally more stable containerd.

You can run a separate registry that persists even after you reset the cluster. You can also use whatever builder you want. And Kind is very stable, because the Kubernetes project uses Kind to test Kubernetes itself.

K3d

The name stands for K3s in Docker. It's conceptually similar to Kind, but with K3s instead of the regular Kubernetes. K3s is a lighter Kubernetes distribution designed for lightweight, resource-constrained environments.

Microk8s

Microk8s is Kubernetes in a Snap. Snaps are a sandbox technology that works really well on Ubuntu but not necessarily as seamlessly on other distributions and operating systems. It has a registry in the cluster that resets with the cluster, unfortunately. And you can use whatever builder you want.

Recommendations

Okay, that was a lot of options, and a whole lot of information. To navigate that, here are some opinions that might help you in making a choice:

For getting started, Docker for Desktop is really easy and really convenient—it even has a GUI!

The problem is that it gets bricked from time to time, and needs to be reset. When that happens, your registry goes out the window, which means you have to rebuild everything again. That's annoying, but it's not a dealbreaker.

Once your whole team is using Kubernetes, we recommend you invest in setting up Kind. It has a solid, well-tested happy path and it makes good technical choices.

You can swap out components, but you don’t have to worry that some developer accidentally turned on an obscure VM driver—hello minikube!—and lost a whole day's work. And you can run Kind in CI too, which is a nice bonus.

Does that mean you’re never gonna use the other options?

No! There are good reasons to use all of them!

And that’s it.

To summarize:

Have fun folks! Just wait until you start having to mix-and-match local and remote resources... But that's a discussion for some other time.

For more content like this, please subscribe to our newsletter!