Develop and test container applications on macOS using Lima and Minikube

A guide to have a solid Docker-Desktop-free setup on your macOS workstation

Gone are the days playing with slow starting `Docker for Desktop` macOS application. Also, given the recent update by Docker Inc. to monetize Docker Desktop, developers are looking to migrate to other alternatives with less friction. Here is my recipe to bring back all the joy of developing applications locally with docker, docker-compose and Kubernetes without hacking existing container development workflows.

---

#1 Easy way (good for k8s development)

This setup consists of two main components:

1. Hyperkit

2. Minikube

Minikube requires a container runtime to start a local k8s cluster. It can be docker, hyperkit, kvm2 etc. The out-of-the-box solution is to install hyperkit, docker-cli, docker-compose, and minikube like this:

brew install hyperkit docker docker-compose minikube

Next, one can create a local k8s cluster using minikube, like this

minikube start --driver=hyperkit

It creates a local k8s cluster using hyperkit as runtime. This approach inherently creates a docker daemon within minikube, and can be accessed by setting docker environment with this command:

eval $(minikube docker-env)

After that command, you will be able to perform docker-client operations.

This setup is a quick win because it allows one to run their containers as part of k8s cluster and docker-compose works out of the box(except if minikube is running on multiple nodes).

The con with this approach is application developers who are used to docker and docker-compose with local image registry can find it annoying to work with minikube image registry. In addition, the services linked via default network in docker-compose files, For ex: an application and a database server, now may not work out of the box as you need to configure networking on minikube cluster. Let us improve this setup with a different approach.

---

#2 Best way (good for application as well as k8s development)

This approach avoids using hyperkit and uses a tool called Lima (Linux virtual machines (on macOS, in most cases)). This exciting project uses QEMU + NerdCtl to provide a way to spin up emulated Linux machines on macOS to run Containerd applications. We all know that on Linux, we can install Docker daemon without any licensing issues, so the process goes like this:

1. Install Lima and docker client

2. Create a Linux virtual machine with Docker daemon installed

3. For local docker application development (docker-compose), use this daemon

4. For local k8s development, spin up Minikube with Lima’s docker as runtime

Application development (core setup)

One can install Lima and docker client from brew:

brew install lima docker docker-compose

After installing Lima, one needs to create a Linux virtual machine by passing a YAML file to limactl.

limactl start docker.yaml

You can find the docker.yaml file from here: gist. It is an updated version from examples directory: https://github.com/lima-vm/lima/tree/master/examples

That command starts a vm and outputs few commands to create a new docker context called `lima` and use it as default. This finishes our docker daemon setup. Now, you can go ahead to perform docker-cli commands(even docker-compose), and they will work out of the box.

docker run hello-world

The CPU footprint at this point will be really less than hyperkit + minikube solution.

K8s development (extending core setup)

In case, if you want to spin up containers in local k8s cluster, you can have Minikube:

brew install minikube

and then start a local cluster, like this, using Lima’s docker daemon:

minikube start --driver docker --container-runtime=containerd

If you notice carefully, we are using `docker` as minikube driver, unlike `hyperkit` in the first solution. It will take some time to launch, but you should see minikube cluster provisioned successfully after some time.

Whenever you work with docker application development(building a microservice), you can simply pause your minikube cluster for performance reasons:

minikube pause

And use Lima’s docker daemon for local application development.

Once you need to work on k8s development, you can resume your k8s minikube cluster:

minikube resume

And start testing your microservice deployment on minikube.

If you want to stop Lima VM completely, you can run the following command:

limactl stop docker

---

Final words:

I hope now you learned two alternative ways to Docker Desktop on macOS. The other notable flavors include:

1. Minikube with podman as runtime

2. Rancher Desktop with docker as runtime

Finally, stay safe and have a nice time with your family.

References:

• https://github.com/lima-vm/lima

• https://itnext.io/replace-docker-desktop-with-lima-88ec6f9d6a19

• https://rancherdesktop.io/

• https://podman.io/

• https://github.com/moby/hyperkit

• https://brew.sh/