We've all heard the jokes from practically anyone with their development environments on Docker for Mac: it makes your Mac sound like a jet plane at takeoff.

However, their counterpart developers on Linux simply go and develop with no such problems.

Why is Docker for Mac a quantifiably worse experience than running Docker on a Linux machine? We'll explore the reasons in this post.

Containers vs. Virtual Machines

First, a word on container architecture and how it differs from your standard Virtual Machine (VM).

Generically speaking, both are similar in that you're running "computers inside your computer". The difference comes in how this occurs.

(Source: https://wiki.aquasec.com/display/containers/Docker+Architecture)

Container Disk Changes

As you can see above, Containers make use of your Host OS and its kernel, and therefore are "closer to the iron". For example, in order for a Container to read/write from your Host OS hard drive, it has to:

• Mount the disk on the Container natively (i.e. it has direct access to the disk on the Host OS thanks to the kernel)
• Work as if you were directly on the Host OS

VM Disk Changes

VM's run an additional operating system on top of your Host Operating System, as well as an additional abstraction layer (called the Hypervisor) for the "Guest OS" to talk to the Host OS. For example, in order for a VM to read/write from your Host OS hard drive, it has to:

• Mount the disk on the Guest OS
• Mount the disk from the Host OS on the Hypervisor
• Have the Hypervisor synchronize changes between both

Docker for macOS

Now, while it may be called Docker for macOS, it is architecturally different than Docker on Linux.

(Source: https://collabnix.com/how-docker-for-mac-works-under-the-hood/)

As you can see above, instead of accessing the Host OS directly, Docker for macOS instead has to spin up its own Linux VM.

It then can only access the kernel of that VM, which then has to go through the steps above to synchronize the disks of your Containers and the Host OS.

Whereas Docker for Linux essentially has a direct line to the Host OS (and, by extension, the disk, network, GPU, etc), Docker for macOS has to go through several abstractions to do low-level tasks.

Development Machine Implications

Your typical Docker development setup is usually as follows:

Host OS

• Developer tooling (IDE's, text editors, linters, etc)
• Source code editing
• Source control

Container

• Application code and dependencies
  • Some kind of hot or live reloading mechanism when code changes
• A copy or reference to the host OS source code

The jet plane taking off when you do a docker-compose up on macOS? It's your Host OS' resources hard at work to synchronize low-level I/O (specifically disk and network) between the Host OS and Containers; this is on top of having to run the Containers themselves.

This is also why you see the Hyperkit process usually consuming much of your CPU even at idle. All that synchronizing work between these layers is not trivial!

Options for macOS

This is the part where many would tell you to "just develop on Linux". While it is true that Docker on Linux is the architecture as intended (and therefore is the best experience), simply switching is not tenable for most folks.

The options below will get you closer to parity with the experience on Linux. At the very least, the jet plane taking off might only happen every once in a while versus all the time.

Docker for Mac Edge Build (with Mutagen)

As of this date, Docker has a blessed approach for minimizing resource consumption on disk changes using something under the hood called Mutagen. You won't have to worry about the details though, as they package it as part of the Docker for Mac Edge build.

Instructions

1. Install Docker for Mac Edge Build
2. In the Docker UI, go to Resources => File Sharing and specify what folders you want mounted to Docker containers

Pros

• "Blessed" by the Docker team
• Minimal setup -- use docker-compose and docker files as normal
• Drastically reduces CPU on hot/live reloads
• File changes are considerably faster

Cons

• Still working out the kinks
• Requires Docker Edge Build

docker-sync

An alternative that's been on the scene for several years now is called docker-sync.

docker-sync is essentially a container running in parallel with your own containers whose job is to efficiently let your container know when files change. It is, in effect, another abstraction layer to speed up the process.

Instructions

1. Install docker-sync
2. Modify your docker-sync.yml according to your dev setup

Pros

• Works on across Docker platforms
• Drastically reduces CPU on hot/live reloads
• File changes are considerably faster

Cons

• Configuration modifications needed
• More docker resources consumed due to additional parallel containers
• Additional orchestration needed to spin your containers up and down
• Occasionally has syncing issues; i.e. the container doesn't get updated with Host OS changes, and it needs restarting

In Summary

Docker was mainly built with Linux in mind. As it proved out its utility, it was eventually ported to macOS and Windows.

Since both operating systems are vastly different than Linux under the hood, virtualization was the only feasible way to get things working. This unfortunately results in these low-level inefficiencies that we otherwise take for granted.

With Mutagen being packaged as part of Docker for Mac in the future, there is hope for macOS developers that the "jet engine" problem starts to diminish.

However, as it currently stands, the best developer experience for Docker still remains its native Linux.