Containers, honestly

Nearly everything you will run from here on arrives as a container, so this chapter is about the runtime rather than about any application. We will install Docker, run something disposable, take it apart, and then break the firewall on purpose.

Save an hour for the firewall part.

What a container is #

A container is a process on your machine, running with a restricted view of it.

A process started from a packaged filesystem, with the kernel lying to it about what exists. Not a virtual machine. It sees a filesystem that is not yours, a process table containing almost nothing, and a network interface that is not your network interface. Underneath, it is scheduled by the same kernel as everything else, and ps on the host will show it to you like any other program.

Two kernel features do the work. Namespaces control what a process can see: its own view of the filesystem, process ids, network, hostname and users. Cgroups control what it can use: how much memory, how much CPU, how many processes.

A virtual machine, by contrast, runs its own kernel on emulated hardware. That is a far stronger boundary and a far heavier one.

It matters practically. Containers start instantly and cost almost nothing, which is why running twelve of them on a mini PC is reasonable. And a kernel vulnerability is a shared problem, which is why you patch the host, why you do not run untrusted images, and why "it is only in a container" is not a security argument on its own.

Installing Docker, and who you are trusting #

The instruction you will find everywhere is to pipe a script from the internet into a shell. Do not.

The script is probably not malicious. The habit is: you agree to run whatever that URL returns today, as root, without looking. Do it the boring way instead, which also gets you upgrades through the normal package system.

These commands support Ubuntu and Debian by reading the distribution ID. Stop if the result is anything else; use that distribution's official Docker instructions instead.

$ os_id=$(. /etc/os-release && printf '%s' "$ID")
$ os_codename=$(. /etc/os-release && printf '%s' "$VERSION_CODENAME")
$ case "$os_id" in ubuntu|debian) ;; *) printf 'unsupported here: %s\n' "$os_id"; false;; esac
$ sudo install -m 0755 -d /etc/apt/keyrings
$ curl -fsSL "https://download.docker.com/linux/${os_id}/gpg" \
    | sudo tee /etc/apt/keyrings/docker.asc > /dev/null
$ sudo chmod a+r /etc/apt/keyrings/docker.asc

Now the step almost nobody does. Check the key is the one Docker says it should be:

$ gpg --show-keys --with-fingerprint /etc/apt/keyrings/docker.asc

It should be:

9DC8 5822 9FC7 DD38 854A  E2D8 8D81 803C 0EBF CD88

If it does not match, stop.

Note what that check is and is not worth. You fetched the key from Docker's own domain over HTTPS, and comparing it against a value printed on the same website would prove very little. The value is worth having because it goes in your playbook: from then on, a key that changes stops your automation instead of being trusted silently. Cross-check it against a second source you trust if you want more than that.

Then the repository and the packages:

$ echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] \
    https://download.docker.com/linux/${os_id} ${os_codename} stable" \
    | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
$ sudo apt update
$ sudo apt install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin
$ docker --version

Record the version your package repository installed.

Configure the daemon before you run anything, because one of these settings prevents a problem that is annoying to fix later:

{
  "log-driver": "json-file",
  "log-opts": { "max-size": "20m", "max-file": "3" },
  "live-restore": true,
  "no-new-privileges": true
}

Log rotation is the important one. By default a container's log file grows until the disk is full, and a chatty application can do that in a week. Twenty megabytes, three files, per container.

live-restore can keep compatible containers running while the Docker daemon is temporarily unavailable; it does not guarantee every daemon upgrade or networking change is interruption-free. no-new-privileges prevents new containers from gaining privileges through mechanisms such as setuid binaries unless a service explicitly overrides it.

$ sudo systemctl restart docker

Take one apart #

Run something disposable. This image answers every HTTP request with a description of the request, which makes it perfect for poking at:

$ sudo docker run -d --name demo -p 127.0.0.1:8080:80 traefik/whoami:latest
$ curl -s localhost:8080 | head -3

Now the vocabulary, using the thing in front of you.

Image and container. The image is the packaged filesystem plus metadata: read-only, shareable, the same bytes for everyone. The container is one running instance of it, with a thin writable layer on top. Delete the container and that writable layer goes with it. Ten containers from one image share the image and have ten separate writable layers.

Tags lie. traefik/whoami:latest is a label pointing at an image, and the publisher can move it whenever they like. Two machines pulling :latest a month apart get different software. A digest is a hash of the image content and cannot be moved:

$ sudo docker image inspect traefik/whoami:latest --format '{{index .RepoDigests 0}}'

Record the digest your registry returned; do not copy a digest from the book or from another architecture.

Pin digests for anything you care about. It means upgrades are a deliberate act: you change the digest, in a commit, having read what changed. It also means a compromised publisher account cannot quietly replace what you are running.

Ports. -p 127.0.0.1:8080:80 reads right to left: port 80 inside the container is published to port 8080 on the host, on the loopback address only. The address part matters. It is the second half of this chapter.

Networks. Containers get their own network namespace and attach to Docker bridges. Containers on the same user-defined bridge reach each other by name. Separate bridges block direct name/address reachability unless another route or published host port connects them; ordinary bridge networks usually still provide internet egress. Later chapters give databases no published port and attach them only to the application bridge.

Limits. By default a container can use every core and all the memory on the machine, and one runaway process takes the whole server down with it, including your SSH session. Say what it may have:

$ sudo docker rm -f demo
$ sudo docker run -d --name demo --memory 256m --pids-limit 150 --cpus 0.5 \
    -p 127.0.0.1:8080:80 traefik/whoami:latest

Why you are not in the docker group #

Every tutorial tells you to add yourself to the docker group so you can drop the sudo. Understand what that does before you decide.

The Docker daemon runs as root and listens on a Unix socket. Anything that can talk to that socket can ask it to start a container, and containers can be asked for things like "mount the host's root filesystem inside me, and run as root".

So membership of the docker group is equivalent to being root, with no password prompt and no sudo log entry. It takes one docker run with the right arguments, not an exploit.

Two reasonable positions. Type sudo docker, accept the friction, and keep root access as a deliberate act. Or add yourself to the group, knowing you have decided your login account is effectively root and that anything which compromises your shell has the machine.

This book types sudo. Later services do not receive the socket or a proxy to it; Traefik uses reviewed route files and Alloy reads only the host log directory it needs.

Where the data goes #

A container's writable layer dies with the container. So anything that must survive lives outside it, and there are two ways to arrange that.

A bind mount maps a directory from the host into the container. You choose the path, you can see the files with ls, and you back them up like any other directory.

$ sudo docker rm -f demo
$ sudo docker run -d --name demo -v /srv/demo/data:/data traefik/whoami:latest

A named volume lets Docker manage the storage. Better isolated, awkward to inspect, lives under /var/lib/docker/volumes.

For a home server, prefer bind mounts for anything you care about. When you want to back up your photographs, or move them to a bigger disk, or check how much space they take, being able to point at a directory is worth a great deal.

A file instead of a command #

Those docker run lines are already unwieldy and we have barely started. The same problem as chapter 5, with the same answer: describe it instead of typing it.

services:
  demo:
    image: traefik/whoami@sha256:<digest-you-recorded>
    restart: unless-stopped
    ports:
      - "127.0.0.1:8080:80"
    mem_limit: 256m
    pids_limit: 150
    cpus: 0.5
    security_opt:
      - no-new-privileges:true

$ sudo docker rm -f demo
$ sudo docker compose up -d
$ sudo docker compose ps
$ sudo docker compose logs -f demo
$ sudo docker compose down

restart: unless-stopped means the container comes back after a reboot or a crash, but stays down if you deliberately stopped it. That is what you want on a home server: it survives a power cut and respects a deliberate stop.

Now break the firewall #

Change the port line to publish on every address, the way most documentation and nearly every copy-pasted compose file does:

    ports:
      - "8080:80"

$ sudo docker compose up -d

Confirm your firewall still says what it said in chapter 4:

$ sudo ufw status verbose

Require an incoming deny policy, the LAN SSH rule from chapter 4, and no rule permitting port 8080. Account for any other rules you deliberately retained.

Now go to another machine on your network and ask it what it thinks:

$ curl -s -o /dev/null -w "HTTP %{http_code}\n" http://192.168.1.20:8080/

Require an HTTP success status from the other LAN machine. That demonstrates the published container port bypassing the host's UFW INPUT policy. Internet reachability remains a separate router and IPv6 question.

To prove the host firewall still governs traffic delivered to INPUT, start a temporary host listener in a second server terminal:

$ python3 -m http.server 8081 --bind 0.0.0.0

From the other machine, this request should time out under the deny policy:

$ curl --max-time 5 http://192.168.1.20:8081/

Stop the temporary listener with Ctrl-C. Port 22 would be a bad control because chapter 4 explicitly allowed it. The host listener on 8081 is blocked, while Docker's published port on 8080 is reached. The firewall works; it was never asked about the forwarded container traffic.

Why #

Because published ports are not delivered to your machine. They are routed through it.

Docker adds a rule to the nat table that rewrites the destination of packets arriving for port 8080, changing it to the container's own address. That happens early, before the filtering step. By the time the packet reaches the filtering stage, it is no longer addressed to your server, so it goes down the FORWARD path rather than INPUT.

Every ufw rule you have written lives in INPUT. The packet never goes there. Docker's own documentation puts it plainly: packets are diverted "before it reaches the INPUT and OUTPUT chains that ufw uses".

Docker then hooks its own chains into FORWARD, including one called DOCKER-USER that exists specifically so you have somewhere to put rules that run before Docker's own accepts.

Walk a packet through it. The three modes are the three ways this ends:

Fixing it #

Two answers, and the order matters.

Publish to loopback. 127.0.0.1:8080:80. The redirect rule then only matches packets addressed to 127.0.0.1, so a packet arriving from the network never gets rewritten, stays addressed to your machine, goes to INPUT, and meets your deny policy like everything else.

This is the right default, and it is what this book uses. Almost nothing should be published to the world directly. The chapter after next puts a single reverse proxy in front of everything, and the applications behind it do not need host ports at all.

Verify it:

$ ss -tlnp | grep 8080

Require the local address to be 127.0.0.1:8080, not 0.0.0.0:8080 or [::]:8080. Then try again from the other machine and require it to fail.

Filter in DOCKER-USER when a port genuinely must be published widely and you still want rules about who reaches it:

$ sudo docker info | grep -i 'firewall backend' || true
$ sudo iptables -I DOCKER-USER 1 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
$ sudo iptables -I DOCKER-USER 2 -i <lan-interface> -p tcp \
    -m conntrack --ctorigdstport 8080 ! -s 192.168.1.0/24 -j DROP

The established-connection rule preserves replies to container-initiated traffic. The second rule uses connection tracking because destination translation has already changed port 8080 by this stage; it scopes the drop to that original published port, interface and source policy. This example is IPv4 only. Add and test the matching IPv6 policy whenever you publish on IPv6. Docker's experimental nftables backend has no DOCKER-USER chain, so inspect the active backend and use an equivalent nftables base chain when applicable. Persistent policy belongs in the playbook, after you have tested it without losing access.

Clean up after yourself #

Images and stopped containers accumulate, and on a machine that will also hold your photographs, disk is not free.

$ sudo docker system df
$ sudo docker compose down
$ sudo docker image prune