Your first app / Chapter 7

By the end of this chapter you should be able to

  • Read a compose file somebody else wrote and say what you would change and why
  • Separate data you could lose from data you could not, before there is any
  • Run a real application with its database and cache, on loopback only
  • Prove your photos survive a restart, and know exactly where on disk they are

This chapter installs the first application your family would notice if you turned it off.

Immich is a photo and video library: phones back up to it automatically, it does faces and search and albums, and it looks and behaves close enough to the commercial products that people will actually use it. It is also the most demanding thing in this book, because it wants a database, a cache, a machine learning service and a lot of disk.

Two things to settle before any of that. What the vendor's compose file says, and where the data is going to live.

Read the file before you run it #

Immich publishes a compose file and tells you to download it. That is already better than the projects that tell you to pipe a script into a shell. Download it, and then read it, which is the part people skip:

$ curl -fsSL -o compose.vendor.yaml \
    https://github.com/immich-app/immich/releases/latest/download/docker-compose.yml
$ curl -fsSL -o example.env \
    https://github.com/immich-app/immich/releases/latest/download/example.env

Four services. What each is for:

immich-server is the application: the web interface, the API your phone talks to, and the background work of generating thumbnails and transcoding video. It used to be split into a server and a separate microservices container, and those were merged, so if you find a guide with three application services it is out of date.

immich-machine-learning does the parts that need a model: face detection, clustering, and the search that lets you type "beach" and get pictures of a beach. Separate because it is heavy and because it is the piece with optional hardware acceleration.

database is PostgreSQL with a vector extension bolted on, which is what makes similarity search possible. Note that this is not a stock Postgres image, which is exactly why Immich runs its own rather than sharing one with anything else.

redis, which is actually Valkey, the community fork of Redis. It is a queue and a cache. Nothing in it is precious.

Why does this application get its own database?Show me why

Because that database is not a general-purpose one. It carries a specific version of a specific vector extension, and Immich's migrations expect it. A shared PostgreSQL serving four applications would make every one of them wait for whichever was slowest to support a major version upgrade, and one of them would eventually be this.

The rule: share a database server between applications with boring requirements, and give a dedicated one to anything that needs specific extensions. Immich needs specific extensions.

What it gets right #

Look at these two lines:

  redis:
    image: docker.io/valkey/valkey:9@sha256:8e8d64b405ce18f41b8e5ee20aa4687a8ed0022d1298f2ce31cdcf3a76e09411
  database:
    image: ghcr.io/immich-app/postgres:14-vectorchord0.4.3-pgvectors0.2.0@sha256:bcf63357191b76a916ae5eb93464d65c07511da41e3bf7a8416db519b40b1c23

Digests, on both. Chapter 6 argued for pinning, and this project pins the two images where a surprise hurts most. An unexpected database version fails badly.

shm_size: 128mb on the database is there because PostgreSQL uses shared memory for sorting and hashing, and Docker's default of 64 MB is small enough to cause failures under load that look like nothing in particular.

What you are going to change #

Five things, all of them from earlier chapters.

The version is a moving tag. IMMICH_VERSION=v3 in the env file follows the v3 release line, so docker compose pull can bring you a different application. For the two images where it matters they pinned digests; for the application itself they left you a tag. Pin it, and put the digest in your git history so an upgrade is a commit you made deliberately.

The port is published to everything. '2283:2283' binds every interface on the machine. You proved in chapter 6 what that means, and your firewall will not save you. Publish it on loopback.

restart: always should be unless-stopped. Chapter 6 covered the difference and this is exactly the case where it bites: you stop Immich to look at something, and it comes back on the next daemon restart.

There are no resource limits. Immich's machine learning and video transcoding will use everything you have, including the headroom your SSH session needs.

The database password is in a plain file. DB_PASSWORD=postgres in .env, sitting next to your compose file. Change it now to something random, and know that this is a problem we do not fix properly until chapter 9. Until then it is a file on a machine only you can log into.

The vendor pinned digests on the database and the cache but not on the application. Sloppy?Show answer

Not necessarily. It is a defensible split. The database and the cache are infrastructure that Immich itself depends on, and a mismatch there produces confusing, hard-to-diagnose failures, so they nail them down. The application tag is the thing users are expected to move deliberately when they want new features.

The reason to pin it anyway is that "deliberately" should mean a change you made in a file, not a change that happens because you happened to run pull on a Tuesday. Their default is reasonable for a project serving thousands of setups. Yours can be stricter.

Where the data goes #

This decision is expensive to change later and free to make now, while there is nothing to move.

Not all of an application's data is the same kind of thing. "Back up the data directory" treats it as one category. During a restore you find you preserved sixty gigabytes of regenerable thumbnails and lost the database.

There are three tiers, and everything on your server belongs to one of them.

Three tiers of data Ask of any directory: can it be regenerated? If yes it is replaceable. If no, ask whether losing it would only cost rework. If so it is expensive. If losing it means the thing is gone for good, it is irreplaceable. The tier decides where it lives and what a restore has to recover. A directory an app wants Could you regenerate it from something you still have? yes Replaceable thumbnails, transcodes, model caches no Would losing it cost you rework, rather than the thing itself? yes Expensive databases: albums, faces, accounts, shared links no Irreplaceable the original photographs, the documents, the things themselves The tier decides where it lives, what a backup must carry, and what a restore has to get right first.
Three questions to ask about any directory an application wants. The answers decide where it lives and what happens to it during a restore.

Irreplaceable. Your photographs. Nothing regenerates these.

Expensive. Losing it is a bad week, not a catastrophe. Immich's database is the clearest example: lose it and you still have every photograph, but the albums, the face names, the shared links and the user accounts are gone. The library would need rescanning and reorganising by hand.

Replaceable. Deleting it costs you time and nothing else. Thumbnails, transcoded versions, the machine learning model cache. It might be tens of gigabytes and it is all derivable.

So, for Immich:

What Tier Where it goes
Original photos, videos and profile images irreplaceable selected children of /srv/homeserver/data/photos
The database expensive /srv/homeserver/appdata/immich-db
Thumbnails and transcodes replaceable inside the upload location, sadly
Machine learning model cache replaceable a named volume, ignore it

Immich mixes original and generated files

Immich puts originals and generated media under the same UPLOAD_LOCATION. With Storage Templates disabled, uploaded originals are under upload/; enabling Storage Templates moves managed originals under library/. Profile images are under profile/. The thumbs/ and encoded-video/ children are derived data.

The exact set of children can change with an Immich release, so include the whole upload root and exclude only paths you have confirmed are replaceable. Chapter 11 excludes thumbs/ and encoded-video/; it does not assume that library/ is the only place holding originals. Recheck the Immich backup documentation when upgrading.

Make the tree, owned by you, before you start anything:

$ sudo mkdir -p /srv/homeserver/{data/photos,appdata/immich-db}
$ sudo chown -R $USER:$USER /srv/homeserver
$ sudo chmod 750 /srv/homeserver
$ ls -la /srv/homeserver

The name homeserver there is just a directory. It has nothing to do with your machine's hostname; call the parent whatever you like, as long as everything lives under one path you can point a backup at.

Why /srv and not /opt or a home directory?Show me why

Convention, mostly, and conventions are worth following where they cost nothing. The filesystem hierarchy standard describes /srv as data served by this system, which is exactly what this is. /opt is for add-on software packages. A home directory works but ties your data's location to an account, and accounts get renamed.

The practical benefit arrives later: when you add a second disk, /srv/homeserver/data is a clean mount point, and nothing else on the system cares.

Side readingStorage layout, and moving a category laterThe directory structure that lets you add a disk without rebuilding, and the migration done carefully once.

Write your own compose file #

Not a copy of theirs with edits, though the result will look similar. Yours, in the git repository you started in chapter 5, with the five changes made:

~/homeserver/immich/compose.yaml
name: immich

services:
  immich-server:
    image: ghcr.io/immich-app/immich-server:v3@sha256:<the digest you pinned>
    restart: unless-stopped
    env_file: [.env]
    environment:
      TZ: Europe/Paris
      DB_HOSTNAME: database
      REDIS_HOSTNAME: redis
      DB_USERNAME: postgres
      DB_DATABASE_NAME: immich
    ports:
      - "127.0.0.1:2283:2283"
    volumes:
      - /srv/homeserver/data/photos:/data
      - /etc/localtime:/etc/localtime:ro
    depends_on: [redis, database]
    mem_limit: 4g
    pids_limit: 1000
    cpus: 4.0
    security_opt: [no-new-privileges:true]
    healthcheck:
      disable: false

  immich-machine-learning:
    image: ghcr.io/immich-app/immich-machine-learning:v3@sha256:<digest>
    restart: unless-stopped
    volumes:
      - model-cache:/cache
    mem_limit: 4g
    pids_limit: 500
    cpus: 3.0
    security_opt: [no-new-privileges:true]

  redis:
    image: docker.io/valkey/valkey:9@sha256:8e8d64b405ce18f41b8e5ee20aa4687a8ed0022d1298f2ce31cdcf3a76e09411
    restart: unless-stopped
    mem_limit: 256m
    pids_limit: 150
    healthcheck:
      test: redis-cli ping || exit 1

  database:
    image: ghcr.io/immich-app/postgres:14-vectorchord0.4.3-pgvectors0.2.0@sha256:bcf63357191b76a916ae5eb93464d65c07511da41e3bf7a8416db519b40b1c23
    restart: unless-stopped
    environment:
      POSTGRES_PASSWORD: ${DB_PASSWORD}
      POSTGRES_USER: postgres
      POSTGRES_DB: immich
      POSTGRES_INITDB_ARGS: '--data-checksums'
    volumes:
      - /srv/homeserver/appdata/immich-db:/var/lib/postgresql/data
    shm_size: 128mb
    mem_limit: 2g
    healthcheck:
      disable: false

volumes:
  model-cache:

Get the digests by pulling the tags once and asking:

$ sudo docker pull ghcr.io/immich-app/immich-server:v3
$ sudo docker image inspect ghcr.io/immich-app/immich-server:v3 \
    --format '{{index .RepoDigests 0}}'

The limits above are a starting point for a machine with 16 GB, not a law. Immich's own guidance is that machine learning is the memory-hungry part. If a container is being killed during a large import you will see it in docker compose logs, and raising a limit deliberately is fine. Having no limit at all is what you are avoiding.

`--data-checksums` is not optional

It tells PostgreSQL to checksum every page it writes, so silent corruption on disk becomes a loud error instead of a wrong answer. Set it at creation, where it is free.

If you get it wrong, pg_checksums --enable can fix it later on a cleanly stopped cluster, so it is recoverable. But it takes downtime proportional to the size of your database, and it is not a job you want to discover you need.

Your temporary .env next to it now contains only the value that must remain secret:

~/homeserver/immich/.env
DB_PASSWORD=<something long and random>

Generate the password rather than inventing one: openssl rand -base64 24 | tr -d '/+=' gives you something you will never need to type.

That file must not reach git

.env now contains a database password. Chapter 5 asked you to add a .gitignore; add .env to it now, before your first commit in this directory, because removing a secret from git history is much harder than not committing it.

Chapter 9 replaces this arrangement with something you could publish. Until then, one file, on one machine, not in version control.

Hardware acceleration, if you have it #

In chapter 2 you found out whether /dev/dri/renderD128 exists and which driver claimed the GPU. If it does not exist, skip this section: Immich will transcode on the CPU, more slowly, and everything else works.

Immich ships the configuration as separate files you pull in with extends, which is a compose feature for inheriting a service definition from another file:

$ curl -fsSL -o hwaccel.transcoding.yml \
    https://github.com/immich-app/immich/releases/latest/download/hwaccel.transcoding.yml

Then in your immich-server service, choosing the profile that matches your hardware, which for Intel integrated graphics is quicksync:

    extends:
      file: hwaccel.transcoding.yml
      service: quicksync

Read that file before including it. It is short, and it is mounting a device from your host into a container, which is a thing you should look at rather than adopt.

A device file is not proof of anything

Passing /dev/dri into a container proves only that the container can see a device node. It does not prove the driver works, that the container's user has permission to use it, or that ffmpeg was built with support for it.

The test is to transcode something and look. Upload a video large enough to need transcoding, then watch sudo docker compose logs -f immich-server while it processes. Hardware transcoding shows up in the ffmpeg command line Immich logs, and it finishes in a fraction of the time. If you are unsure, run the same video with acceleration disabled and compare the clock. That is a measurement. The device file is not.

I cannot tell whether hardware transcoding is working

Design a test that distinguishes real acceleration from a device file that merely exists

Paste this into a new agent session. It carries everything the agent needs to know about where you are, and asks it to walk you through the problem rather than fix it for you.

I have passed /dev/dri into my container and I want to know whether transcoding is actually using the GPU rather than falling back to the CPU. I would like to work out how to tell the difference on my own machine rather than assuming it works.

First run #

Commit the non-secret files on your laptop, then repeat chapter 5's rsync deployment. On the server, create ~/homeserver/immich/.env locally; the rsync exclusion leaves it in place.

$ cd ~/homeserver/immich
$ sudo docker compose up -d
$ sudo docker compose ps
$ sudo docker compose logs -f immich-server

The first start is slow. The database creates itself, the server runs migrations, and the machine learning container downloads models on first use. Watch the logs until they go quiet rather than guessing.

Now, how do you reach it? It is published on loopback, so from the server itself. Which is why chapter 3 left one thing switched on:

$ ssh -L 2283:127.0.0.1:2283 admin@192.168.1.20

That opens a tunnel: connections to port 2283 on your laptop are carried over your existing SSH connection and delivered to port 2283 on the server's loopback. Open http://localhost:2283 in your browser and you are talking to Immich, over an encrypted connection, without having published anything to your network.

This is worth keeping in your toolkit. Any admin interface you would rather not expose can be reached this way, and it needs no configuration on the server at all.

The `AllowTcpForwarding local` line, earning its place

The hardening file in chapter 3 set forwarding to local rather than turning it off. This is why. Local forwarding is what you just used; remote forwarding, which would let a session open a door back into your network, stays refused.

Create the admin account in the browser, then a normal account for yourself if you like, and upload a handful of test photos. Not your library. Half a dozen pictures, ideally including one video so you can watch transcoding happen.

Immich will not start

Read the startup logs and work out which service is failing

Paste this into a new agent session. It carries everything the agent needs to know about where you are, and asks it to walk you through the problem rather than fix it for you.

I ran docker compose up -d for Immich and it is not coming up healthy. I am not sure which of the four services is the problem or how to read what the logs are telling me.

Prove it survives #

Find out now, with six photos, whether a restart loses your data.

$ sudo docker compose down
$ sudo docker compose up -d

down removes the containers. Every writable layer they had is gone. Wait for it to come back, reload the browser, and your photos should be there, because they were never in a container: they are in /srv/homeserver/data/photos, and the database is in /srv/homeserver/appdata/immich-db.

Go and look at them on disk, which is the part that makes it concrete:

$ ls /srv/homeserver/data/photos
$ find /srv/homeserver/data/photos/upload \
    /srv/homeserver/data/photos/library \
    /srv/homeserver/data/photos/profile -type f 2>/dev/null | head
$ du -sh /srv/homeserver/data/photos/*

You will see several children under the upload root. Originals are under upload/ by default, or under library/ after you enable Storage Templates; profile images are separate. Open one of the files you found and confirm it is an original. Immich stores ordinary files, but the directory names and layout are application-managed and can change between versions.

Then reboot the machine and confirm everything comes back on its own:

$ sudo reboot
$ ssh admin@192.168.1.20
$ cd ~/homeserver/immich
$ sudo docker compose ps

restart: unless-stopped means you should not have to do anything.

Why did `docker compose down` not lose the photos, when it deleted the containers?Show answer

Because the photos were never inside the containers. The bind mount means the container sees /data and the kernel serves it files from /srv/homeserver/data/photos on the host. Removing the container removes the process and its writable layer, and touches nothing on the host filesystem.

This is the whole reason chapter 6 spent time on volumes and bind mounts. It is also why "where does the data live" is the question to ask about any container before you run it: if the answer is "in the container", you are one docker compose down away from finding out.

Not yet, for the real library #

You have a working photo server. Wait until chapter 15 before putting your real library on it. Three concrete reasons for now. It is reachable only over SSH forwarding, so nobody else in your household can use it. Its database password is in a plain file. And there is no backup, so the machine is currently the only copy of anything you put on it.

Chapter 11 fixes the last of those, and chapter 15 is the checklist that says when you have earned it. Test photos until then.

Done when

  • You can name the four services and say what each one does
  • Your compose file is yours: digests pinned, port on loopback, unless-stopped, limits set
  • /srv/homeserver/data and /srv/homeserver/appdata exist and you can say which tier each holds
  • .env has a generated password and is in .gitignore
  • sudo docker compose ps shows four services up, and healthy where they report health
  • You reached the web interface over ssh -L, without publishing a port to your network
  • Test photos uploaded, including one video
  • docker compose down then up -d and the photos are still there
  • You found and opened an original under upload/ or library/, and checked profile/ separately
  • The machine reboots and Immich comes back without you

What you picked up

  • Read a vendor's compose file before running it. This one is well made and still needs five changes, all of them things you already knew.
  • Give an application its own database when it has opinions about extensions, and share one when it does not.
  • Data comes in three tiers: irreplaceable, expensive, replaceable. Decide which is which before any of it exists.
  • Immich mixes originals and generated files under one path, which matters when you get to backups.
  • A device file is not proof of hardware acceleration. Measure it.
  • ssh -L reaches a loopback-only service without publishing anything, and it is the reason forwarding was left on.
  • Data survives down because it was never in the container. That is the point of the bind mount.

Settings

Your values

The book is written with placeholder names so it makes sense to everybody. Put your own in and every chapter, every command and every copy-paste prompt updates to match.

Nothing here is sent anywhere. It is saved in this browser, so it comes back next time. A different browser or a private window gets the placeholders again.

Live preview

$ ssh admin@192.168.1.20
$ sudo ufw allow from 192.168.1.0/24 to any port 22 proto tcp
$ sudo hostnamectl set-hostname homeserver
$ sudo timedatectl set-timezone Europe/Paris

Real commands from chapters 2, 3 and 4. They change as you type.

The account you log in as. Not root, and not necessarily the same name you use on your laptop.

Introduced in Chapter 2, Meet your server

The book's placeholder is admin

What the machine calls itself. You choose it, and it shows up in your shell prompt and your logs.

Introduced in Chapter 3, A safe front door

The book's placeholder is homeserver

The IP address your server has on your home network, from ip -brief addr.

Introduced in Chapter 2, Meet your server

The book's placeholder is 192.168.1.20

The address range and prefix shown by ip route or ip -brief addr, written in CIDR form. Copy the real prefix; do not guess /24.

Introduced in Chapter 2, Meet your server

The book's placeholder is 192.168.1.0/24

The address traffic goes to on its way out of your house, from ip route.

Introduced in Chapter 2, Meet your server

The book's placeholder is 192.168.1.1

In Region/City form, or Etc/UTC if you would rather read logs in UTC.

Introduced in Chapter 3, A safe front door

The book's placeholder is Europe/Paris

A registered name you control. Chapter 8 uses it for the LAN route; chapter 12 uses a separate private Tailscale name remotely.

Introduced in Chapter 8, One door, many rooms

The book's placeholder is example.com

The email identity allowed to administer the tagged server in your Tailscale policy.

Introduced in Chapter 10, Your own private network

The book's placeholder is you@example.com

The mailbox that should receive actionable home-server alerts.

Introduced in Chapter 14, Knowing it is alive

The book's placeholder is alerts@example.com

Once you save, the prose and the commands read with your names, the copy buttons copy your values, and the copy-paste prompts describe your machine accurately. That last one matters: an assistant told your network is 192.168.1.0/24 when it is not will send you chasing the wrong thing.

Anything you leave empty keeps the book's placeholder.