systemd-tmpfiles: Runtime and Cache Dirs

Your Service Worked Yesterday. Then You Rebooted.

It’s 2 AM. The box rebooted for a kernel update, and your service is in a restart loop. The log says Failed to bind socket: No such file or directory and points at /run/myapp/myapp.sock. You ls /run/myapp. Nothing. You mkdir it by hand, the service starts, and you go back to bed. Next reboot, same thing.

/run is a tmpfs. It starts empty on every boot, so any directory you made there by hand is gone. Two systemd features exist to rebuild it: tmpfiles.d and the directory directives in unit files (RuntimeDirectory= and friends). My take: for a directory that belongs to one service, use the unit directives. Keep tmpfiles.d for everything else, and there is a lot of “everything else”.

Below are the real line format, the defaults on this machine, and the traps. Everything here was checked against systemd 262 on Arch, using the local man pages.

Full example: Clone the working files at github.com/KingPin/sumguy-examples/linux/systemd-tmpfiles

Why /run Is Empty (and Why That’s a Feature)

/run lives in RAM, which is the point: sockets, PID files, and locks from the last boot are stale by definition. If you want the background on how that works, tmpfs and ramfs covers it. The side effect is that your daemon’s directory has to be created again on every boot, with the right owner, before the daemon starts.

You can do that in three ways. A mkdir in ExecStartPre= works, but it runs as the service user or needs a + prefix hack. A tmpfiles.d line works, and runs at boot. Or the unit asks systemd to do it. Let’s start with tmpfiles.d, because it is also what runs /tmp cleanup on your whole system.

tmpfiles.d: The Line Format

Each line in a .conf file describes one path. Fields are separated by whitespace:

#Type Path                  Mode User  Group Age  Argument
d     /var/spool/myapp      0770 myapp myapp 12h  -
f     /var/lib/myapp-README 0644 myapp myapp -    Data lives in /srv/myapp-data
L+    /var/lib/myapp-legacy -    -     -     -    /srv/myapp-data

Read left to right: type, path, mode, user, group, age, argument. Use - for any field you want to skip. Whitespace after the start of the argument field is part of the argument, so a trailing note there will end up inside your file.

Two defaults to know. If you omit the mode, directories get 0755 and everything else gets 0644. If you omit user and group, systemd-tmpfiles uses whoever ran it, which at boot is root.

Leading parent directories are created implicitly, owned by root with mode 0755. If your parent needs different permissions, add a d line for it.

The Types You Will Actually Use

The man page lists about thirty. These are the ones that earn their keep:

A few of these in action. Z is the one I reach for after restoring from backup, when ownership is wrong across a whole tree:

# Restore left everything owned by root. Fix it recursively.
Z  /srv/myapp-data   ~0750 myapp myapp -  -

# Keep one subdirectory out of the age cleaner.
x  /var/tmp/myapp-scratch/keep

# Remove a stale lock file, but only at boot (the ! modifier).
r! /var/lib/myapp-stale.lock

One gotcha on Z: a plain mode of 0640 applies to directories too, so they lose their execute bit and nobody can enter them. That is why the line above uses ~0750. The ~ prefix only removes bits: a file that has no execute bits ends up 0640, while directories keep 0750. The man page calls this out as useful with Z. Test on a copy first.

Another sharp edge: w only writes if the file already exists. People use it expecting it to create the file, and nothing happens. Use f to create, w to modify.

The Modifiers: !, -, +, =

Modifiers go right after the type letter, and I verified each against the man page.

• ! marks a line as safe only at boot. Without --boot, it is skipped. Use it for things that would break a running system, like deleting X11 lock files.
• - means a failure during create does not make systemd-tmpfiles return an error. Good for w- /proc/sys/vm/swappiness - - - - 10 in a container with a read-only /proc.
• + changes behavior per type: f+ truncates, w+ appends, L+ replaces, C+ descends into existing non-empty directories.
• = checks that existing objects have the right file type, and removes them if they do not. If a FIFO sits where a parent directory should be, it gets replaced.

You can stack them. r! appears in the vendor x11.conf, and z- and d- show up in other vendor files.

Age: How /tmp Gets Cleaned

The Age field is where tmpfiles.d earns its “tmp” name. Set it on a d line and systemd-tmpfiles --clean deletes anything older than that. The units are s, m or min, h, d, w, ms, and us. A bare number means seconds. Set it to 0 and the contents go every time --clean runs. Put - or leave it empty and nothing is cleaned.

Here is what actually runs on this machine, from /usr/lib/tmpfiles.d/tmp.conf:

q /tmp 1777 root root 10d
q /var/tmp 1777 root root 30d

Files in /tmp older than 10 days are deleted. Files in /var/tmp older than 30 days are deleted. These are systemd 262 defaults on Arch. Your distro may ship different numbers, so run systemd-tmpfiles --cat-config | grep -E '^[dDqQv]\s+/(var/)?tmp\s' before you trust mine. The q type is the btrfs-aware d, so on ext4 it behaves exactly like d.

Who triggers the clean? A timer. systemd-tmpfiles-clean.timer runs 15 minutes after boot (OnBootSec=15min) and then every day (OnUnitActiveSec=1d). If timers are new to you, systemd timers vs cron explains the OnBootSec= and OnUnitActiveSec= pair.

The age check looks at more than mtime

“Older than 10 days” means every timestamp. By default systemd-tmpfiles checks access time, creation time, change time, and modification time for files. For directories it skips ctime, since deleting a child changes it. A file you wrote three weeks ago but read yesterday survives.

You can narrow the check with an age-by prefix:

# Only look at mtime for files (m) and directories (M), ignore atime.
d /var/tmp/myapp-scratch 0750 myapp myapp mM:7d -

I hit this one while testing the example kit. A touch -d '10 days ago' file did not get cleaned, because touch cannot backdate ctime and ctime was now. If your cleanup test mysteriously does nothing, that’s why.

The cautionary tale

Say you stash a 40 GB database dump in /var/tmp “just until Friday”. It sits untouched for a month. The timer fires. The dump is gone. Nobody at systemd did anything wrong: /var/tmp has a 30 day age on this machine, and you put long-lived data in a directory with an expiry date.

Same shape for /tmp, with a 10 day fuse. A long job that writes a state file to /tmp and then sleeps for two weeks will find it deleted. Applications can protect a directory with a BSD flock: systemd-tmpfiles skips any path with a lock held, and everything below it.

Pick the right home. Data that must survive goes under /var/lib, /srv, or wherever a StateDirectory= points. And on distros where /tmp is a tmpfs, the whole directory vanishes at reboot anyway.

Precedence: Who Wins When Files Collide

tmpfiles.d reads from several directories. Files with the same name override by priority, highest first:

1. /etc/tmpfiles.d/
2. /run/tmpfiles.d/
3. /usr/lib/tmpfiles.d/

The man page also lists /usr/local/lib/tmpfiles.d/ as a source. Packages install into /usr/lib, and /etc is yours.

To change a vendor file, copy it to /etc/tmpfiles.d/ under the same name and edit your copy. To switch one off, symlink it to /dev/null:

sudo ln -s /dev/null /etc/tmpfiles.d/tmp.conf

That masks the vendor /tmp cleanup completely. Do it only if you mean it.

If two different files name the same path, the file with the lexicographically earliest name wins and the other lines are logged as errors. This is why vendor files are prefixed with numbers like 20-. If you want yours to win, name it so it sorts first.

Running It by Hand

You do not need to reboot to test. These flags are all in the 262 man page:

# Show what would change, touch nothing (-n short form needs v262)
systemd-tmpfiles --dry-run --create /etc/tmpfiles.d/myapp.conf

# Apply creation rules
sudo systemd-tmpfiles --create /etc/tmpfiles.d/myapp.conf

# Run the age cleaner
sudo systemd-tmpfiles --clean

# Remove contents of D lines and r/R targets
sudo systemd-tmpfiles --remove /etc/tmpfiles.d/myapp.conf

# Print every config file, with filename comments
systemd-tmpfiles --cat-config

# Same, but only the active lines (plus a filename header per file)
systemd-tmpfiles --tldr

--dry-run arrived in version 256, and its -n short form in 262. --inline (treat each argument as a config line) came in 261. On an older distro some of these will be missing.

To test without root and without risk, point at a scratch tree:

mkdir -p /tmp/tt/etc
echo "myapp:x:$(id -u):$(id -g)::/:/usr/bin/nologin" > /tmp/tt/etc/passwd
echo "myapp:x:$(id -g):" > /tmp/tt/etc/group
systemd-tmpfiles --dry-run --create --root=/tmp/tt ./myapp.conf

With --root, user and group names are read from /etc/passwd and /etc/group inside that root, not from your system. The two echo lines map myapp to your own UID so the whole thing runs unprivileged. The example kit’s test.sh does the same for a full create and clean pass. Remember the man page’s warning: --root is not a sandbox.

One more use of --remove: the boot sequence runs systemd-tmpfiles --remove --create, so removal happens before creation. Be careful with D lines. They get emptied on every boot.

The Other Way: Unit-File Directory Directives

Now the alternative. In the [Service] section, five settings tell systemd to create directories for you:

[Service]
User=myapp
Group=myapp
ExecStart=/usr/local/bin/myapp

RuntimeDirectory=myapp
RuntimeDirectoryMode=0750
StateDirectory=myapp
CacheDirectory=myapp
LogsDirectory=myapp
ConfigurationDirectory=myapp

The man page table says where each lands for system units:

Names must be relative and cannot contain ... List several with spaces. Each directory is created when the unit starts, and for everything except ConfigurationDirectory= it is owned by User= and Group=. Modes default to 0755 and each has a matching ...Mode= setting. Run systemd-analyze verify myapp.service on a new unit before you install it. The example unit passes clean.

The big difference is lifetime. On stop, RuntimeDirectory= is deleted. The State, Cache, Logs, and Configuration directories stay. That gives you what the man page promises: no ordering problem, because systemd creates the directory right before the process starts. With tmpfiles.d you have to trust that boot already ran it.

Keeping the runtime dir across restarts

By default /run/myapp is wiped on every stop, including a restart. If your daemon leaves a socket that clients hold open, that hurts. RuntimeDirectoryPreserve=restart keeps it across both automatic restarts (via Restart=) and systemctl restart. yes keeps it even after a plain stop. The default is no. Either way, a reboot clears /run.

Removing the directories later

The State, Cache, and Logs directories never go away on their own. To clear them, use the systemctl clean command the man page points to:

sudo systemctl clean myapp.service --what=cache

The --what= option takes values such as configuration, state, cache, logs, and runtime. Without it, cache, runtime, and the fd store go. Stop the unit first. I did not run this one because it needs root and a loaded unit.

DynamicUser= changes the paths

With DynamicUser=yes, state, cache, and logs go under /var/lib/private, /var/cache/private, and /var/log/private, host directories that unprivileged users cannot enter. Symlinks hide the difference, so the service still sees /var/lib/myapp. This stops a recycled UID from reading an old service’s data. If you set RuntimeDirectoryPreserve= to anything other than no, runtime dirs move under /run/private the same way.

That also explains a classic mystery: you ls -ld /var/lib/myapp and see a symlink, not a directory. Nothing is broken.

If you use DynamicUser= and the kernel supports id-mapped mounts, the directories are owned by nobody on the host and mapped to the service’s UID inside. The man page notes existing directories are left alone.

So Which One?

Use the unit directives when the directory belongs to one service. They tie the lifetime to the unit, they set ownership for you, and they sidestep the “did tmpfiles run before my service” question. The tmpfiles.d man page itself tells you to prefer RuntimeDirectory= for daemon runtime dirs.

Use tmpfiles.d for these cases:

• Directories shared by several services or users.
• Age-based cleanup (Age has no equivalent in the unit directives).
• Symlinks (L+), file contents (f, w), and Z permission fixups.
• /tmp and /var/tmp policy.
• Anything the unit directives refuse to do, like a path outside /run, /var/lib, /var/cache, /var/log, and /etc.

Using tmpfiles.d for a single daemon’s /run dir is like hiring a forklift to move a couch. It works, but you now own an extra file, an ordering assumption, and a cleanup rule nobody remembers. When something breaks, reading systemd status output and journalctl queries will show you which one failed. If your service is socket-activated, the same logic applies to the socket directory, which socket activation covers.

The SumGuy Take

For your own service, write RuntimeDirectory=myapp and delete the mkdir line. For /tmp and /var/tmp, know your distro’s defaults before you park anything there. This machine ships 10 days and 30 days, and the cleaner will not warn you.

Your 2 AM self will thank you for putting the directory in the unit file. Put the data that matters under /var/lib, where nothing ages it out.

Common Questions

Does systemd-tmpfiles run at boot?

Yes. systemd-tmpfiles-setup.service runs early in boot and creates files and directories from your tmpfiles.d configuration. It also executes lines marked with !, because the boot sequence passes --boot. A separate daily timer, systemd-tmpfiles-clean.timer, handles age-based cleanup after boot.

How do I stop systemd from deleting files in /tmp?

Set a longer Age or none on the /tmp line. Copy /usr/lib/tmpfiles.d/tmp.conf to /etc/tmpfiles.d/tmp.conf and edit the 10d value, or symlink that file to /dev/null to disable the cleanup. Excluding one path with an x line also works.

What is the difference between d and D in tmpfiles.d?

The d type creates a directory and ages its contents. The D type does the same, but systemd-tmpfiles --remove also empties the directory. Since boot runs --remove before --create, a D directory is emptied on every boot, while a d directory keeps its contents.

Do I need tmpfiles.d if I use RuntimeDirectory=?

No. The RuntimeDirectory= setting creates /run/<name> when the unit starts, sets the owner from User=, and removes the directory when the unit stops. You only need tmpfiles.d for age-based cleanup, symlinks, shared directories, or paths outside the standard locations.

Related Reading

• Tmpfs vs Ramfs: When Your Disk Is Too Slow and Your RAM Is Just Sitting There
• Systemd Timers vs Cron: Scheduling That Doesn’t Suck
• Systemd Socket Activation: Start Services Only When Someone Actually Knocks
• journald Structured Logging, No ELK
• Diagnosing Slow Linux Boot with systemd-analyze