Jonathan Wrede

shutdown-blame: systemd-analyze blame for shutdown

A small tool that reads the journal of the previous boot and lists the units systemd stopped during shutdown, sorted by how long each stop took.

The problem

When a Linux machine boots slowly, systemd-analyze blame lists the units by start time and systemd-analyze critical-chain shows what waited for what. For shutdown there is no such command. A slow shutdown usually shows up as the message "A stop job is running for ..." with a counter that runs to 90 seconds, and the common advice is to boot with a debug shell or install a shutdown hook and then read the raw log.

There is demand for something better:

  • systemd/systemd#3371, "Provide systemd-analyze blame equivalent for system shutdown", has been open since May 2016 and has 58 reactions.
  • Three questions about "A stop job is running" (1, 2, 3) have about 400,000 views together.

How it works

systemd already writes what is needed into the journal. Every stop job gets a "Stopping ..." and a "Stopped ..." entry with a job ID, a unit name and a microsecond timestamp. When a unit runs into its stop timeout, systemd logs that too, followed by one entry per process it kills. If the journal is persistent, all of this is still on disk after the reboot.

shutdown-blame reads these structured fields with journalctl -o json for the previous boot. It finds the start of the shutdown (the logind message "System is powering down", a signal to PID 1 such as SIGRTMIN+3, or as a fallback the first stop job after the last start job), pairs every stop job by job ID and prints them sorted by duration. Units of user service managers are included, which covers the frequent case "A stop job is running for User Manager for UID 1000".

This is the output for the test container described below, stopped with docker stop:

Shutdown of boot bcc6c0a68390 began 2026-11-01 05:06:51 UTC (Received SIGRTMIN+3.).
The journal stopped 4.353s later; anything after that (final unmounts, killing leftover processes, firmware) is not recorded.

     TOOK    STARTED  UNIT                           NOTES
   4.216s    +0.014s  ignores-term.service           timed out in stop-sigterm; SIGKILL: sh, sleep
   3.009s    +0.021s  slow-stop.service
   2.220s    +0.023s  user@1500.service
   2.207s    +0.024s  user 1500: user-slow.service   timed out in stop-sigterm; SIGKILL: sh, sleep

It is a single Python file without dependencies, so it can be copied onto a machine that has just shut down slowly. --json prints the same report in a machine-readable form.

Testing

The unit tests run against a journal captured from a real container shutdown and against constructed cases: two shutdowns in one boot, a crash without shutdown, the same unit name in the system and a user manager, a failed unmount, and non-UTF-8 messages.

The end-to-end test boots systemd in a privileged container with three services: one stops at once, one has a 3-second ExecStop, and one ignores SIGTERM with TimeoutStopSec=4. A lingering user runs a user service that also ignores SIGTERM. The container is shut down twice, once with docker stop and once with systemctl poweroff, and the report is checked against the expected durations, timeouts and killed processes. CI runs this on Ubuntu 24.04, Debian 12 and Fedora 42. One difference turned up there: systemd 257 on Fedora kills timed-out units with SIGABRT, so the tool reports the signal it finds in the log.

Limitations

  • The journal must be persistent (/var/log/journal must exist).
  • journald stops near the end of the shutdown. The final unmounts, the last killing of leftover processes and firmware delays are not in the journal, so the tool cannot show them.
  • It lists durations. It does not compute a dependency chain like critical-chain.
  • It has been tested in containers. A reboot of a physical machine writes the same journal entries, but CI does not test that.

The code and installation instructions are at github.com/Jwrede/shutdown-blame.