Systemd
Cheat sheet
| Task | Command |
|---|---|
| list all failed units on the system | systemctl list-units --failed
|
| Return code denotes if a service is down | systemctl is-failed nginx.service
|
| show the status of a service/unit/timer | systemctl status ssh.service
|
| show a service in detail | systemctl show vboxweb.service
|
| kernel messages | journalctl -kjournalctl -ab
|
| follow messages | journalctl -f
|
| logs for a specific unit. Eg. SSH | journalctl -u ssh.service
|
| logs for a specific syslog facility. Eg. authpriv and auth logs: | journalctl -q SYSLOG_FACILITY=10 SYSLOG_FACILITY=4
|
| Analyze why things are slow | systemd-analyze blame
|
Switch and change targets (runlevel)
In the traditional init system such as SystemV, whether a system boots into a graphical interface or to a text-based console is determined by the default runlevel. In systemd, the idea of runlevels has been replaced by targets. A target is simply a collection of units (services) that should start. Now, to change a system's 'runlevel', we need to use tell systemd to change its target.
Instead of using init 5 to start a graphical system:
# systemctl isolate graphical.target
or init 3 for a text-based system:
# systemctl isolate multi-user.target
You can see the default target:
# systemctl get-default
graphical.target
You can change the default target:
# systemctl set-default multi-user.target
# systemctl set-default graphical.target
Journal
Systemd keeps tracks of service logs.
Clearing Journal
Clear all the entries by running:
# journalctl --vacuum-time=1seconds
Or
# journalctl --vacuum-size=1M
Limiting Log Sizes
Edit /etc/systemd/journald.conf and define a value for SystemMaxUse. Eg. SystemMaxUse=100M.
Tasks
Creating a Timer
You can use systemd to create a timer which can periodically run something, similar to Cronjobs.
A systemd timer can trigger a service. To make use of a systemd timer, you will need to create the service definition (that tells systemd how to run what you want to run) and then the timer definition itself (which tells Systemd when to run it, and how often).
For our service component, we would want a one-shot service that just executes a script. Create a service at /etc/systemd/system/gather-metrics.service:
[Unit]
Description=Logs system statistics to the systemd journal
Wants=gather-metrics.timer
[Service]
Type=oneshot
ExecStart=/bin/gather-script.sh
[Install]
WantedBy=multi-user.target
For our timer component, we want to run this every 10 minutes. Create a timer at /etc/systemd/system/gather-metrics.timer:
[Unit]
Description=Gathers metrics from this server
Requires=gather-metrics.service
[Timer]
Unit=gather-metrics.service
OnCalendar=*-*-* *:0/10:00
[Install]
WantedBy=timers.target
The OnCalendar specification takes the form of "DOW YYYY-MM-DD HH:MM:SS". An asterisk is a wildcard and matches on any value. The timer will trigger on the next date that matches this format.
Other useful specifications are:
| OnCalendar | The job will run... |
|---|---|
*-*-* *:*:00
|
Every minute |
*-*-* *:0/10:00
|
Every 10 minutes |
*-*-* *:00:00
|
Every hour |
*-*-* */6:00:00
|
Every 6 hours |
*-*-* 00:15:30
|
Every day at 12:15:30 AM |
*-*-* 9-17:00:00
|
Every day between 9 AM and 5 PM each hour, on the hour |
* *-01,04,07,10-01 00:00:00
|
Every quarter (on the first of January, April, July, October) at 00:00:00 |
* *-01-01 00:00:00
|
Every year (on the first of January) at 00:00:00 |
Weekly, or Mon *-*-* 00:00:00
|
Every Monday at 00:00:00 |
Mon...Fri *-*-* 00:00:00
|
Every weekday at 00:00:00 |
Mon *-05~03
|
On the next Monday which is 3 days from the end of May |
Mon..Fri *-08~04
|
On the next weekday which is 4 days from the end of August |
Creating a Service
Creating a service in systemd requires writing a .service file. The service file defines the service, any dependencies, and what to actually run. Service files are typically placed in /etc/systemd/system/. The most basic service file will look something like this:
[Unit]
Description=Run a script that does not fork
[Install]
WantedBy=multi-user.target
[Service]
Type=simple
ExecStart=/root/scripts/foreground-script.sh
Restart=always
We are using Type=simple because the bash script we're executing doesn't fork or daemonize itself. For programs that do daemonize or run in the background, use Type=oneshot.
Other options that may be of interest are listed below.
| options | behavior |
|---|---|
| Restart=always | restart when the program exits, regardless of the exit code. |
| User=bob
Group=bob |
Run the service with a particular user / group |
Once the file has been created, reload systemd and enable the service:
## Reloads the service files so systemd is aware of the new one
# systemctl daemon-reload
# systemctl enable mag-read
SystemV-like Init Scripts
Systemd supports the old SystemV startup scripts that are placed in /etc/rc.local and the /etc/rc.d directories using the rc-local.service.
If you need to quickly run something at startup without wanting to deal with Systemd services, just edit the /etc/rc.local file and enable the service with systemctl enable rc-local.service.
Custom service script
Alternatively, create a custom systemd service that runs your startup scripts. For example, create a file at /usr/lib/systemd/system/startup.service with the following:
[Unit]
Description=Custom SystemV-like Startup
After=network.target
ConditionFileIsExecutable=/usr/local/sbin/custom-startup-initd
[Service]
Type=oneshot
ExecStart=/usr/local/sbin/custom-startup-initd
TimeoutSec=0
StandardOutput=tty
RemainAfterExit=yes
SysVStartPriority=99
[Install]
WantedBy=multi-user.target
A few things of interest in this service file:
- The
ConditionFileIsExecutablewill only run the script if it's executable. It basically does atest -xon the file before this continues. RemainAfterExitwill cause systemd to show the service as active even after the script has finished execution.
In /usr/local/sbin/custom-startup-initd:
#!/bin/sh
StartupDir="/usr/local/boot/startup"
# Run all scripts starting with 'S'
for i in `ls $StartupDir/S*` ; do
ScriptName=`basename $i`
sh $StartupDir/$ScriptName
done
# Success exit code.
exit 0
Make the directory:
# mkdir -p /usr/local/boot/startup/
Then place your scripts in the startup directory starting with the letter 'S'.
Enable the new service by running:
# systemctl enable startup
# systemctl start startup
Verify this:
# systemctl status startup.service
Editing a service
You can create overrides to an existing service by running systemctl edit something.service.
For example, to make Docker start after ZFS is ready, add the following overrides:
# systemctl edit docker.service
After=zfs-mount.service
Requires=zfs-mount.service
Wants=zfs-mount.service
BindsTo=zfs-mount.service
This will create an override.conf file under /etc/systemd/system/docker.service.d/override.conf.
Run something in a cgroup
Systemd makes it easy to run something within a new cgroup. For example:
# Use up to 2GB memory and begin swapping if it exceeds this limit
$ systemd-run --user --pty --property MemoryHigh=2G firefox
# Use up to 2GB memory and kill with OOM if it exceeds this limit
$ systemd-run --user --pty --property MemoryMax=2G firefox