Why monitor systemd services
Systemd restarts services silently. A service that keeps crashing and restarting burns resources and serves errors — but nobody gets paged unless you set it up. Nerve gives you a push notification on failure without deploying a full monitoring stack.
Method 1: OnFailure unit (best)
Create a helper unit that runs on failure of any service.
# /etc/systemd/system/[email protected]
[Unit]
Description=Nerve failure alert for %i
[Service]
Type=oneshot
Environment=NERVE_DSN=nerve://TOKEN:[email protected]
ExecStart=/bin/sh -c 'echo "systemd FAILED: %i on $(hostname)" | /usr/local/bin/nerve send --severity critical'
Then add OnFailure= to any service you want to monitor:
# /etc/systemd/system/my-app.service
[Unit]
Description=My Application
OnFailure=nerve-notify@%n.service
[Service]
ExecStart=/usr/local/bin/my-app
Restart=on-failure
RestartSec=5
[Install]
WantedBy=multi-user.target
sudo systemctl daemon-reload
Method 2: timer that checks for failed units
A systemd timer that runs every 5 minutes and alerts on any failed service.
# /etc/systemd/system/nerve-check-failed.service
[Unit]
Description=Check for failed systemd units
[Service]
Type=oneshot
Environment=NERVE_DSN=nerve://TOKEN:[email protected]
ExecStart=/bin/sh -c 'FAILED=$(systemctl list-units --state=failed --no-legend --plain | head -10); if [ -n "$FAILED" ]; then echo "failed units on $(hostname):\n$FAILED" | nerve send --severity critical; fi'
# /etc/systemd/system/nerve-check-failed.timer
[Unit]
Description=Check failed units every 5 min
[Timer]
OnBootSec=2min
OnUnitActiveSec=5min
[Install]
WantedBy=timers.target
sudo systemctl enable --now nerve-check-failed.timer
Method 3: journalctl watcher
Follow the journal in real-time for critical service failures.
journalctl -f -p err --no-pager -o cat \
| while read -r line; do echo "$line" | nerve send --severity alert; done
Alert on specific service restart loops
# Detect restart loops: alert if a service restarts more than 3 times in 10 minutes
SERVICE="my-app.service"
COUNT=$(journalctl -u "$SERVICE" --since "10 min ago" | grep -c "Started" || true)
if [ "$COUNT" -gt 3 ]; then
echo "$SERVICE restart loop: $COUNT restarts in 10 min" | nerve send --severity critical
fi
Quick start
go install github.com/nerve-ink/nerve-cli/cmd/nerve@latest
export NERVE_DSN="nerve://TOKEN:[email protected]"
echo "systemd FAILED: nginx.service on prod-1" | nerve send --severity critical
FAQ
How do I get a notification when a systemd service fails?
Use OnFailure=nerve-notify@%n.service in your unit file. The helper unit runs nerve send when the main service enters the failed state.
Does this work without Prometheus or Grafana?
Yes. Nerve is a standalone CLI tool. No monitoring stack required — just a systemd unit or timer that calls nerve send.
Can I monitor multiple services at once?
Yes. Use systemctl list-units --state=failed in a timer to catch all failed services, or add OnFailure= to each critical unit individually.