Systemd alerts

Systemd service failure alerts to your phone.

A service crashed at 3 AM. Get an encrypted push instead of finding out from users in the morning.

Why monitor systemd services

Systemd restarts services silently. A service that keeps crashing and restarting burns resources and serves errors — but nobody gets paged unless you set it up. Nerve gives you a push notification on failure without deploying a full monitoring stack.

Method 1: OnFailure unit (best)

Create a helper unit that runs on failure of any service.

# /etc/systemd/system/[email protected]
[Unit]
Description=Nerve failure alert for %i

[Service]
Type=oneshot
Environment=NERVE_DSN=nerve://TOKEN:[email protected]
ExecStart=/bin/sh -c 'echo "systemd FAILED: %i on $(hostname)" | /usr/local/bin/nerve send --severity critical'

Then add OnFailure= to any service you want to monitor:

# /etc/systemd/system/my-app.service
[Unit]
Description=My Application
OnFailure=nerve-notify@%n.service

[Service]
ExecStart=/usr/local/bin/my-app
Restart=on-failure
RestartSec=5

[Install]
WantedBy=multi-user.target
sudo systemctl daemon-reload

Method 2: timer that checks for failed units

A systemd timer that runs every 5 minutes and alerts on any failed service.

# /etc/systemd/system/nerve-check-failed.service
[Unit]
Description=Check for failed systemd units

[Service]
Type=oneshot
Environment=NERVE_DSN=nerve://TOKEN:[email protected]
ExecStart=/bin/sh -c 'FAILED=$(systemctl list-units --state=failed --no-legend --plain | head -10); if [ -n "$FAILED" ]; then echo "failed units on $(hostname):\n$FAILED" | nerve send --severity critical; fi'
# /etc/systemd/system/nerve-check-failed.timer
[Unit]
Description=Check failed units every 5 min

[Timer]
OnBootSec=2min
OnUnitActiveSec=5min

[Install]
WantedBy=timers.target
sudo systemctl enable --now nerve-check-failed.timer

Method 3: journalctl watcher

Follow the journal in real-time for critical service failures.

journalctl -f -p err --no-pager -o cat \
  | while read -r line; do echo "$line" | nerve send --severity alert; done

Alert on specific service restart loops

# Detect restart loops: alert if a service restarts more than 3 times in 10 minutes
SERVICE="my-app.service"
COUNT=$(journalctl -u "$SERVICE" --since "10 min ago" | grep -c "Started" || true)
if [ "$COUNT" -gt 3 ]; then
  echo "$SERVICE restart loop: $COUNT restarts in 10 min" | nerve send --severity critical
fi

Quick start

go install github.com/nerve-ink/nerve-cli/cmd/nerve@latest
export NERVE_DSN="nerve://TOKEN:[email protected]"
echo "systemd FAILED: nginx.service on prod-1" | nerve send --severity critical

FAQ

How do I get a notification when a systemd service fails?

Use OnFailure=nerve-notify@%n.service in your unit file. The helper unit runs nerve send when the main service enters the failed state.

Does this work without Prometheus or Grafana?

Yes. Nerve is a standalone CLI tool. No monitoring stack required — just a systemd unit or timer that calls nerve send.

Can I monitor multiple services at once?

Yes. Use systemctl list-units --state=failed in a timer to catch all failed services, or add OnFailure= to each critical unit individually.