Skip to content

Knowledgebase

Investigating a failed systemd service

servers

A failed systemd unit records that a service did not complete its expected lifecycle. Read the failure evidence before repeatedly restarting it. The useful cause may be a configuration error, unavailable dependency, permissions, exhausted resources or an application exception.

Gather the details you will need

Identify the exact unit name and failure time. Confirm you are authorized to manage it and understand whether restarting interrupts users or duplicates work. Preserve relevant logs and note the last deployment or configuration change.

01 Investigate a failed systemd service

  1. List failed units with systemctl --failed, then inspect systemctl status UNIT --no-pager -l using the actual name. Record the exit code and first causal error.
  2. Read the matching journal window with journalctl -u UNIT --since TIME. Check the application's private log when the journal only reports a wrapper failure.
  3. Inspect the effective unit definition with systemctl cat UNIT, including overrides. Compare its executable, working directory, user, environment-file paths and required dependencies with the installed application.
  4. Run the application's supported configuration check where available. Verify disk space, file ownership and required sockets without exposing secret environment values.
  5. Correct the identified cause, reload systemd definitions only if the unit changed, and start or restart the service once. If a start-limit blocked it, reset that state only after fixing the fault.

02 Example: identify the first failure of a systemd service

Replace example-app.service with the failing Linux unit. Run the status and journal checks from an authorised session before repeatedly attempting a restart. These commands inspect the current state and a bounded set of recent service messages.

Command example
systemctl status example-app.service --no-pager -l
sudo journalctl -u example-app.service --since '-30 minutes' --no-pager -n 80
systemctl show example-app.service -p ExecMainStatus -p Result -p FragmentPath

Read the result: Start with the earliest relevant application error rather than the final message that the unit failed. The exit status, result and unit-file path help distinguish a bad command, missing environment value, dependency problem or process failure. A restart-limit message may only be a consequence of repeated earlier crashes.

Preserve secrets in environment files and redact sensitive log values. Correct the identified cause, then use a reviewed restart and verify the actual customer-facing operation. A green service state alone does not test the application.

Does clearing failed status fix the service?

No. Resetting the recorded state does not repair its underlying problem. Confirm a successful start and an application-level health check after correcting the cause.

Confirm the expected result

Confirm the unit reaches its expected active state and the application responds correctly. Some oneshot services finish successfully and become inactive, so interpret status according to service type. Observe logs long enough to catch an immediate crash loop.

If the problem continues

If the error returns, stop repeated retries and compare the new first error with the original. A daemon-reload rereads unit definitions; it does not validate application configuration or restart the process automatically. Restore the previous tested application/configuration when a recent release caused the failure.

Was this answer helpful?

Need a little more help?

Our support team can help you find the next step.

Contact support

Powered by WHMCompleteSolution