How to find out why a systemd service failed to start with journalctl -xeu

systemctl start myapp printed nothing. The service is dead anyway.

That is normal for most services. A Type=simple service counts as started the moment systemd forks it. If the binary is missing, or the User= does not exist, systemctl start still reports success. The failure happens a millisecond later, and nobody tells you.

So the first step is never “start it again”. It is to ask systemd what happened.

Read two lines of systemctl status

sudo systemctl status myapp

Most of the output is noise. Two lines carry the answer:

     Active: failed (Result: exit-code) since Sat 2026-09-26 09:14:02 UTC; 3min ago
   Main PID: 4121 (code=exited, status=203/EXEC)

Result: says what kind of failure it was. status= says how the process ended. Together they usually name the problem before you read a single log line.

Run it with sudo. status shows the last 10 journal lines for the unit. An ordinary user often sees no lines at all, and gets no warning that anything is hidden.

What journalctl -xeu shows

The error message tells you to run journalctl -xe. Add -u and the unit name:

sudo journalctl -xeu myapp

Each letter does one thing:

The -e limit catches people. After a reboot, journalctl -xeu myapp cannot show a failure from before it. Ask for the previous boot:

sudo journalctl -u myapp -b -1 -e

The -x text is generic. It explains what “Failed with result” means, not why your service did it. Leave it off when you paste output into a bug report or a chat. The systemd manual asks for that too.

The lines you are looking for come from systemd[1], not from your program:

systemd[1]: myapp.service: Main process exited, code=exited, status=203/EXEC
systemd[1]: myapp.service: Failed with result 'exit-code'.

Your program’s own output, if it got far enough to print any, sits just above them.

What each result means

The word in Result: narrows the search.

Exit codes 200 and up are systemd’s, not your program’s

A status from 1 to 199 came from your program. Read its own log lines.

A status from 200 to 243 is different. Programs rarely use these numbers. They mean systemd failed while setting the process up, before your program ran a single instruction. Your program has no logs to read, because it never started.

The ones you will meet:

203/EXEC is the most common by far. Check these in order:

1. The file does not exist at that path. A typo, or a deploy that has not run.
2. The file is not executable. chmod +x.
3. It is a script with no #! line. The kernel cannot run it.
4. It lives on a filesystem mounted noexec.

ExecStart= takes an absolute path or a bare command name. A bare name is searched in a fixed list of directories chosen when systemd was built, not in your shell’s PATH. /opt/myapp/bin is never on it.

start-limit-hit hides the real error

This one reads like the problem. It is not.

systemd allows 5 starts in 10 seconds by default. A service with Restart=always that crashes on startup burns through that in a moment. Then systemd stops trying:

systemd[1]: myapp.service: Start request repeated too quickly.
systemd[1]: myapp.service: Failed with result 'start-limit-hit'.

The cause is the first failure, five attempts earlier. Scroll up, or search for it:

sudo journalctl -u myapp -b -g 'Failed with result'

Fix that, then clear the counter so systemd lets you start it straight away:

sudo systemctl reset-failed myapp
sudo systemctl start myapp

timeout depends on the service type

The default start timeout is 90 seconds. Only some types can hit it.

The other types wait for something before they count as started. notify waits for the program to report READY=1. forking waits for the parent to exit. dbus waits for the bus name. If the wait runs past TimeoutStartSec=, systemd gives up and kills the process.

Type=simple never times out on start, because systemd does not wait for anything. Type=oneshot has no start timeout by default.

A notify service that times out has usually been told it is notify when it does not speak the protocol. Either fix the Type= or raise TimeoutStartSec= for a program that is slow to warm up.

“Dependency failed for” means look elsewhere

systemd[1]: Dependency failed for My App.

Nothing is wrong with your service. Something it Requires= failed first. List every failed unit on the machine:

systemctl --failed

Then start this guide again with that unit’s name.

Check the unit file before you start it

Most of the setup failures above are visible in the unit file. Let systemd read it first:

sudo systemd-analyze verify /etc/systemd/system/myapp.service

It reports unknown directives, missing dependencies, and an ExecStart= command that is missing or not executable. That last one is 203/EXEC, caught before it happens.

After editing a unit file, run sudo systemctl daemon-reload. Without it, systemd keeps using the version it loaded last.

One host at a time is the slow part

Everything above assumes you know which service failed, on which machine.

On one server that is fine. On ten, a service that died at 3 a.m. stays dead until somebody notices the thing it served is broken. systemctl --failed is accurate. It just has to be run on every host, by someone.

Get told when any service fails, on any host

Point the CL Agent at a Central Logging instance. It ships each journal entry as the JSON object journalctl -o json prints, every field included.

That matters here, because of where systemd files its failure messages. They are logged by PID 1. Their _SYSTEMD_UNIT is systemd’s own, not your service’s. The service name is in a separate field, UNIT. journalctl -u quietly searches both. A query has to do the same, or it misses exactly the lines you want:

SELECT json_extract(msg, '$.MESSAGE') AS message
FROM logs
WHERE json_valid(msg)
  AND (json_extract(msg, '$._SYSTEMD_UNIT') = 'myapp.service'
    OR json_extract(msg, '$.UNIT') = 'myapp.service');

Filter on _SYSTEMD_UNIT alone and you get the program’s output with the failure cut out.

The “Failed with result” line also carries a fixed MESSAGE_ID. It is the same on every systemd machine, and it is logged for every kind of failure. “Failed to start” is not: a Type=simple service has already “started” before its binary fails to run, so that line may never appear. Key on the ID instead. Every service failure across the fleet, last 24 hours:

SELECT datetime(CAST(json_extract(msg, '$.__REALTIME_TIMESTAMP') AS INTEGER) / 1000000, 'unixepoch') AS at,
       json_extract(msg, '$._HOSTNAME')   AS host,
       json_extract(msg, '$.UNIT')        AS unit,
       json_extract(msg, '$.UNIT_RESULT') AS result
FROM logs
WHERE json_valid(msg)
  AND json_extract(msg, '$.MESSAGE_ID') = 'd9b373ed55a64feb8242e02dbe79a49c'
  AND CAST(json_extract(msg, '$.__REALTIME_TIMESTAMP') AS INTEGER) / 1000000
      >= CAST(strftime('%s', 'now', '-1 day') AS INTEGER)
ORDER BY at DESC;

UNIT_RESULT holds the same word systemctl status shows after Result:. Keep the CASTs. journalctl’s JSON quotes every value, and SQLite compares text to a number without complaint and gets it wrong.

To be told instead of asking, create an alert rule with alert on results:

SELECT 1 FROM logs
WHERE timestamp > CAST(strftime('%s','now','-10 minutes') AS INTEGER)
  AND json_valid(msg)
  AND json_extract(msg, '$.MESSAGE_ID') = 'd9b373ed55a64feb8242e02dbe79a49c'
LIMIT 1

Rules run every five minutes. The ten-minute window stops a failure that lands on the boundary from slipping between two runs. Set Max Frequency to at least 15 minutes, or a service in a restart loop pages you on every run.

A job you run on purpose and expect to fail sometimes can be excluded by name:

  AND json_extract(msg, '$.UNIT') NOT IN ('certbot.service')

When systemctl status is enough

One server, and you are the one who restarts things? systemctl status and journalctl -xeu are the whole toolkit, and this page covers them. Stop here.

Shipping the journal starts to pay at the second host, or at the first failure nobody saw.

Where to go next

• How to view logs for one systemd service with journalctl -u — -u in depth, and the five reasons it prints nothing.
• How to find out why a Linux server rebooted — --list-boots and -b -1, for failures that happened before the reboot.
• How to show only errors in journalctl — -p err, and why a program’s errors can hide at info.
• How to search journalctl logs for a specific string — -g, and matching on fields such as UNIT instead.
• How to get JSON output from journalctl — the fields used in the queries above.
• journalctl: “No journal files were found” — when journalctl -xeu itself prints nothing.
• How to check cron logs on Linux — the scheduled-job version of “it ran, and then what?”
• Alerting — rules, channels and Max Frequency.