You copied /var/log/journal off a dead server. grep finds nothing in it.
That is expected. Journal files are a binary format, not text. cat, grep
and less all return garbage or silence. journalctl reads them, and it will
read a file from any machine, not just the one it is running on. This guide
covers the three flags that open somebody else’s journal and the four reasons
one looks empty when it is not.
journalctl --file /path/to/system.journal
Or point it at the whole directory:
journalctl -D /path/to/journal-dir
Both accept every flag you already use. -u, -p, --since, -o json,
-g, --list-boots — all of them work against a copied file.
| What you have | Command |
|---|---|
One .journal file |
journalctl --file ./system.journal |
| Several files | journalctl --file './*.journal*' |
| A copied journal directory | journalctl -D ./journal/9f2c.../ |
| A whole root filesystem, mounted | journalctl --root /mnt/rescue |
--file takes a glob and may be repeated. journalctl interleaves the files in
timestamp order, so you get one merged stream rather than one file after
another. The short form is -i.
--root is the one for disaster recovery. Mount the failed machine’s disk,
point journalctl at the mount point, and it looks under the usual journal paths
relative to that root instead of yours.
Quote the glob. Unquoted, your shell expands it before journalctl sees it, and
only the first filename lands on --file — the rest arrive as match arguments
and journalctl rejects them.
/var/log/journal/<machine-id>/system.journal
/var/log/journal/<machine-id>/[email protected]
/var/log/journal/<machine-id>/user-1000.journal
The <machine-id> directory comes from /etc/machine-id. Point -D at that
directory itself — the one holding the .journal files. Point --root at the
mounted filesystem and let it find var/log/journal for you.
Files with an @ in the name are archived. journald finished with them and
will never write to them again. The one without is the file it was writing when
the machine stopped.
If the journal was volatile, everything lives under /run/log/journal/ instead
and a reboot already destroyed it. Nothing to copy.
Journal files are readable by the systemd-journal group. Many distributions
also grant adm and wheel through file ACLs.
Copy a journal to a laptop and those groups mean nothing there. Read it as
root, or chown the copy to yourself:
sudo chown -R "$USER" ./journal-copy
journalctl -D ./journal-copy
journalctl skips files it cannot open, and skipping every file looks the same as an empty journal.
A file renamed with a trailing ~ — system.journal~ — is one journald was
writing when it was stopped uncleanly, or one it found corrupted.
That is precisely the file you want after a crash. And *.journal does not
match it.
journalctl --file './*.journal*'
journalctl reads every entry that was fully written, so a ~ file usually
gives up most of its contents.
-b means something different hereThis one is quiet and wrong rather than empty.
journalctl -b -1 normally means “the boot before the one I am in now”. Point
journalctl at somebody else’s journal and there is no “now” to count back from.
The offset resolves against the boots in that file.
So -b -1 on a copied journal is the second-to-last boot recorded in the copy,
which is a different thing from the second-to-last boot of your laptop, and a
different thing again from what you meant if the copy only holds one boot.
List them first and use the ID:
journalctl -D ./journal-copy --list-boots
journalctl -D ./journal-copy -b 9f2c1a4e8b7d4c5fa1b2c3d4e5f60718
An ID cannot be misread. An offset can.
When there is no matching boot, current systemd says:
No journal boot entry found for the specified boot (-1).
Older versions word it differently. Either way it means the boot is not in the files you opened, not that journalctl failed.
Copying an active journal from a running machine is safe — journalctl reads whatever was fully written. But “fully written” stops at the moment of the copy, and journald buffers.
Flush first if the machine is still up:
sudo journalctl --sync
Then copy. If the machine is not still up, this is moot, and so are the last few seconds of its log. See how to find out why a Linux server rebooted for what that costs during an incident.
journalctl --file ./system.journal --verify
--verify checks internal consistency. Run it on anything recovered from a
machine that panicked, a failing disk, or an image you pulled out of a backup.
A file that fails verification may still yield most of its entries, but you
should know before you build a timeline on it.
Once it opens, it is an ordinary journal.
Everything in the last hour before the machine went quiet:
journalctl -D ./journal-copy --since "2026-09-01 08:14" --until "2026-09-01 09:14"
One unit, newest first:
journalctl -D ./journal-copy -u postgresql -r
Errors and worse:
journalctl -D ./journal-copy -p err
-p err is a level and worse, so it also shows critical, alert and emergency.
It will also miss most application errors — see
how to show only errors in journalctl
for why.
Straight into a parser:
journalctl -D ./journal-copy -o json -a > incident.ndjson
Keep the -a. Without it journalctl replaces any field over 4096 bytes with
null, which on a forensic copy means the stack traces go missing from your
evidence. Getting JSON output from journalctl
covers that in full.
journalctl -m
--merge shows entries interleaved from all available journals rather than
only this machine’s, so a foreign journal sitting in the standard paths joins
your own timeline.
For an explicit copy, name it and skip the guesswork:
journalctl --file '/var/log/journal/*/*.journal*' --file './journal-copy/*.journal*'
Useful for two or three hosts. Past that you are hand-building a log server out
of scp and directory names.
Reading a copied journal is recovery. It works when three things are true: the disk survived, the journal was persistent, and someone thought to copy the files before the instance was destroyed.
Cloud instances fail all three routinely. Terminate one and its disk goes with it. Nothing was copied, because nobody knew there would be an incident.
Ship the entries as they happen and the copy step disappears:
journalctl -o json -a --cursor-file=/var/lib/journal-ship.cursor \
| curl -fsS -X POST --data-binary @- \
"https://logs.example.com/api/v1/ingest_logs/YOUR-SOURCE-TOKEN"
The CL Agent does this on a timer and only advances its cursor after the server answers 2xx.
Then the same questions become queries, across every host at once:
SELECT json_extract(msg, '$._HOSTNAME') AS host,
datetime(MAX(CAST(json_extract(msg, '$.__REALTIME_TIMESTAMP') AS INTEGER)) / 1000000, 'unixepoch') AS last_line,
COUNT(*) AS lines
FROM logs
WHERE json_valid(msg)
AND json_extract(msg, '$._HOSTNAME') IS NOT NULL
GROUP BY host
ORDER BY last_line DESC;
A host whose last_line stopped advancing is a host that stopped. You find
that out without mounting anything.
Use __REALTIME_TIMESTAMP rather than the timestamp column. The column holds
server receipt time, so a backlog delivered after an outage sorts as if it just
happened. Keep the json_valid(msg) guard — json_extract errors on the first
row that is not JSON, and one line from a syslog relay is enough to stop the
query.
Not everything belongs on a log server.
A one-off post-mortem on a machine you already have a disk image of needs no infrastructure. Neither does reading a customer’s journal that arrived in a support ticket, or auditing a box that has no network route to anywhere.
For those, --file and --root are the whole toolchain, and they cost
nothing.
💌 Get notified on new features and updates