How to view kernel logs with journalctl -k (and why dmesg shows less)

journalctl -k --since "3 days ago" does not show you three days of kernel messages.

If the server rebooted yesterday, it shows you one day. -k limits the output to the current boot, and --since does not lift that limit. Nothing on screen tells you. This guide covers reading kernel messages with journalctl, getting past the boot limit, and when to reach for dmesg instead.

The short answer

sudo journalctl -k

-k is short for --dmesg. It shows only the kernel’s messages, from the current boot, oldest first.

Behind the scenes, -k adds two filters. One is the match _TRANSPORT=kernel. The other is -b, the current boot. The man page says so in one line: “This implies --boot=0 unless explicitly specified otherwise.”

Kernel messages from before the last reboot

-k with –since stays inside this boot

journalctl -k --since "3 days ago"

This ANDs three things: kernel messages, this boot, and the last three days. If this boot started yesterday, you get yesterday onwards. The two days before the reboot are missing. They are often the two days you wanted.

There are two ways around it.

Name the boot yourself. -b all removes the boot filter:

journalctl -k -b all --since "3 days ago"

Order matters on most servers. Up to systemd 257, -k turns the boot filter back on wherever it appears. So journalctl -b all -k still shows only this boot. Put -b all after -k. systemd 258 fixed this, but Debian 13, Ubuntu 24.04 and RHEL 9 all ship older versions. journalctl --version prints yours on its first line.

Skip -k and use the match. This works on every version, in any order:

journalctl _TRANSPORT=kernel --since "3 days ago"

It is the same filter -k adds, without the boot limit attached. Use this one in scripts.

One earlier boot

journalctl --list-boots
journalctl -k -b -1

--list-boots prints one line per boot, with its offset and its first and last entry. -b -1 is the boot before this one, -b -2 the one before that. Here -b comes after -k, so it wins on every version.

No earlier boots at all?

Then the journal is not kept on disk. journald writes to /run/log/journal in memory, and a reboot wipes it.

Storage=auto is the usual setting. It keeps the journal on disk only if /var/log/journal exists. Create it:

sudo mkdir -p /var/log/journal
sudo systemctl restart systemd-journald

That keeps kernel messages from the next boot onwards. Whatever happened before the last reboot is already gone. Finding out why a Linux server rebooted covers what you can still learn without it.

journalctl -k or dmesg?

dmesg reads the kernel’s ring buffer directly. That buffer has a fixed size and lives in memory. So dmesg has three limits journalctl -k does not.

  • It only covers this boot. The buffer starts empty at every boot.
  • It drops old lines. On a busy machine, new messages overwrite the oldest. A server up for months may no longer have its boot messages in the buffer. The journal kept a copy when they arrived.
  • Its timestamps count seconds since boot. dmesg -T converts them to clock time. The util-linux man page warns that the result can be wrong: the clock it uses “is not updated after system SUSPEND/RESUME”.

dmesg still has one advantage. It works when journald does not, for instance in a rescue shell or a minimal container host.

Want the journal’s output to look like dmesg? Use monotonic timestamps:

journalctl -k -o short-monotonic

“dmesg: read kernel buffer failed: Operation not permitted”

The kernel setting kernel.dmesg_restrict is 1. With it set, reading the ring buffer needs the CAP_SYSLOG capability, which in practice means root. Debian’s kernels turn it on by default.

Check it:

sysctl kernel.dmesg_restrict

Then pick one:

sudo dmesg
journalctl -k

The second works without sudo if your user is in the systemd-journal group, or adm on Debian and Ubuntu. Those groups can read the system journal, and the journal does not check dmesg_restrict. Not in either group? journalctl shows only your own entries, which never include the kernel’s, and prints a hint that you are not seeing messages from the system. The four ways journalctl says it found nothing covers that hint.

Narrowing kernel messages down

Every other journalctl filter works with -k. They AND together.

journalctl -k -p err                  # errors and worse
journalctl -k -g "I/O error"          # search the message text
journalctl -k -f                      # follow new kernel messages
journalctl -k -b -1 -p warning        # warnings from the previous boot

-p works well here. The kernel tags each message with a log level, and journald stores that level as the entry’s priority. KERN_ERR becomes err. So -p err on kernel messages means what it says. That is not true for most services. Showing only errors in journalctl covers why.

-g is case-insensitive until the pattern holds an uppercase letter. "I/O error" has one, so it is case-sensitive. Searching journalctl logs for a specific string covers that rule.

Lines that are not from the kernel

-k can show lines like systemd[1]: ... near the top of a boot. Early in boot, before journald starts, some programs write to the kernel’s log device. journald reads them from there, so they carry _TRANSPORT=kernel too.

The kernel’s own lines carry SYSLOG_IDENTIFIER=kernel. Add it to see only those:

journalctl -k SYSLOG_IDENTIFIER=kernel

The kernel lines worth knowing

Line contains What happened
Out of memory: Killed process The OOM killer ended a process to free memory
Memory cgroup out of memory: Killed process Same, inside a container or a unit with MemoryMax=
I/O error, dev sda, sector A disk read or write failed
EXT4-fs error ext4 found a problem in the filesystem

The OOM line is logged at err. The block layer rate-limits its I/O error lines, so a failing disk logs fewer lines than it has failures.

kern.log

Is rsyslog installed? Then the kernel’s messages are in a plain text file too. Debian and Ubuntu’s rsyslog config writes them to /var/log/kern.log. Red Hat’s puts them in /var/log/messages.

On Debian and Ubuntu, rsyslog reads the kernel’s log itself, through its imklog module. It does not need the journal. So kern.log can hold earlier boots on a server whose journal lives in memory. Check it before you give up on a crash.

A Debian 12 or 13 server has no rsyslog unless someone installed it, and so no kern.log.

Kernel errors from every server

journalctl -k reads one machine. An I/O error on one database host is the kind of line nobody goes looking for until the disk is gone.

Ship each journal with the CL Agent. It sends the whole journal, kernel messages included, so each kernel line arrives as a JSON row with _TRANSPORT, PRIORITY and _BOOT_ID intact. Boots stop mattering. A row stays until retention deletes it, whether the server rebooted or not.

Kernel errors from every host in the last day:

SELECT datetime(CAST(json_extract(msg, '$.__REALTIME_TIMESTAMP') AS INTEGER) / 1000000, 'unixepoch') AS logged,
       json_extract(msg, '$._HOSTNAME') AS host,
       json_extract(msg, '$.MESSAGE')   AS message
FROM logs
WHERE json_valid(msg)
  AND json_extract(msg, '$._TRANSPORT') = 'kernel'
  AND CAST(json_extract(msg, '$.PRIORITY') AS INTEGER) <= 3
  AND timestamp > CAST(strftime('%s', 'now', '-1 day') AS INTEGER)
ORDER BY timestamp;

Keep the CAST on PRIORITY. The journal stores every field as a string, and SQLite never treats a string as less than a number. Without the CAST, the query returns nothing and does not complain.

Keep json_valid(msg) first in the WHERE clause, too. One plain-text row in the source stops json_extract with an error.

How many kernel errors did each boot have? This one works on the journal’s boot IDs, so a reboot splits the count:

SELECT json_extract(msg, '$._HOSTNAME') AS host,
       json_extract(msg, '$._BOOT_ID')  AS boot,
       COUNT(*) AS errors
FROM logs
WHERE json_valid(msg)
  AND json_extract(msg, '$._TRANSPORT') = 'kernel'
  AND CAST(json_extract(msg, '$.PRIORITY') AS INTEGER) <= 3
GROUP BY host, boot
ORDER BY host;

Get an alert on OOM kills and disk errors

Add this as an alert rule. It fires when either line arrived in the last 10 minutes:

SELECT 1 FROM logs
WHERE json_valid(msg)
  AND json_extract(msg, '$._TRANSPORT') = 'kernel'
  AND (json_extract(msg, '$.MESSAGE') LIKE '%Killed process%'
       OR json_extract(msg, '$.MESSAGE') LIKE '%I/O error, dev%')
  AND timestamp > CAST(strftime('%s', 'now', '-10 minutes') AS INTEGER)
LIMIT 1;

The time window matters. Without it, one old match keeps the rule firing until retention deletes the row.

An alert on every kernel err is tempting. Try the per-boot query first. Some hardware logs the same harmless error at every boot, and an alert that always fires gets muted.

One limit to know. The agent uploads every 30 seconds. A kernel panic or a hard lockup stops it, so the last few seconds before a crash may never leave the machine. The lines that warned you minutes earlier will have.

Where to go next

💌 Get notified on new features and updates

Only sent when a new version is released. Nothing else.