How to read Nginx access logs

Half your requests are missing from access.log

They are being written. They are in a different file.

Nginx inherits access_log from the enclosing block only if the current block sets none of its own. One access_log line inside one location and that location stops writing to the server-level file. Nothing warns you. nginx -t passes.

This page covers finding the file, reading the format, counting things in it correctly, and what to do once one server is not enough.

Find the file

The package default is /var/log/nginx/access.log. Read the config instead of trusting it:

grep -r access_log /etc/nginx/

That prints every access_log in every included file. Three things to look for.

access_log off; — logging is disabled for that block. This is the other reason the file looks short.

A second path. A vhost or a location writing somewhere else. That is the opening symptom.

A format name at the end of the line. access_log /var/log/nginx/access.log main; means the lines are in a format called main, not the built-in one. Find its definition:

grep -r -A15 log_format /etc/nginx/

If the config sets no access_log anywhere, nginx uses the path it was compiled with:

nginx -V 2>&1 | tr ' ' '\n' | grep http-log-path

And confirm you are reading the config that is loaded:

nginx -t

That prints the file in use. On a box with both a distribution nginx and a hand-built one, this is where the confusion usually ends.

Read the format

With no format named, nginx uses combined. The definition is compiled in:

$remote_addr - $remote_user [$time_local] "$request" $status $body_bytes_sent "$http_referer" "$http_user_agent"

A line looks like this:

203.0.113.9 - - [20/Sep/2026:10:14:02 -0700] "GET /pricing HTTP/1.1" 200 18422 "https://example.com/" "Mozilla/5.0 (X11; Linux x86_64)"

Four of those mislead people.

$remote_addr is the last hop. Put nginx behind Cloudflare or an AWS load balancer and every line says the same handful of addresses. The client is in X-Forwarded-For. Log it with $http_x_forwarded_for, or use the realip module and let it rewrite $remote_addr for you:

set_real_ip_from 10.0.0.0/8;
real_ip_header X-Forwarded-For;

Check the module is compiled in with nginx -V 2>&1 | grep realip. With no set_real_ip_from the module does nothing at all, which is the safe default. Keep the list to the proxies you actually run. Widen it to 0.0.0.0/0 and any client can write whatever address it likes into your logs by sending the header itself.

$time_local is the end of the request, not the start. A request that took 30 seconds is stamped 30 seconds after the user pressed the button. Two lines with the same timestamp did not necessarily arrive together.

$body_bytes_sent excludes the response headers. It is bytes sent on the connection minus header size, so a 304 shows 0 even though nginx answered. Use $bytes_sent for the number your bandwidth bill cares about.

There is no timing field at all. The combined format records nothing about how long anything took. “Check the access log for slow requests” is not a thing you can do until you change the format. See below.

The one nginx escapes, and the one it does not

Nginx escapes some bytes inside logged variables as \xHH. From the escape table in the source: the double quote, the backslash, every control character, DEL, and every byte above 127.

So a quote can never appear inside a quoted field. The record is always parseable by splitting on ". That matters in a moment.

A space is not escaped. Nginx logs $request exactly as the client sent it, and a client can send anything:

198.51.100.7 - - [20/Sep/2026:11:02:00 -0700] "GET /search?q=hello world HTTP/1.1" 200 812 "-" "-"

That is one request line with a space in it. It shifts every field after it by one.

Count status codes without the usual mistake

The status is the ninth whitespace field, so everyone writes this:

awk '$9 >= 500' /var/log/nginx/access.log

On the line above, $9 is HTTP/1.1". Awk compares a non-numeric operand as a string, H sorts after 5, and the test passes. A 200 gets counted as a server error. No warning.

Measured on a 203-line sample: $9 >= 500 returned 32, the true count was 31.

Split on the quote instead. Nginx escapes quotes inside fields, so the character is a reliable delimiter:

awk -F'"' '{split($3, a, " "); print a[1]}' /var/log/nginx/access.log \
  | sort | uniq -c | sort -rn

    138 200
     17 500
     17 404
     14 502
     11 304
      5 301
      1 400

Field 3 of a quote-split line is status bytes, whatever the request line contained.

Count the 5xx and nothing else:

awk -F'"' '{split($3, a, " "); if (a[1]+0 >= 500) n++} END {print n+0}' \
  /var/log/nginx/access.log

a[1]+0 forces a numeric comparison, so an unparseable field scores 0 rather than sorting as text.

Top clients:

awk '{print $1}' /var/log/nginx/access.log | sort | uniq -c | sort -rn | head -20

$1 is safe. Nothing can shift the first field.

Use awk 'END {print NR}' rather than grep -c to count lines. A file with no trailing newline is one line short under some grep builds, and an access log that is being written right now usually has no trailing newline.

Add the fields that are missing

log_format is only valid in the http block. Putting it inside a server block is a config error, and that surprises people who keep their vhosts in separate files.

Add timing:

log_format timed '$remote_addr - $remote_user [$time_local] '
                 '"$request" $status $body_bytes_sent '
                 '"$http_referer" "$http_user_agent" '
                 '$request_time $upstream_response_time';

access_log /var/log/nginx/access.log timed;

sudo nginx -t && sudo systemctl reload nginx

$request_time is seconds with millisecond resolution, measured from the first byte nginx read off the client to the moment the log line is written. $upstream_response_time is the part your application is responsible for. The gap between them is nginx, the network, and a slow client.

Log both. When $request_time is 30 and $upstream_response_time is 0.02, your application is fine and someone is uploading over a phone connection.

Four reasons a line is not there yet

Buffering. access_log /path main buffer=64k flush=5s; holds lines in memory. They appear when the buffer fills or the flush timer expires. tail -f on a buffered log looks broken and is not. flush= without buffer= is a config error, so a config that loads has one or neither.

Conditional logging. access_log /path main if=$loggable; drops the line when $loggable is empty or 0. This is how health checks get filtered out, and how the request you are looking for gets filtered out with them.

The container symlink. The official nginx image points /var/log/nginx/access.log at /dev/stdout. There is no file to tail. The lines are in Docker:

docker logs --tail 100 my-nginx

Viewing Docker container logs covers the rest of that.

A full disk. This one is in the source and in no tutorial. When a write to the access log returns ENOSPC, nginx records the second it happened and skips every access-log line for the remainder of that second. The comment explains why: on some filesystems writing to a full disk blocks for a long time, so nginx would rather drop the line than stall the worker.

It complains about the failure at most once every sixty seconds. So a disk that filled at 14:00 produces one alert line and a gap in the access log that nothing else accounts for. Check free space before you go looking for a config problem.

One server is easy

Four is not.

A user reports a failed checkout. It hit one of your web servers and you do not know which, so you open four terminals. Last month’s requests are in a rotated, gzipped file on a machine you have since rebuilt.

awk on one file is fine. Copy the lines somewhere central and you can ask the question once.

Ship the access log

Nginx can write straight to syslog, which on a systemd host means the journal, which the CL Agent already reads:

access_log syslog:server=unix:/dev/log,tag=nginx_access,severity=info combined;

Reload, then check it arrived:

journalctl -t nginx_access -n 5 -o json

Pick the tag= carefully. It becomes SYSLOG_IDENTIFIER in the journal, and that field is how you separate web logs from everything else at query time.

Syslog logging cannot be buffered, so buffer= and syslog: together is a config error.

Then install the agent and point it at your server:

# /etc/clagent.toml
URL = "https://logs.example.com/api/v1/ingest_logs/YOUR-SOURCE-TOKEN"

Full setup, including a JSON log_format worth using, is in sending Nginx logs to a central server.

Query it

The agent runs journalctl -o json, so each entry arrives as an object and the nginx line sits in MESSAGE.

Most recent requests, every host:

SELECT timestamp,
       json_extract(msg, '$._HOSTNAME') AS host,
       json_extract(msg, '$.MESSAGE')   AS line
FROM logs
WHERE json_valid(msg)
  AND json_extract(msg, '$.SYSLOG_IDENTIFIER') = 'nginx_access'
ORDER BY timestamp DESC
LIMIT 100;

json_valid(msg) goes first and stays first. One plain-text line anywhere in the source makes json_extract raise malformed JSON, and SQLite short-circuits AND in a WHERE clause, so the guard protects the terms after it. It does not protect the SELECT list — the guard has to remove the bad rows, not merely precede the extracts.

5xx responses only, text format, using the same quote trick:

SELECT timestamp,
       json_extract(msg, '$._HOSTNAME') AS host,
       json_extract(msg, '$.MESSAGE')   AS line
FROM logs
WHERE json_valid(msg)
  AND json_extract(msg, '$.SYSLOG_IDENTIFIER') = 'nginx_access'
  AND json_extract(msg, '$.MESSAGE') LIKE '%" 5__ %'
ORDER BY timestamp DESC
LIMIT 200;

" 5__ matches the quote that closes the request line, a space, and a three-digit code starting with 5. It is a pattern match, not a parse. It is also the last query on this page that has to guess, because the fix is to stop logging text.

Then switch the format and stop guessing

Give nginx a JSON log_format and every field becomes a column:

log_format json_combined escape=json
  '{'
    '"remote_addr":"$remote_addr",'
    '"host":"$host",'
    '"request_method":"$request_method",'
    '"request_uri":"$request_uri",'
    '"status":$status,'
    '"body_bytes_sent":$body_bytes_sent,'
    '"request_time":$request_time,'
    '"user_agent":"$http_user_agent"'
  '}';

Keep escape=json. The default escaping writes a quote as \x22, which is not valid JSON, and one user agent containing a quote then breaks that line. Your queries drop it and say nothing.

Now the journald entry holds JSON inside JSON, and this is where people lose an afternoon.

The nested guard

json_valid(msg) says the journald entry is JSON. It says nothing about MESSAGE. Run this against a source where one vhost still logs text:

SELECT json_extract(json_extract(msg, '$.MESSAGE'), '$.status') AS status,
       COUNT(*) AS n
FROM logs
WHERE json_valid(msg)
  AND json_extract(msg, '$.SYSLOG_IDENTIFIER') = 'nginx_access'
GROUP BY status;

Error: stepping, malformed JSON

Guard both levels:

WHERE json_valid(msg)
  AND json_extract(msg, '$.SYSLOG_IDENTIFIER') = 'nginx_access'
  AND json_valid(json_extract(msg, '$.MESSAGE'))

Turn that guard around and you have a useful query in its own right — every line that is still text, which is the list of vhosts you have not converted:

SELECT json_extract(msg, '$._HOSTNAME') AS host, COUNT(*) AS text_lines
FROM logs
WHERE json_valid(msg)
  AND json_extract(msg, '$.SYSLOG_IDENTIFIER') = 'nginx_access'
  AND NOT json_valid(json_extract(msg, '$.MESSAGE'))
GROUP BY host;

The CAST that decides whether the numbers are real

This looks correct and is not:

SELECT json_extract(json_extract(msg, '$.MESSAGE'), '$.status') AS status,
       json_extract(json_extract(msg, '$.MESSAGE'), '$.request_uri') AS uri
FROM logs
WHERE json_valid(msg)
  AND json_valid(json_extract(msg, '$.MESSAGE'))
  AND json_extract(json_extract(msg, '$.MESSAGE'), '$.status') >= 500;

Run it against a source where one producer quoted the status — a second web server, an old format, a reverse proxy in front — and the result includes this row:

status  uri
------  --------
200     /strings

SQLite sorts every TEXT value above every INTEGER value, whatever the text says. '200' >= 500 is true. So does 'abc' >= 500. The comparison does not error; it over-matches, silently, forever.

CAST both sides:

SELECT CAST(json_extract(json_extract(msg, '$.MESSAGE'), '$.status') AS INTEGER) AS status,
       COUNT(*) AS n
FROM logs
WHERE json_valid(msg)
  AND json_extract(msg, '$.SYSLOG_IDENTIFIER') = 'nginx_access'
  AND json_valid(json_extract(msg, '$.MESSAGE'))
  AND CAST(json_extract(json_extract(msg, '$.MESSAGE'), '$.status') AS INTEGER) >= 500
GROUP BY status;

status  n
------  -
500     5
502     4

The uncast version returned ten rows. The cast version returns nine. The extra one was a 200.

A <= threshold fails the other way and matches nothing. Neither says a word. Cast every extracted value you compare as a number.

The queries worth keeping

Slowest paths, once request_time is in the format:

SELECT json_extract(json_extract(msg, '$.MESSAGE'), '$.request_uri') AS uri,
       COUNT(*) AS hits,
       ROUND(AVG(CAST(json_extract(json_extract(msg, '$.MESSAGE'), '$.request_time') AS REAL)), 3) AS avg_seconds,
       ROUND(MAX(CAST(json_extract(json_extract(msg, '$.MESSAGE'), '$.request_time') AS REAL)), 3) AS slowest
FROM logs
WHERE json_valid(msg)
  AND json_extract(msg, '$.SYSLOG_IDENTIFIER') = 'nginx_access'
  AND json_valid(json_extract(msg, '$.MESSAGE'))
GROUP BY uri
ORDER BY avg_seconds DESC
LIMIT 25;

Which server is sick:

SELECT json_extract(msg, '$._HOSTNAME') AS host, COUNT(*) AS errors
FROM logs
WHERE json_valid(msg)
  AND json_extract(msg, '$.SYSLOG_IDENTIFIER') = 'nginx_access'
  AND json_valid(json_extract(msg, '$.MESSAGE'))
  AND CAST(json_extract(json_extract(msg, '$.MESSAGE'), '$.status') AS INTEGER) >= 500
GROUP BY host
ORDER BY errors DESC;

One host with all the errors is a sick host. Errors spread evenly is a sick backend.

Get told about the next one

An alert rule fires when its query returns any row. The engine checks whether the result is empty and reads nothing further, so keep the result small.

Fire when 5xx responses pass 20 in ten minutes:

SELECT 1 WHERE (
  SELECT COUNT(*) FROM logs
  WHERE json_valid(msg)
    AND json_extract(msg, '$.SYSLOG_IDENTIFIER') = 'nginx_access'
    AND json_valid(json_extract(msg, '$.MESSAGE'))
    AND CAST(json_extract(json_extract(msg, '$.MESSAGE'), '$.status') AS INTEGER) >= 500
    AND timestamp > CAST(strftime('%s', 'now', '-10 minutes') AS INTEGER)
) >= 20 LIMIT 1;

Use a scalar subquery for the threshold. SELECT 1 FROM logs WHERE ... HAVING COUNT(*) >= 20 raises HAVING clause on a non-aggregate query, and a rule that errors does not warn you. It never fires.

Rules run every five minutes and deliver within a minute. Set a max frequency, or a two-hour outage pages you twenty-four times. More in alerting on errors in your logs.

Keep the file off the disk ceiling

Nginx does not rotate its own logs. The distribution package ships /etc/logrotate.d/nginx; a hand-built nginx ships nothing, and the access log grows until the disk fills.

The postrotate step is the part that matters. Nginx holds the renamed file open, so logrotate has to tell it to reopen:

/var/log/nginx/*.log {
    daily
    rotate 14
    compress
    delaycompress
    missingok
    notifempty
    postrotate
        [ -f /var/run/nginx.pid ] && kill -USR1 $(cat /var/run/nginx.pid)
    endscript
}

Without the kill -USR1, nginx keeps writing to the rotated file and the new access.log sits at zero bytes.

Rotation is also why “search last quarter” is a different problem from “search today”. Once the lines are on a log server, retention is one setting instead of fourteen gzip files per host.

Where to go next

• Checking Nginx error logs — the other file, and why it looks empty
• Sending Nginx logs to a central server — the full shipping setup
• Checking Apache error logs — the same job on the other web server
• Querying logs with SQL — the query language in more depth
• Alerting on errors in your logs — rules that fire when they should
• Viewing Docker container logs — when nginx runs in a container
• Sending logs to a server with curl — shipping a file without an agent

Central Logging is a single binary you install on a server you already own. See Deploy to get an instance running in about five minutes.