Sending Nginx logs to a central server

Introduction

Someone hammered /login last Tuesday and you want to know who. You SSH in, open /var/log/nginx/access.log, and logrotate already deleted the answer.

Nginx writes logs to local files by default. That works until you run a second web server. Or until a disk fills. Or until the line you need is gone.

This guide covers three things:

1. Reformatting Nginx logs as JSON so you can query them.
2. Getting those logs off the box to a central server.
3. Querying them once they arrive.

Step 1: Log JSON instead of the combined format

The default combined format is a space-separated string built for human eyes. Parsing it means writing regexes. Nginx can emit JSON instead, so skip the regexes.

Add this to the http block of /etc/nginx/nginx.conf:

log_format json_combined escape=json
  '{'
    '"time":"$time_iso8601",'
    '"remote_addr":"$remote_addr",'
    '"host":"$host",'
    '"request_method":"$request_method",'
    '"request_uri":"$request_uri",'
    '"status":$status,'
    '"body_bytes_sent":$body_bytes_sent,'
    '"request_time":$request_time,'
    '"upstream_response_time":"$upstream_response_time",'
    '"referer":"$http_referer",'
    '"user_agent":"$http_user_agent"'
  '}';

Do not skip escape=json. Without it, a user agent containing a quote breaks the JSON on that line. Your queries then drop those rows and say nothing about it.

Then point your access log at it:

access_log /var/log/nginx/access.log json_combined;

Test and reload:

sudo nginx -t && sudo systemctl reload nginx

You should now see one JSON object per line:

{"time":"2026-08-07T09:14:02-07:00","remote_addr":"203.0.113.9","host":"example.com","request_method":"GET","request_uri":"/pricing","status":200,"body_bytes_sent":18422,"request_time":0.041,"upstream_response_time":"0.039","referer":"https://example.com/","user_agent":"Mozilla/5.0"}

Step 2: Get the logs off the server

Two approaches work. Pick one based on whether you run systemd.

Option A: Log to syslog and let the agent ship it

On a systemd host, take the path with the fewest moving parts. Nginx writes to syslog. Syslog lands in the journal. The CL Agent already ships the journal upstream.

Change the access_log line to:

access_log syslog:server=unix:/dev/log,tag=nginx_access,severity=info json_combined;
error_log  syslog:server=unix:/dev/log,tag=nginx_error,warn;

Reload Nginx, then confirm the entries are arriving in the journal:

journalctl -t nginx_access -n 5 -o json

If the agent is already installed, you are done. Nginx logs now flow upstream with the rest of your system logs. The journalctl guide explains how that export works.

Pick the tag= values carefully. They become SYSLOG_IDENTIFIER in the journal, and that field is how you separate web logs from everything else at query time.

Option B: Tail the file and POST it

Maybe you want Nginx to keep writing files. Maybe you do not run systemd. Either way, ship the file directly. The ingest endpoint takes a batch of newline-separated lines in one request:

curl -X POST https://logs.example.com/api/v1/ingest_logs/YOUR-SOURCE-TOKEN \
    --data-binary @/var/log/nginx/access.log

Shipping continuously means remembering where you stopped, or you send the same lines twice. A script that saves the byte offset does the job:

#!/bin/bash
# /usr/local/bin/ship-nginx-logs.sh
set -euo pipefail

LOG=/var/log/nginx/access.log
OFFSET_FILE=/var/lib/nginx-shipper/offset
ENDPOINT="https://logs.example.com/api/v1/ingest_logs/YOUR-SOURCE-TOKEN"

mkdir -p "$(dirname "$OFFSET_FILE")"
offset=$(cat "$OFFSET_FILE" 2>/dev/null || echo 0)
size=$(stat -c %s "$LOG")

# The file was rotated out from under us; start over.
if [ "$size" -lt "$offset" ]; then
    offset=0
fi

if [ "$size" -eq "$offset" ]; then
    exit 0   # nothing new
fi

tail -c "+$((offset + 1))" "$LOG" \
    | gzip \
    | curl -sf -X POST "$ENDPOINT" \
        -H "Content-Encoding: gzip" \
        --data-binary @-

echo "$size" > "$OFFSET_FILE"

Run it every minute from cron:

* * * * * /usr/local/bin/ship-nginx-logs.sh

Three details in that script earn their place.

The rotation check (size < offset) handles logrotate truncating the file under you. Drop the check and the shipper goes quiet until someone notices.

Gzip cuts access logs about 10:1. On a busy server that is the difference between a rounding error and a real bandwidth bill.

set -euo pipefail means the offset only advances when the upload succeeds. A failed POST retries next minute instead of vanishing.

Step 3: Query the logs

Now the JSON pays off. Central Logging stores logs in SQLite and runs SQL queries against them, so json_extract reaches individual fields.

Slowest endpoints in the current window:

SELECT
    json_extract(msg, '$.request_uri') AS uri,
    COUNT(*)                           AS hits,
    ROUND(AVG(json_extract(msg, '$.request_time')), 3) AS avg_seconds
FROM logs
WHERE json_valid(msg)
GROUP BY uri
HAVING hits > 10
ORDER BY avg_seconds DESC
LIMIT 25;

Every client generating 5xx responses:

SELECT
    "timestamp",
    json_extract(msg, '$.remote_addr')    AS client,
    json_extract(msg, '$.status')         AS status,
    json_extract(msg, '$.request_uri')    AS uri
FROM logs
WHERE json_valid(msg)
  AND json_extract(msg, '$.status') >= 500
ORDER BY "timestamp" DESC
LIMIT 200;

Top talkers, which is how you spot a scraper:

SELECT
    json_extract(msg, '$.remote_addr') AS client,
    COUNT(*)                           AS requests
FROM logs
WHERE json_valid(msg)
GROUP BY client
ORDER BY requests DESC
LIMIT 20;

Step 4: Alert on the ones that matter

A query you have to remember to run is a query you will not run. Turn the useful ones into alert rules and let them come to you.

Start with 5xx responses. This rule catches a bad deploy before a customer emails you:

SELECT COUNT(*) AS errors
FROM logs
WHERE json_valid(msg)
  AND json_extract(msg, '$.status') >= 500
HAVING errors > 10;

Central Logging evaluates alert rules every five minutes and notifies through Slack or Telegram. Set a max frequency on the rule. Otherwise a long outage sends you a pager storm instead of one useful alert.

A note on error logs

Nginx error logs are not JSON, and you cannot make them JSON. The format is fixed. Ship them anyway — upstream connection failures and TLS handshake problems show up there and nowhere else. Send them as plain text and search them with full-text search instead of SQL:

upstream timed out
SSL_do_handshake() failed

Both are worth an alert rule.

Summary

Log JSON, not the combined format. Ship through syslog and the agent on systemd, or through an offset-tracking script anywhere else. Let SQL do the analysis.

You end up with one place to search every web server you run, and no per-GB ingest bill.

Central Logging is a single binary you install on a server you already own. See Deploy to get an instance running in about five minutes.