console.log is not logging. It is printing.
It has no levels, no timestamps, no request context, and no structure. When your app crashes at 3am on a server you are not looking at, everything it printed went to a terminal that no longer exists — or into a journal you have to SSH in to read.
This guide replaces that with structured logs you can query. Three parts: log JSON with pino, get the lines off the box, and write the queries that make it worth having done.
Use pino. It writes newline-delimited JSON to stdout and it is fast enough that logging stops being a performance conversation.
npm install pino
// logger.js
import pino from 'pino'
export const logger = pino({
level: process.env.LOG_LEVEL || 'info',
base: {
service: process.env.SERVICE_NAME || 'api',
env: process.env.NODE_ENV || 'development',
},
redact: ['req.headers.authorization', 'req.headers.cookie', '*.password'],
})
import { logger } from './logger.js'
logger.info({ port: 3000 }, 'server started')
logger.warn({ userId: 42, attempts: 3 }, 'repeated login failure')
One line per event:
{"level":30,"time":1786133828970,"pid":1421,"hostname":"web-1","service":"api","env":"production","port":3000,"msg":"server started"}
Three of those settings do real work.
base stamps every line with the service and environment. Once two services ship to the same place, that is the field you filter on constantly. Add it on day one and you never have to backfill it.
redact is not optional. Log a request object without it and you have written session cookies and bearer tokens into a searchable database. Redaction happens before serialisation, so the secret never reaches the output.
level as an environment variable lets you turn on debug logging without a deploy.
This is the part that bites people.
The ingest endpoint splits on newlines. One line in, one row out. A stack trace printed raw becomes fifteen separate rows, and the fourteen after the first are orphans with no timestamp, no level and no context.
Pino solves this by serialising the error into the JSON object, where the newlines become \n inside a string:
try {
await chargeCard(order)
} catch (err) {
logger.error({ err, orderId: order.id }, 'charge failed')
}
That produces one line. The stack is intact inside err.stack, and json_extract(msg, '$.err.message') reaches the message.
Pass the error as err on the object. Do not pass it as the message — logger.error(err.stack) puts you right back to fifteen rows.
Catch the ones that escape too:
process.on('uncaughtException', (err) => {
logger.fatal({ err }, 'uncaught exception')
process.exit(1)
})
process.on('unhandledRejection', (err) => {
logger.fatal({ err }, 'unhandled rejection')
process.exit(1)
})
A log line that says “database timeout” with no request attached is a puzzle. Bind a request id and the line tells you which user, which route, and what else happened on that request.
With Express:
import { randomUUID } from 'node:crypto'
app.use((req, res, next) => {
req.log = logger.child({
requestId: req.headers['x-request-id'] || randomUUID(),
method: req.method,
path: req.path,
})
next()
})
Then log through req.log instead of the root logger. Every line from that request carries the same requestId, and one query pulls the whole story back out.
Fastify does this for you: request.log is already a child logger with a request id.
Two paths. Pick by how you run the process.
If your app runs as a systemd unit, its stdout already goes to the journal, and the CL Agent already ships the journal upstream. There is nothing to add.
# /etc/systemd/system/api.service
[Service]
ExecStart=/usr/bin/node /srv/api/index.js
SyslogIdentifier=api
Restart=always
SyslogIdentifier becomes the field you filter on. Set it per service.
Do not pipe through pino-pretty in production. It turns one JSON object into coloured multi-line text, which is exactly the format you are trying to avoid. Keep it in your dev script only.
The one thing to know about this path: the agent ships journal entries, so your JSON ends up nested inside MESSAGE as a string. Queries need two hops. Step 4 covers it.
Running in Docker without journald, or on a platform where you do not control the process supervisor? Ship from inside the app. Pino accepts any writable stream as its destination, so a batching stream is all it takes.
// cl-transport.js
import { Writable } from 'node:stream'
// Buffers log lines and POSTs them in batches to the ingest endpoint.
export function centralLogging({ endpoint, flushMs = 2000, maxLines = 500 }) {
let buffer = []
let timer = null
async function flush() {
if (buffer.length === 0) return
const body = buffer.join('')
buffer = []
try {
await fetch(endpoint, { method: 'POST', body })
} catch (err) {
// never throw from the logger; the app is not the log shipper's problem
process.stderr.write(`log shipping failed: ${err.message}\n`)
}
}
return new Writable({
write(chunk, _encoding, callback) {
buffer.push(chunk.toString())
if (buffer.length >= maxLines) {
flush()
} else if (timer === null) {
timer = setTimeout(() => { timer = null; flush() }, flushMs)
timer.unref()
}
callback()
},
final(callback) {
if (timer) clearTimeout(timer)
flush().then(() => callback())
},
})
}
// logger.js
import pino from 'pino'
import { centralLogging } from './cl-transport.js'
const stream = centralLogging({
endpoint: `https://logs.example.com/api/v1/ingest_logs/${process.env.CL_SOURCE_TOKEN}`,
})
export const logger = pino({ level: 'info' }, stream)
Four decisions in that file are worth stating.
It batches. One HTTP request per log line would cost more than the work being logged. The bulk endpoint takes newline-separated lines in one body, which is what buffer.join('') produces — pino already terminates each line with \n.
It never throws. A logger that crashes the process when the log server is unreachable has made things worse. Failures go to stderr and the app keeps running.
timer.unref() keeps a pending flush from holding the event loop open. Without it, node index.js will not exit for two seconds after the work is done.
final() flushes on shutdown, so the last few lines before a clean exit are not lost.
Keep the token in an environment variable. Committing it puts a write credential for your log server in your git history.
This transport is deliberately simple. It has no disk buffer and no retry, so a log server outage loses the lines sent during it. That is usually the right trade for application logs and the wrong one for an audit trail. If you need the guarantee, write to a file and ship the file.
Running in a container with no journald? The log driver is the shortest path.
docker run --log-driver=syslog \
--log-opt syslog-address=udp://rsyslog-host:514 \
--log-opt tag=api \
my-api
Central Logging has no syslog listener, so this needs an rsyslog relay in front of it. The Docker guide covers that setup, and the OPNsense and Synology guide covers the relay itself.
Option B above avoids the relay entirely, which is usually why people pick it.
Central Logging stores logs in SQLite and runs SQL over them, so json_extract reaches each field.
Every error from one service, most recent first:
SELECT
"timestamp",
json_extract(msg, '$.msg') AS message,
json_extract(msg, '$.err.message') AS error,
json_extract(msg, '$.requestId') AS request_id
FROM logs
WHERE json_valid(msg)
AND json_extract(msg, '$.level') >= 50
AND json_extract(msg, '$.service') = 'api'
ORDER BY "timestamp" DESC
LIMIT 100;
Pino levels are numbers: 10 trace, 20 debug, 30 info, 40 warn, 50 error, 60 fatal. >= 50 is “things that broke”.
The whole story of one request:
SELECT "timestamp", json_extract(msg, '$.msg') AS message
FROM logs
WHERE json_extract(msg, '$.requestId') = 'e3f1c8a2-...'
ORDER BY "timestamp";
That query is the reason to bind a request id. Without it you are reading interleaved lines from every concurrent request and guessing which belong together.
Which errors are happening most, which is not the same as which happened last:
SELECT
json_extract(msg, '$.err.message') AS error,
COUNT(*) AS occurrences
FROM logs
WHERE json_valid(msg)
AND json_extract(msg, '$.level') >= 50
GROUP BY error
ORDER BY occurrences DESC
LIMIT 25;
The full stack, once you know which error you care about:
SELECT json_extract(msg, '$.err.stack') AS stack
FROM logs
WHERE json_extract(msg, '$.err.message') = 'connect ETIMEDOUT'
ORDER BY "timestamp" DESC
LIMIT 1;
The agent sends journal entries, and your application’s JSON sits inside MESSAGE as a string. Reaching a field takes two hops:
SELECT
"timestamp",
json_extract(json_extract(msg, '$.MESSAGE'), '$.msg') AS message,
json_extract(json_extract(msg, '$.MESSAGE'), '$.err.message') AS error
FROM logs
WHERE json_extract(msg, '$.SYSLOG_IDENTIFIER') = 'api'
AND json_extract(json_extract(msg, '$.MESSAGE'), '$.level') >= 50
ORDER BY "timestamp" DESC
LIMIT 100;
Miss the second hop and every query returns nothing, with no error to explain why. It is the single most common mistake on this path.
The timestamp column is when the log server received the line, not when your app emitted it. Batching every two seconds keeps them close. A network outage does not: a backlog arrives all at once and sorts as if it just happened.
Pino’s own time field is milliseconds since the epoch. Sort by it when order matters:
ORDER BY json_extract(msg, '$.time') DESC
Turn the queries you care about into alert rules. This one catches a bad deploy:
SELECT COUNT(*) AS errors
FROM logs
WHERE json_valid(msg)
AND json_extract(msg, '$.level') >= 50
HAVING errors > 20;
Fatal errors deserve a rule of their own, because one is already too many:
SELECT COUNT(*) AS fatals
FROM logs
WHERE json_valid(msg)
AND json_extract(msg, '$.level') >= 60
HAVING fatals > 0;
Rules run every five minutes and notify through Slack, Telegram or Pushover. Set a max frequency on the rule, or an outage sends you a pager storm instead of one useful alert.
Log JSON with pino. Put the error on the object as err so a stack trace stays one line and one row. Bind a request id and log through the child logger.
On systemd, journald and the agent already do the shipping — just remember the second json_extract hop. Everywhere else, batch and POST straight from the app.
Central Logging is a single binary you install on a server you already own. See Deploy to get an instance running in about five minutes.
💌 Get notified on new features and updates