Observability

Pull-based scraping and a query language worth learning properly.

Start with

promtool check config prometheus.yml

The gotcha

Label cardinality is the failure mode. One unbounded label will take the server down.

Visualization over Prometheus, Loki and almost anything else.

Start with

docker run -p 3000:3000 grafana/grafana-oss

The gotcha

A dashboard nobody opens is not monitoring. Alerts are the product; dashboards are for the investigation.

One vendor-neutral path for traces, metrics and logs, so switching backends is a config change.

Start with

otelcol --config config.yaml

The gotcha

Sampling decisions made at the collector are hard to undo later. Decide what you keep before you need it.

Indexes labels rather than content, which keeps it affordable at volume.

Start with

logcli query '{app="api"} |= "error"'

The gotcha

Because it does not index content, a bad label schema makes queries slow in a way more indexing will not fix.