PromQL selectors, functions and the pitfalls that fail silently — absent(), counter resets, vector matching, cardinality and histogram aggregation order.
23 of 23 entries
Including 5 pitfalls — the PromQL failures that produce silence rather than an error.
The latest sample for every series matching the labels, within a staleness window of 5 minutes by default.
metric{label="value"}http_requests_total{job="api", status="500"}This is reference text: nothing is evaluated and no Prometheus server is contacted. The two behaviours worth committing to memory both fail silently. An expression over a metric that does not exist returns no samples rather than zero, so an alert on it never fires — absent() is the only way to catch that. And a binary operator matches series on their full label set, so a label present on one side and not the other silently drops the pair.
PromQL Reference runs entirely on your device — this is maintained reference text, and no Prometheus server is contacted.
You’re on the Free plan. Compare plans
There is a whole group called Pitfall, and it is the reason this exists rather than a link to the documentation. The failures that cost the most in PromQL are not errors — they are silence.
An expression over a metric that does not exist returns no samples rather than zero, so an alert on it can never fire and nothing anywhere reports a problem. `absent()` is the only way to alert on a metric that has stopped arriving, and almost nobody reaches for it until after the outage it would have caught.
The other silent one is vector matching. Binary operators match series by their full label set unless `on` or `ignoring` narrows it, so a label present on one side and absent on the other silently drops the pair — the division you wrote returns fewer series than expected, or none, with no indication why.
Aggregation order for histograms gets its own entries because it is the mistake most often shipped. `rate()` must be applied to the buckets before `histogram_quantile`, and `le` has to survive the aggregation. Taking the quantile first and averaging afterwards produces a number that is not a quantile of anything, and it looks entirely plausible on a graph.
Cardinality is in there too, because it is the usual cause of a Prometheus falling over: every distinct combination of label values is a separate stored series, so a user ID or a full URL as a label multiplies that without bound.
Because an expression over a missing series returns no samples, not zero — so the comparison has nothing to evaluate and the alert cannot fire. `absent()` returns 1 when a selector matches nothing, and is the only way to alert on a metric that has gone away.
Vector matching. A binary operator pairs series by their complete label set, so one extra label on either side prevents the match and the pair is silently dropped. `on (instance)` or `ignoring (pod)` narrows what has to agree.
`histogram_quantile(0.99, sum by (le) (rate(bucket[5m])))` — rate over the buckets, aggregate keeping `le`, quantile last. Any other order produces a number that is not a quantile. And the accuracy is bounded by your bucket boundaries, so a p99 falling in a 1s-to-10s bucket is interpolated across that whole range.