Halloween DealsAmazon USHalloween week deals: useful picks before month-endAmazon US: current deals, useful picks and tech finds.Check DealsPC HealthRecommendedCrashes, freezes, slowdowns? Check your PC nowSpot repairable issues before they interrupt work.Check PCHalloween DealsAmazon USOctober picks are moving - compare before checkoutAmazon US: current deals, useful picks and tech finds.See Picks×
Skip to content
World desk11 min

How to Query Prometheus: A Practical PromQL Guide for Self-Hosted Monitoring

A practical PromQL guide for self-hosted Prometheus covering selectors, label matching, range functions, aggregation, instant and range evaluation, API queries, and cardinality pitfalls.
Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

To query Prometheus, write a PromQL expression in the Prometheus expression browser or send that expression to the HTTP API. Use an instant query when you need the value at one timestamp, and a range query when you need regularly spaced values across a period. Start with a precise metric-and-label selector, inspect the result in the table view, then add range functions such as rate(), aggregation, and comparisons. This workflow prevents the most common mistakes: choosing a function that does not match the metric type, returning thousands of unintended series, or automating a query that has not been validated.

What PromQL is and why it matters

Prometheus includes PromQL (Prometheus Query Language), a functional language for selecting and aggregating time-series data in real time. A time series is identified by a metric name and a set of labels; every sample has a timestamp and a numeric value. PromQL lets you select those series, transform them, compare them, and reduce them to the dimensions that answer an operational question.

As an Amazon Associate I earn from qualifying purchases.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

For example, a raw request counter can tell you how many requests a target has recorded, while a PromQL expression can turn that counter into requests per second, group the result by service, and calculate the percentage returning server errors. The metric names and labels in the examples below are illustrative: your exporters and application instrumentation may use different names.

Choose instant or range evaluation first

PromQL can be evaluated at one point in time or repeatedly over an interval. The distinction affects both the interface you use and the shape of the result.

Query mode What Prometheus evaluates Expression browser view HTTP endpoint Typical use
Instant One timestamp (usually now, or a timestamp you specify) Table /api/v1/query Current health, a single alert value, or checking a selector
Range The expression at equally spaced steps between a start and end time Graph /api/v1/query_range Trends, dashboards, and historical analysis

A range query is not one calculation over a continuous stream. Prometheus evaluates the expression at each requested step, so a one-hour query with a 60-second step produces one result per minute. A smaller step gives finer resolution but increases work and response size.

Start with a metric and label selector

Select every series for a metric

The simplest selector is a metric name:

http_requests_total

This returns every currently available series with that metric name. In a self-hosted installation, a broad selector can expand to hundreds or thousands of series because each distinct label set is a separate time series. Use the table view first so you can see exactly which labels and values are present.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Filter with label matchers

Put label matchers in braces after the metric name:

http_requests_total{job="prometheus",group="canary"}

PromQL supports four matcher operators:

Operator Meaning Example
= Label equals an exact value {job="prometheus"}
!= Label does not equal a value {job!="batch"}
=~ Label matches a regular expression {job=~"api|web"}
!~ Label does not match a regular expression {job!~"test|dev"}

Regular expressions use RE2 syntax and are fully anchored. A matcher such as job=~"api" matches the complete label value api, not a value that merely contains the letters. To match a value containing that text, make the expression explicit, for example job=~".*api.*". Use the most selective matchers you can; filtering early reduces the number of series later operators must process.

Remember that labels define separate series

http_requests_total{job="api",instance="10.0.0.4:8080"} and the same metric with a different instance are different series. If your question is about a service rather than an individual target, you will usually need an aggregation such as sum by (job) later in the expression.

Use range selectors for recent samples

A range selector adds a lookback interval in square brackets. This selects all samples for each matching series in that interval:

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.
http_requests_total{job="prometheus"}[5m]

The interval is left-open and right-closed: samples exactly at the ending boundary are included, while a sample exactly at the beginning is excluded. Common duration units include seconds (s), minutes (m), hours (h), days (d), weeks (w), and years (y).

A range selector by itself is a range vector, which is a set of samples over time for each series. Most arithmetic and aggregation operators work on instant vectors instead, so range vectors are normally passed to a function such as rate() or increase().

Understand PromQL’s four expression types

An expression can evaluate to one of four documented types:

  • Instant vector: one sample for each matching series at a single timestamp. Metric selectors normally produce this type.
  • Range vector: a sequence of samples for each series over a lookback interval, produced by a selector such as [5m].
  • Scalar: one numeric value, useful for constants or calculations that reduce a result to a single number.
  • String: part of the expression model, but currently unused by the documented PromQL functions and operators.

Range queries accept an expression whose root is a scalar or an instant vector. Prometheus evaluates that root repeatedly at each requested step; a bare range vector is not a valid root for a range query.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Match the function to the metric type

Counters: use rate() or increase()

A counter is intended to increase over time, apart from resets when a process restarts. Apply counter-oriented functions to a range of samples:

rate(http_requests_total[5m])

rate() estimates the per-second increase over the selected five-minute window and accounts for counter resets. Use increase() when you want the estimated total increase during the interval instead of a per-second value:

increase(http_requests_total[1h])

The lookback window is a trade-off. A short window reacts quickly but is more affected by sparse scrapes and noise; a longer window is steadier but slower to reflect a change. The window should contain enough samples for the scrape interval and the behavior you are measuring.

Gauges: inspect the value or use gauge functions

A gauge can rise and fall, so treating it as a counter produces a misleading result. CPU temperature, memory in use, and queue depth are typical gauge-style measurements. Select a gauge directly for its current value, or use a function designed for a range of gauge samples when you need an average, minimum, maximum, or other summary. Always check how the exporter or application defines the metric before choosing a function.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Aggregate only along dimensions that matter

Aggregation reduces many series to a smaller set. The most useful forms are:

  • sum by (label) (expression) keeps one output series for each distinct value of the listed label.
  • sum without (label) (expression) removes the listed label dimensions while retaining the other labels.
  • Other aggregation operators, including avg, min, max, and count, use the same grouping idea.

For current request counts grouped by job:

sum by (job) (http_requests_total)

For a per-second request rate grouped by job:

sum by (job) (rate(http_requests_total[5m]))

Keep labels that explain the answer and remove dimensions that are irrelevant to the question. If you aggregate away a useful label too early, you cannot recover it later in the expression. Conversely, retaining high-cardinality labels can leave you with an expensive query and a result that is difficult to read.

Compare and combine series carefully

Binary operators can compare or combine vectors and scalars, but matching vectors by their full label sets is often stricter than expected. Two sides with different labels may produce no result, while many-to-many matches can multiply output series or be rejected. Aggregate each side to compatible dimensions before combining them, and use explicit vector-matching clauses only when you understand the resulting cardinality.

For a fleet-wide error percentage, aggregate the numerator and denominator at the same level:

100 * sum(rate(http_requests_total{status=~"5.."}[5m])) / sum(rate(http_requests_total[5m]))

The regular expression selects status labels beginning with 5. This expression returns one overall percentage because both sums remove the original label dimensions. If you need one percentage per service, group both sums by the same service label instead.

Useful PromQL examples

See whether Prometheus targets are up

up

Filter it to the Prometheus job:

up{job="prometheus"}

An instant table query shows the latest value for each matching target. A value of 1 indicates the target was up for the latest scrape and 0 indicates it was not.

Calculate CPU usage from idle CPU time

100 * (1 - avg by (instance) (rate(node_cpu_seconds_total{mode="idle"}[5m])))

This calculates the percentage not spent idle for each instance, assuming the exporter exposes an idle CPU-seconds counter with the shown label. The avg by (instance) operation combines CPU modes or cores into one value per instance according to the labels present in your data.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Look at a historical value with offset

http_requests_total offset 1h

offset 1h evaluates the selector one hour in the past. You can use offsets in comparisons, rates, and aggregations to compare a current expression with an earlier period, provided the historical samples are retained.

Run a query in the Prometheus web UI

  1. Open the Prometheus expression browser supplied by your server.
  2. Enter a simple metric selector, such as up, and execute it in the Table view. Confirm the metric exists and inspect its labels before adding functions.
  3. Add label matchers to narrow the result. If the table suddenly contains many more rows than expected, stop and refine the selector.
  4. For counters, add a range and a counter function, such as rate(http_requests_total[5m]).
  5. Add aggregation only after you can explain each label in the unaggregated result.
  6. Switch to the Graph view when you need a time trend. Choose a time range and an appropriate step so the graph is detailed enough without producing an unnecessarily large result.

Keeping the first checks in the table view makes errors visible: an empty result, an unexpected label, or an accidental high-cardinality selector is easier to diagnose before it is hidden in a graph or dashboard panel.

Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.Support on Ko-Fi

Query Prometheus through the HTTP API

Instant queries with /api/v1/query

External programs can submit a PromQL expression to the versioned API under /api/v1. Use GET or POST with the query parameter for an instant evaluation:

curl -G 'http://localhost:9090/api/v1/query' 
  --data-urlencode 'query=up{job="prometheus"}'

The response is JSON. The returned data type depends on the expression, and an instant vector normally contains one current sample per matching series. A script should check both the HTTP status and the API response status before processing the data.

What’s actually slowing this PC down?

Pick the symptom - the matching free tool is one click away.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Range queries with /api/v1/query_range

Send the expression with start, end, and step:

curl -G 'http://localhost:9090/api/v1/query_range' 
  --data-urlencode 'query=sum(rate(http_requests_total[5m]))' 
  --data-urlencode 'start=2026-10-15T00:00:00Z' 
  --data-urlencode 'end=2026-10-15T01:00:00Z' 
  --data-urlencode 'step=60s'

Use RFC3339 timestamps or Unix timestamps accepted by the API for the interval. Replace the illustrative times with the period you actually need. The step is the spacing between evaluations; it is not the scrape interval and should be chosen with the desired chart or report resolution in mind.

Interpret API errors

The API uses HTTP status codes to distinguish common failures:

  • 400: malformed or missing parameters, such as an invalid time value.
  • 422: the expression is syntactically accepted but cannot be executed.
  • 503: the query timed out or was aborted.

Log the expression, time range, step, and returned status in automation. A retry can help with a transient timeout, but repeatedly retrying an expensive query can increase load; first narrow the selector, shorten the interval, or increase the step.

Diagnose empty, wrong, or slow results

An empty result

  • Run the metric name alone to verify that the metric is present.
  • Remove label matchers one at a time and check spelling, capitalization, and label values.
  • Confirm that the selected time range contains samples and that the series has not gone stale after a target or label change.
  • Check the metric type and function. A counter function applied to a different kind of metric may not answer the intended question.

Too many series or a slow query

  • Replace a bare high-cardinality selector with exact matchers.
  • Aggregate before combining vectors, and keep only labels needed in the output.
  • Use a longer range only when it adds decision-making value; historical queries over many series are more expensive.
  • For a graph, use a step that matches the visual resolution instead of requesting every possible sample.

A rate or percentage looks wrong

  • Verify that the source is a counter and that the range contains multiple scrapes.
  • Look for counter resets caused by restarts; rate() and increase() are designed to account for them, but a very short window can still be noisy.
  • Ensure numerator and denominator use the same filters and aggregation dimensions.
  • Check that status, method, service, and instance labels have not created an unintended many-to-many combination.

The query parses but cannot execute

Separate syntax problems from execution problems. First test the smallest selector in the table view, then add one function, matcher, or operator at a time. If the complete expression triggers a 422 or 503 response, inspect cardinality and time range before changing the business logic.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

A safe workflow for dashboards and alerts

  1. Discover: query the metric name and inspect its labels and sample values.
  2. Define the question: decide whether you need a current value, a rate, a count over time, a percentage, or a historical comparison.
  3. Choose evaluation: use an instant query for a point-in-time decision and a range query for a trend.
  4. Apply correct semantics: use a range selector with counter functions and treat gauges according to their behavior.
  5. Control dimensions: aggregate by the labels that operators need to see; remove the rest.
  6. Validate cardinality: inspect the table result before putting the expression in a dashboard or alert.
  7. Automate: call the appropriate API endpoint, handle 400, 422, and 503 responses, and record the query parameters for troubleshooting.

This order keeps exploration separate from presentation. You can change a graph’s step or an API’s time range without silently changing what the metric means, and you can catch stale series or an accidental label explosion before they affect an operational workflow.

PromQL decision checklist

  • Do you need one timestamp or repeated values over an interval?
  • What metric name and label set identify the data?
  • Are the labels filtered with exact, negative, or anchored RE2 matchers?
  • Is the metric a counter or a gauge?
  • Does the function require a range selector such as [5m]?
  • Which labels should remain after aggregation?
  • Could vector matching multiply series or leave the two sides unmatched?
  • Is the selector narrow enough for the requested time range and step?
  • Have you verified the result in the table view before using it in a graph, alert, script, or service?

Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.

Leave a Reply

Your email address will not be published. Required fields are marked *

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

More from the Wire

  1. World desk4 min
    How to Spot an AI Voice Scam Before Sending MoneyDon’t rely on how a caller sounds. Pause, call back through a known number, and verify the emergency with another trusted person before sending money.
  2. Mountain View desk4 min
    Google’s SynthID Detector: How to Check AI-Generated Images, Video and AudioGoogle’s SynthID Detector looks for an embedded watermark in supported images, video and audio. Here is what its results do—and do not—show.
  3. Redmond desk20 min
    How to create a link to File or Folder in Windows 11Windows 11 gives you several ways to point to a file or folder without moving or duplicating it. You can create a desktop shortcut,…
Recommended PC Tool
Recommended PC Tool
Crashes, No Sound, or Screen Glitches?Free driver scan
Windows Errors? Fix Them Before They SpreadFree repair scan

Two free Windows tools

One Free Minute Could Fix That PC

Before you go - each of these free tools takes about a minute and tackles what quietly slows a Windows PC down.

Special offer. View Outbyte info, uninstall instructions, EULA, and Privacy Policy.