October DealsAmazon USOctober deal check: compare before you payAmazon US: current deals, useful picks and tech finds.Check DealsClean PCRecommendedOne scan can reveal what keeps slowing WindowsLook for cleanup and repair opportunities.Run ScanOctober DealsAmazon USDeal season is back - check today's better picksAmazon US: current deals, useful picks and tech finds.See Picks×
Skip to content
World desk9 min

How to Attach Runbook Links, Logs, and Traces to Alertmanager Notifications

Attach diagnostic destinations to Prometheus alert annotations and render them in Alertmanager notifications, choosing common or per-alert values based on whether links differ.
Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Put runbook, log, and trace destinations in Prometheus alert-rule annotations, then render those annotation values in the Alertmanager receiver template. Use .CommonAnnotations when every alert in a notification group shares the same links; range over .Alerts and render each alert’s annotations when destinations differ. The annotation-and-template workflow carries links into notifications, but it does not generate a logs query or application trace URL for you: those destinations must follow the URL format and context required by your observability stack.

How the links travel from a Prometheus rule to a notification

The workflow has two template stages, each with its own data and purpose:

As an Amazon Associate I earn from qualifying purchases.

  1. Prometheus evaluates the alert rule. Its annotations can contain longer informational values, including a description or runbook link. Prometheus can evaluate annotation templates using alert variables such as $labels and $value.
  2. Alertmanager formats the notification. The receiver’s notification template receives alert data, including annotations. It can print values shared by the notification group or format each alert separately.

These stages are separate. A Prometheus rule annotation uses Prometheus alert variables; an Alertmanager notification template uses notification data such as .CommonAnnotations and .Alerts. Putting a URL in an annotation alone does not guarantee it will appear in a message: the receiver must render it.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

A useful division of responsibility is to put alert-specific context in the rule annotation, then decide in the Alertmanager template how that context should be presented to the target receiver. This keeps the destination attached to the alert while leaving message formatting to the notification layer.

#1 Best Overall
Sale
TOZO A1 Wireless Earbuds Bluetooth 5.3 Light Weight in Ear Headphones | IPX5 Waterproof,2 Mics for AI Calls,Immersive Premium Sound Bass Headset with Charging Case,32 EQ via App Black
  • [Ultra-Lightweight Ear Buds Designed for Small Ears] Each earbud weighs only 3.7g and features a compact, ergonomic in-ear design made especially for small ears. Secure, low-profile, and comfortable for workouts, all-day wear or overnight sleeping.The ultra-slim profile sits flush in your ear with no hard edges poking or rubbing, so you can even sleep on your side without discomfort.
  • [Immersive Stereo Sound with TOZO OrigX Technology] TOZO OrigX tuning delivers clear vocals, balanced mids, and natural stereo sound for music, podcasts, and videos.
  • [Long Battery Life for Daily Use] Get up to 7 hours of playtime on a single charge, with up to 32 hours total using the charging case—ideal for workdays, commuting, and extended listening sessions.
  • [Bluetooth 5.3 & Stable Connection] Bluetooth 5.3 provides fast pairing, stable wireless performance, and reduced dropouts as you move around home or office.
  • [Deep Bass with Clear Vocals] High-performance drivers produce punchy bass while keeping vocals clean and detailed for everyday listening.

How to add a runbook, logs, and traces to an alert rule

Add stable annotation keys such as runbook_url, logs_url, and traces_url under the alert rule’s annotations. For a destination that is the same for every firing instance of the alert, use a static URL. If the observability product documents a URL format that accepts alert context, an annotation can be templated using variables such as $labels or $value.

- alert: ApiHighErrorRate
  expr: ...
  annotations:
    summary: "High error rate for {{ $labels.service }}"
    runbook_url: "https://docs.example.invalid/runbooks/api-high-error-rate"
    logs_url: "https://observability.example.invalid/logs?service={{ $labels.service }}"
    traces_url: "https://observability.example.invalid/traces?service={{ $labels.service }}"

This is an illustrative shape, not a tested, drop-in configuration. The expr value and destination URL formats are intentionally not specified. Replace the example destinations with addresses and query syntax supported by your own stack. In particular, templating a label into a query parameter is only appropriate if that product’s URL scheme accepts it and the value is encoded in the form that scheme requires.

Choose useful destinations, not just URL-shaped text

  • A runbook link should take the responder to the procedure relevant to the alert, rather than to a general documentation landing page.
  • A logs link should open a useful view for the alert’s service or other relevant context. The annotation mechanism does not define a universal logs URL or query language.
  • A traces link should point to an application-trace view supported by your tracing product. Alertmanager’s own tracing configuration is not a substitute for this application link.

There is no universal logs or traces URL recipe established by the Prometheus and Alertmanager template mechanism. The query syntax, URL parameters, time-window behavior, and encoding requirements are specific to the observability product. Confirm those details in that product’s documentation before building a templated destination.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Keep rule annotations manageable

Prometheus supports annotation templates, but that does not mean every piece of link-building logic belongs in a rule. Keep the rule annotations straightforward. If constructing a destination requires complicated logic, the Prometheus template examples recommend linking to a console for more involved needs rather than making the alert-rule template do too much.

Rank #2
Sale
Smart Watch for Men Women Compatible with iPhone Samsung Android Phone 1.69 inch Full Touch Screen IP68 Waterproof Bluetooh Fitness Tracker Heart Rate/Sleep Monitor (Black)
  • 【IP68 Waterproof Multifunctinal Smart Watch】Torjalph fitness activity tracker smart watch is IP68 waterproof technology ,you can wear it when swimming,handwashing,it's rain-proof which means that you perform multiple exercise modes outside without worrying about sweat or rain corroding your fitness smart watch,and this watch equipped with a 230 mAh large capacity battery magnetic charging ,and this sports activity tracker watch can be used 5-7 days If fully charged.
  • 【Larger 1.83'' Full Screen Watch & Multi-dial Selection】1.83 inch full HD color touch screen smart watch for men / women provides excellent 240*284 resolution image quality ,This Bluetooth smart watch offers a better touch experience and remarkable immersion and ,more than 200 watch faces for you to choose from App"Da Fit ", you can change the dial style according to your mood to wear keeping the world on your wrist
  • 【Accurate Activity Tracker Smart Watch 】sports watch activity tracker uses sweat and dust resistant breathable silicone strap, our Fitness Tracker Smart Watch for Android iOS Phones supports 12 sports modes,walking,running,bicycle,swimming,badminton,basketball,football,rope skipping. This bluetooth calls smart watch for men women is lightweight and stylish, bringing more convenient and relaxing sports experience while also providing more protection.
  • 【Your 24 Hour Personal Daily Health Monitor】Fitness watch for women men full touch screen IP67 waterproof bluetooth smartwatch for Android iOS Phones with heart rate blood pressure monitor multifunction Andriod smart watch with steps counter sleep monitor sports fitness watch, monitoring blood oxygen saturation allows you to understand your physical condition in real time to track fitness progress ,to know your health status in time.
  • 【More Functions Bluetooth Call Smart Watch】This fitness activity smart watch full screen for iPhone and Android phones includes a bunch of new features including:The bluetooth smart watch can hang up the phone, receive and display WhatsAPP /Text /Facebook /Twitter /Messenger reminder.Support sedentary reminder, alarm clock, stopwatch, timer,weather information,music control, remote camera control compatible with Android 5.0 and above or iOS 9.0 and above smartphones.

How to render shared links in an Alertmanager notification

When the relevant annotations are common to the entire notification group, render them using .CommonAnnotations in the receiver’s message field. For example, a Slack receiver can use the following message shape:

receivers:
- name: service-alerts
  slack_configs:
  - channel: '#alerts'
    text: >-
      {{ .CommonAnnotations.summary }}
      Runbook: {{ .CommonAnnotations.runbook_url }}
      Logs: {{ .CommonAnnotations.logs_url }}
      Traces: {{ .CommonAnnotations.traces_url }}

.CommonAnnotations represents annotations shared by the alert group. That makes it a concise choice when one runbook, logs view, and traces view accurately serve every alert represented in the notification.

Do not assume the common form is suitable simply because the rule uses the same annotation keys. The values must be shared across the group. If the URLs vary between alerts, a common-value message cannot preserve each alert’s distinct destination; render them per alert instead.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

How to preserve different links for alerts in the same group

When alerts in one notification group have different annotations, range over .Alerts and print each alert’s annotations alongside useful alert labels. Alertmanager’s notification examples use this pattern for per-alert details.

Rank #3
New Bee Guitar Cable 10ft Electric Instrument Bass AMP Cord
  • [Low Noise High Fidelity] New bee guitar cord with 22AWG oxygen-free-copper center conductors provides enhanced signal transfer, the PVC insulation material and the high-density stranded copper double shield provide noise rejection for quiet operation
  • [Ideal for Choice] New bee 1/4 straight and right angle metal connectors, ensures a secure connection in any instrument, pedal, or amp jack. L shape great fro on stage guitar stands
  • [Wide Compatible] Suitable for mixer, power amplifier, microphone, effector, decoder, equalizer, electoronic organ, electric guitar, electric bass and other electronic equipments. Widely used in the installation of smart home, karaoke speaker, professional stage audio, etc
  • [Easy Disassemble] Making incidental repairs quick and easy! Very nice, flexible cable
  • [Dont Hesite to Order] 24 hours customer services and professional technology team are standing by
{{ range .Alerts }}
Alert: {{ .Labels.alertname }}
Runbook: {{ .Annotations.runbook_url }}
Logs: {{ .Annotations.logs_url }}
Traces: {{ .Annotations.traces_url }}
{{ end }}

This shows the key distinction: .CommonAnnotations is for shared group values; .Annotations inside the alert iteration is for the individual alert. The exact message layout is up to the receiver template. Include enough alert identity to make each set of links understandable when several alerts arrive together.

Use the shared form for a compact notification when destinations truly match. Use per-alert rendering when links depend on a service, alert instance, or other alert context. The right choice is determined by whether the values are common, not by whether a notification contains one alert or many.

Static URLs or alert-templated URLs?

Approach Use it when Trade-off
Static URL in an annotation The destination is stable and suitable for all instances of the alert, such as one runbook page. It is simple to maintain, but does not tailor the destination to alert labels or values.
Templated URL in an annotation The observability product documents a URL format that can use alert context, and the needed values are available as alert variables. It can make a destination more specific, but the URL format and encoding must be valid for that product.
Link to a console for complex needs Building the final destination would require complicated logic in the rule annotation template. The alert points to a console rather than encoding all destination logic in the rule; the exact console behavior depends on the stack.

Neither approach is universally better. Choose a static URL when it already provides a useful destination. Add alert context only when the destination format supports it and that context materially helps the responder.

Free tools Windows power users keep installed

One-click scans. No signup required.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Built-in receiver templates, reusable templates, or a webhook

Alertmanager supports built-in integrations and a generic webhook. The destination determines what the output needs to look like: a message field for one receiver may not be the right representation for another. Render or transform the annotation values to fit the target’s expected format rather than assuming one template works unchanged everywhere.

Rank #4
BSIMB 15.6" Digital Family Calendar & Chore Chart, No Subscription Fees
  • Digital Family Calendar: BSIMB digital calendar wall touch screen puts an end to your chaotic life. Designed for busy families, the digital calendar keeps you organised and in step with your family. Assign different colours and tasks to each family member to make family scheduling easier and more intuitive.
  • Easy Setup and Auto-Sync: Plug in, connect to WLAN and auto-syn your electric calendar in minutes. Digital wall calendar compatible with Google calendar, iCloud calendar, Outlook calendar, Cozi calendar and Yahoo calendar. Add events directly on the device or through the ecalendar app. Electronic calendar stays current whether you're at work or traveling—no manual updates needed.
  • All-In-One Home Assistant: Smart calendar builds habits and encourage independence with personalised chores. Display daily meal plan to reduce the daily ‘What's for dinner?’ stress. To-do lists keep track of everything you want. Turning chores into valuable achievements with stars and rewards, making life easier and more enjoyable.
  • Manage Anytime Anywhere: BSIMB electronic calendar for family stay connected with the powerful ecalendar app. Access and manage your digital wall calendar anytime, anywhere with free features like stars & rewards, photo screensaver, chore charts, meals plans, to-do lists etc. Electronic planner paid features like Magic Import tailor advanced tools for users with special needs.
  • Stay in Touch with Loved Ones: Share photos or videos from your phone to electronic wall calendar with app, instantly update your life moment, and keep in touch with your family, especially those who are far away; you can also surprise your smart calendar digital wall by sending him a wish card; it's also great for keeping tech-illiterate elders in touch with the youngsters.

Put a short message directly in the receiver

For a small, receiver-specific message, render the annotation values in the relevant receiver field, as in the Slack example above. This keeps the message close to the configuration that uses it.

Use a named template when content is longer or reusable

For longer content or formatting reused across receiver configurations, load a named notification template through Alertmanager’s templates configuration and invoke it from the receiver field. Alertmanager’s official examples demonstrate a receiver’s text field calling a named template from a configured file. The exact template layout should match the deployed configuration and receiver.

Transform the structured payload for a webhook

A webhook receiver receives a structured notification payload that includes annotations and backlink fields. If the destination is a webhook consumer, have it use or transform those values into the format expected by that system. A webhook does not make the destination URL scheme universal; the receiving system still needs to know how to interpret the annotation values.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.Support on Ko-Fi

What ExternalURL and GeneratorURL do—and do not do

Alertmanager exposes two useful link fields, but neither should be mistaken for an annotation containing a runbook, logs, or application trace destination:

  • ExternalURL is a backlink to the Alertmanager instance.
  • GeneratorURL identifies the entity that caused the alert.

These links can help a recipient return to Alertmanager or inspect the alert’s origin. They are distinct from the destinations you add as runbook_url, logs_url, and traces_url. If responders need those destinations in the notification, add and render them explicitly.

Alertmanager tracing is not an application trace link

Alertmanager has an optional tracing configuration for instrumenting Alertmanager’s own distributed tracing. Its settings include items such as endpoint, sampling fraction, transport, TLS, compression, and timeout. That feature concerns traces of Alertmanager itself; it does not inject an application trace URL into a notification.

For a recipient to open a trace of the application event behind an alert, put the application trace destination in an alert annotation and render that annotation in the receiver template. Keep the two uses of “tracing” separate when troubleshooting: one concerns observability of Alertmanager, and the other is a link responders follow to inspect an application trace.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Implementation checklist before sending the configuration live

  1. Add the annotations to the intended alert rule. Use stable keys for runbook, logs, and traces so the receiver template can refer to them consistently.
  2. Decide whether each destination is shared or alert-specific. Use .CommonAnnotations only for values shared by the group; otherwise iterate over .Alerts.
  3. Check the destination URL format. Confirm the observability product’s supported query and time-window URL behavior. Do not assume a label embedded in a URL is encoded or interpreted correctly without checking.
  4. Render the values in the correct receiver field. Match the message or payload format to the built-in integration or webhook consumer that will receive it.
  5. Check recipient reachability and permissions. A correct internal URL is not useful if the people receiving the notification cannot reach the system or are not authorized to view the linked data.
  6. Consider escaping and untrusted content. Treat annotation content as text supplied through alert configuration and context. The Alertmanager template reference distinguishes text and HTML evaluation, which affects escaping. Use the appropriate output context and avoid assuming a URL is safe merely because it is an annotation.
  7. Validate against the deployed versions and operational process. The examples here illustrate the documented mechanism, not tested configurations. Confirm template syntax, URL encoding, and receiver formatting for the exact Prometheus, Alertmanager, and observability-product versions in use.

Reloading Alertmanager configuration

Alertmanager documentation describes configuration reload through SIGHUP or an HTTP POST to /-/reload. Check the documentation for the deployed version and your production operating procedure before using either method. A reload mechanism is not a substitute for validating that the annotation values, template expressions, and receiver-specific message format are correct.

Common implementation mistakes

  • Adding annotations but not rendering them. An annotation is the handoff point; the notification template must include the value for it to appear in the outgoing message.
  • Using group-common values for links that differ. Render each alert’s annotations when destinations vary within the group.
  • Expecting Prometheus variables in the Alertmanager template. Prometheus evaluates rule annotations with alert variables such as $labels; Alertmanager later formats notification data such as .CommonAnnotations and .Alerts.
  • Treating a backlink as a diagnostic destination. ExternalURL and GeneratorURL serve different purposes from runbook, logs, and trace annotations.
  • Assuming every observability product uses the same deep-link format. The annotations transport a value; they do not define how a logs or traces URL is constructed.
  • Confusing Alertmanager tracing with application traces. Alertmanager’s optional tracing configuration instruments Alertmanager; application trace links belong in the alert annotations.
  • Ignoring who can open an internal link. Check that notification recipients can reach the linked system and have appropriate access.

Choosing the right rendering pattern

Situation Pattern Reason
Every alert in the group has the same destinations Render .CommonAnnotations It presents one shared set of links without repeating them for each alert.
Each alert can point to a different service or context Range over .Alerts and render each .Annotations It keeps each destination associated with the alert that supplies it.
A link is fixed for an alert type Use a static annotation value There is no need to build a dynamic URL when the same destination is useful.
A link needs context such as a service label Template the annotation only with a documented destination URL format Alert context can be included when the target understands the resulting URL.
The notification is lengthy or shared across receivers Load and invoke a named template Reusable content can be managed separately from a short receiver field.
A destination expects a structured payload Use a webhook and transform the notification fields as needed The webhook payload carries annotations, but the receiving system controls how to consume them.

The reliable pattern is to make Prometheus annotations the source of diagnostic destinations and make the Alertmanager receiver responsible for presenting them. The key implementation decision is whether a link is shared by the notification group or belongs to an individual alert. URL construction remains specific to the observability product, and the recipient must be able to access the resulting destination.

Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.

Leave a Reply

Your email address will not be published. Required fields are marked *

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

More from the Wire

  1. World desk4 min
    How to Spot an AI Voice Scam Before Sending MoneyDon’t rely on how a caller sounds. Pause, call back through a known number, and verify the emergency with another trusted person before sending money.
  2. Mountain View desk4 min
    Google’s SynthID Detector: How to Check AI-Generated Images, Video and AudioGoogle’s SynthID Detector looks for an embedded watermark in supported images, video and audio. Here is what its results do—and do not—show.
  3. Redmond desk20 min
    How to create a link to File or Folder in Windows 11Windows 11 gives you several ways to point to a file or folder without moving or duplicating it. You can create a desktop shortcut,…
Recommended PC Tool
Recommended PC Tool
Crashes, No Sound, or Screen Glitches?Free driver scan
Windows Errors? Fix Them Before They SpreadFree repair scan

Two free Windows tools

One Free Minute Could Fix That PC

Before you go - each of these free tools takes about a minute and tackles what quietly slows a Windows PC down.

Special offer. View Outbyte info, uninstall instructions, EULA, and Privacy Policy.