monitoring-skill

Monitoring and observability with Prometheus, Grafana, ELK Stack, and distributed tracing.

⭐2stars🔀0forks📁View on GitHub🕐Updated Dec 31, 2025⬇️2downloads

When & Why to Use This Skill

This Claude skill provides a comprehensive framework for mastering monitoring and observability across the modern tech stack. It integrates industry-standard tools like Prometheus for metrics, Grafana for visualization, and the ELK Stack for log management, alongside distributed tracing. Designed for SREs and DevOps professionals, it facilitates the implementation of the three pillars of observability, helping teams maintain system reliability through SLIs/SLOs, advanced alerting, and efficient troubleshooting.

Use Cases

  • Building real-time dashboards in Grafana to monitor SRE golden signals such as latency, traffic, errors, and saturation.
  • Writing complex PromQL queries to analyze service performance and setting up automated alerting rules for proactive incident management.
  • Centralizing system logs using the ELK Stack to perform deep-dive root cause analysis and log correlation during outages.
  • Implementing distributed tracing with OpenTelemetry to map request flows and identify bottlenecks in microservices architectures.
  • Optimizing infrastructure health by managing high cardinality metrics and configuring Prometheus federation for large-scale environments.
namemonitoring-skill
descriptionMonitoring and observability with Prometheus, Grafana, ELK Stack, and distributed tracing.
sasmp_version"1.3.0"
bonded_agent06-monitoring-observability
bond_typePRIMARY_BOND
- nametool
typestring
requiredfalse
enum["prometheus", "grafana", "elk", "jaeger"]
default"prometheus"
strategyexponential_backoff
initial_delay_ms1000
max_retries3
loggingstructured
metricsenabled

Monitoring & Observability Skill

Overview

Master the three pillars of observability: metrics, logs, and traces.

Parameters

Name Type Required Default Description
pillar string No all Observability pillar
tool string No prometheus Tool focus

Core Topics

MANDATORY

  • Prometheus metrics and PromQL
  • Grafana dashboards
  • ELK Stack basics
  • SLIs, SLOs, error budgets
  • Alerting rules

OPTIONAL

  • Distributed tracing
  • OpenTelemetry
  • Custom exporters
  • Log correlation

ADVANCED

  • High cardinality handling
  • Recording rules
  • Federation
  • Continuous profiling

Quick Reference

# PromQL
sum(rate(http_requests_total[5m])) by (service)
histogram_quantile(0.99, rate(http_request_duration_seconds_bucket[5m]))
100 * sum(rate(http_requests_total{status=~"5.."}[5m])) / sum(rate(http_requests_total[5m]))

# Prometheus API
curl http://localhost:9090/api/v1/targets
curl 'http://localhost:9090/api/v1/query?query=up'
curl -X POST http://localhost:9090/-/reload

# Alertmanager
amtool silence add alertname="HighLatency" --duration=2h
amtool alert

SRE Golden Signals

Signal Metric
Latency histogram_quantile(0.99, ...)
Traffic sum(rate(requests_total[5m]))
Errors rate(errors_total[5m])
Saturation node_memory_MemAvailable_bytes

Troubleshooting

Common Failures

Symptom Root Cause Solution
No data Scrape failing Check targets page
Alert not firing PromQL error Test in UI
High cardinality Too many labels Reduce labels
Slow queries Too much data Add aggregation

Debug Checklist

  1. Check targets: /targets
  2. Test query in UI
  3. Check logs: journalctl -u prometheus
  4. Verify time sync (NTP)

Recovery Procedures

Prometheus OOM

  1. Check cardinality
  2. Reduce retention
  3. Add federation

Resources