Scrape selective metrics in Prometheus

Prometheus provides the ability to filter specific metrics post-scraping, via the metrics_relabel_config stanza.

This is useful in reducing the number of metrics before they are consumed and sent to a remote write long term storage.

We will use the node_exporter Prometheus exporter as an example, and send only metrics matching the regex node_cpu.*.

Prometheus scrape config without filtering

global:
  scrape_interval: 1m

scrape_configs:
  - job_name: 'node-exporter-01'
    static_configs:
      - targets: [ 'localhost:9100' ]

This scrapes and stores all node exporter metrics.

Prometheus scrape config with filtering

global:
  scrape_interval: 1m

scrape_configs:
  - job_name: 'node-exporter-01'
    static_configs:
      - targets: [ 'localhost:9100' ]
    metric_relabel_configs:
    - source_labels: [__name__]
      action: keep
      regex: '(node_cpu)'

This will scrape all metrics, but drop anything that does not match the entries in the regex section.