Infrastructure Monitoring: Prometheus vs InfluxDB vs VictoriaMetrics

Monitoring Is Not Optional

If you cannot measure it, you cannot manage it. Infrastructure monitoring tells you:

• Is the system healthy right now?
• What is the trend over time?
• When should you scale up (or down)?
• What caused the outage at 3 AM?

Prometheus: The Cloud-Native Standard

Prometheus is the CNCF-graduated monitoring system. It uses a pull-based model (scrapes targets) and the powerful PromQL query language.

# docker-compose.yml
services:
  prometheus:
    image: prom/prometheus:v2.54.0
    container_name: prometheus
    command:
      - '--config.file=/etc/prometheus/prometheus.yml'
      - '--storage.tsdb.path=/prometheus'
      - '--storage.tsdb.retention.time=30d'
      - '--web.enable-lifecycle'
    volumes:
      - ./prometheus/prometheus.yml:/etc/prometheus/prometheus.yml
      - prometheus-data:/prometheus
    mem_limit: 512m

# prometheus.yml
global:
  scrape_interval: 15s
  evaluation_interval: 15s

scrape_configs:
  - job_name: 'node'
    static_configs:
      - targets: ['node-exporter:9100']

  - job_name: 'docker'
    static_configs:
      - targets: ['cadvisor:8080']

  - job_name: 'traefik'
    static_configs:
      - targets: ['traefik:8080']
    metrics_path: /metrics

  - job_name: 'postgres'
    static_configs:
      - targets: ['postgres-exporter:9187']

  # Service discovery for Docker containers
  - job_name: 'docker-services'
    docker_sd_configs:
      - host: unix:///var/run/docker.sock
    relabel_configs:
      - source_labels: [__meta_docker_container_label_prometheus_scrape]
        regex: "true"
        action: keep

PromQL examples:

# CPU usage percentage per container
100 - (rate(container_cpu_usage_seconds_total[5m]) * 100)

# Memory usage percentage
container_memory_usage_bytes / container_spec_memory_limit_bytes * 100

# HTTP request rate per service
rate(traefik_service_requests_total[5m])

# 95th percentile latency
histogram_quantile(0.95, rate(http_request_duration_seconds_bucket[5m]))

# Disk usage prediction (when will disk be full?)
predict_linear(node_filesystem_avail_bytes{mountpoint="/"}[1h], 24*3600) < 0

# Alert: High error rate
rate(http_requests_total{status=~"5.."}[5m]) / rate(http_requests_total[5m]) > 0.05

Alerting rules:

# alerts.yml
groups:
  - name: infrastructure
    rules:
      - alert: HighCPU
        expr: 100 - (avg by(instance) (rate(node_cpu_seconds_total{mode="idle"}[5m])) * 100) > 80
        for: 5m
        labels:
          severity: warning
        annotations:
          summary: "High CPU usage on {{ .Labels.instance }}"

      - alert: DiskSpaceLow
        expr: (node_filesystem_avail_bytes{mountpoint="/"} / node_filesystem_size_bytes{mountpoint="/"}) * 100 < 15
        for: 10m
        labels:
          severity: critical
        annotations:
          summary: "Disk space below 15% on {{ .Labels.instance }}"

      - alert: ContainerDown
        expr: absent(container_last_seen{name=~"traefik|postgres|redis"}) == 1
        for: 1m
        labels:
          severity: critical
        annotations:
          summary: "Critical container {{ .Labels.name }} is down"

InfluxDB: The Time-Series Database

InfluxDB is a purpose-built time-series database with its own query language (Flux) and an HTTP write API (push-based).

services:
  influxdb:
    image: influxdb:2.7-alpine
    environment:
      DOCKER_INFLUXDB_INIT_MODE: setup
      DOCKER_INFLUXDB_INIT_USERNAME: admin
      DOCKER_INFLUXDB_INIT_PASSWORD: supersecret
      DOCKER_INFLUXDB_INIT_ORG: techsaas
      DOCKER_INFLUXDB_INIT_BUCKET: metrics
    volumes:
      - influxdb-data:/var/lib/influxdb2
    mem_limit: 512m

  telegraf:
    image: telegraf:1.32-alpine
    volumes:
      - ./telegraf/telegraf.conf:/etc/telegraf/telegraf.conf:ro
      - /var/run/docker.sock:/var/run/docker.sock:ro
    mem_limit: 128m

# telegraf.conf
[agent]
  interval = "10s"
  flush_interval = "10s"

[[outputs.influxdb_v2]]
  urls = ["http://influxdb:8086"]
  token = "your-token"
  organization = "techsaas"
  bucket = "metrics"

[[inputs.cpu]]
  percpu = true
  totalcpu = true

[[inputs.mem]]

[[inputs.disk]]
  ignore_fs = ["tmpfs", "devtmpfs"]

[[inputs.docker]]
  endpoint = "unix:///var/run/docker.sock"
  container_names = []
  timeout = "5s"

[[inputs.net]]

Flux query examples:

// CPU usage over last hour
from(bucket: "metrics")
  |> range(start: -1h)
  |> filter(fn: (r) => r._measurement == "cpu")
  |> filter(fn: (r) => r._field == "usage_idle")
  |> filter(fn: (r) => r.cpu == "cpu-total")
  |> map(fn: (r) => ({r with _value: 100.0 - r._value}))
  |> aggregateWindow(every: 1m, fn: mean)

// Container memory usage
from(bucket: "metrics")
  |> range(start: -6h)
  |> filter(fn: (r) => r._measurement == "docker_container_mem")
  |> filter(fn: (r) => r._field == "usage_percent")
  |> group(columns: ["container_name"])

VictoriaMetrics: The Efficient Alternative

VictoriaMetrics is a Prometheus-compatible time-series database that uses significantly less storage and memory. It accepts PromQL queries and Prometheus remote_write, making it a drop-in replacement.

services:
  victoriametrics:
    image: victoriametrics/victoria-metrics:v1.106.0
    container_name: victoriametrics
    command:
      - '--storageDataPath=/storage'
      - '--retentionPeriod=90d'
      - '--httpListenAddr=:8428'
    volumes:
      - vm-data:/storage
    mem_limit: 256m

  # Prometheus scrapes targets, remote_writes to VictoriaMetrics
  prometheus:
    image: prom/prometheus:v2.54.0
    volumes:
      - ./prometheus/prometheus.yml:/etc/prometheus/prometheus.yml
    mem_limit: 256m

# prometheus.yml with remote_write to VictoriaMetrics
remote_write:
  - url: http://victoriametrics:8428/api/v1/write

VictoriaMetrics also provides vmagent (lightweight scraper) as a Prometheus replacement:

services:
  vmagent:
    image: victoriametrics/vmagent:v1.106.0
    command:
      - '--promscrape.config=/etc/prometheus/prometheus.yml'
      - '--remoteWrite.url=http://victoriametrics:8428/api/v1/write'
    volumes:
      - ./prometheus/prometheus.yml:/etc/prometheus/prometheus.yml
    mem_limit: 64m

Comparison

Our Monitoring Stack Recommendation

For small-to-medium self-hosted infrastructure:

Metrics: VictoriaMetrics (or Prometheus for ecosystem)
Logs: Grafana Loki + Promtail
Uptime: Uptime Kuma
Dashboards: Grafana
Alerting: Grafana Alerting → Ntfy/Slack

Total RAM: ~1GB for full observability of 50+ services.

At TechSaaS, we use Grafana + Loki + Promtail for log monitoring and Uptime Kuma for availability checking. For clients who need metrics monitoring, we deploy VictoriaMetrics with vmagent — it handles the same workload as Prometheus with one-fifth the memory and one-tenth the disk space. The PromQL compatibility means all existing Grafana dashboards work without changes.