Skip to content

ClickHouse Backend

An alternative to InfluxDB for the Pro edition. InfluxDB stays the default; ClickHouse is fully wired but less traveled.

sudo smoking-pi install --edition pro --database clickhouse

The backend is fixed at install (TSDB_TYPE); smoking-pi up and every other command then add the ClickHouse overlay and profile by themselves.

How the pieces fit

  • Schema — smokeping.latency, one row per RRD sample, created by the exporter (see below).
  • Exporter — rrd2clickhouse.py reads the SmokePing RRDs and inserts rows.
  • Datasource — the official grafana-clickhouse-datasource plugin, baked into the Grafana image at build time.
  • Dashboards — six ClickHouse variants under provisioning/dashboards-clickhouse/, provisioned only in ClickHouse mode: Individual Pings, DNS Resolvers, HTTP by Version and the three side-by-side comparisons. They are the per-target views; the Overview, Target Detail, Connectivity & Quality, Wi-Fi Link, CPE Microcut Detection, DNS Wizard and Diagnostics dashboards read InfluxDB and have no ClickHouse version (decision of 2026-10-15: no known ClickHouse install asks for them).

Things that are not obvious

The schema is created by the exporter, not the init hook

shared/modules/clickhouse/init/ is mounted at /docker-entrypoint-initdb.d, but it never runs. Docker seeds a fresh named volume from the image, and the official ClickHouse image ships a populated /var/lib/clickhouse; its entrypoint sees "Database directory appears to contain a database" and skips the init hook entirely. Nothing in the logs marks this as a problem — you simply get no smokeping database.

The exporter therefore applies CREATE DATABASE / TABLE IF NOT EXISTS on connect. That is idempotent and works whether or not the hook ever fires. The SQL files are kept as documentation of the intended schema.

The client must select the database after creating it

Because the schema may not exist yet, the exporter connects without a database (naming a missing one fails the connection outright), creates smokeping and smokeping.latency, and only then sets the client's default database. Skip that last step and every unqualified insert goes to default.latency, which does not exist — the log fills with Table default.latency does not exist. Maybe you meant smokeping.latency? while the schema check above looks fine. This is exactly what happened between the Sprint 13 revival and 2026-09-19: the fix that let the exporter create the schema also stopped it from ever writing to it.

The related trap: ClickHouse rejects a whole batch for one column the table does not declare (Unrecognized column 'rrd_file'). The columns the exporter writes are listed once, in INSERT_COLUMNS, and a test checks every one of them against the CREATE TABLE the exporter runs.

The datasource takes host/port, not url

The official plugin reads host and port from jsonData and ignores the top-level url field that most Grafana datasources use. Configure only url and its health check fails with [config] invalid server host. Either empty or not set — which reads like a networking problem and is not one.

ClickHouse mode builds its own provisioning tree

editions/pro bind-mounts provisioning/ read-only, and Docker cannot create a mountpoint for a file inside a read-only bind mount — so neither the entrypoint nor a compose overlay can drop a provider file into the scanned directory. In ClickHouse mode the entrypoint instead builds a writable tree at /var/lib/grafana/provisioning-active containing only the ClickHouse datasource and the ClickHouse dashboard provider, and points GF_PATHS_PROVISIONING at it. The InfluxDB dashboards are deliberately absent — they query a datasource that is not configured in this mode.

Loss is a percent, and the denominator comes from the RRD

packet_loss is 0–100. The RRD loss data source is a count of lost pings, and probes disagree on how many they send (FPing 10, DNS 5), so the exporter reads the count of ping1..pingN data sources from each RRD and divides by that. This matches rrd2influx.py, which stores the same quantity as a 0–1 ratio.

category and measurement_type use the same vocabulary as the InfluxDB exporter — topsites / netflix / dns / custom, and latency / dns_latency — not the raw RRD directory names. Dashboard predicates must match that vocabulary; filtering on category = 'DNS_Resolvers' returns nothing.

Not exported to ClickHouse: the InfluxDB exporter's target_host measurement, which records the host behind each target over time (the Netflix slots change appliance every night). In ClickHouse mode only the current host is known, from the target's row in the config.

Nor is target_labels, the service label of each target over time (Service taxonomy): ClickHouse mode has no labels in the time-series database, and its dashboards have no Service category variable. The labels themselves are config-manager's and are the same in either mode (the web admin, smoking-pi taxonomy and the assistants show them). TODO: a target_labels table written on change, if a ClickHouse install asks for it.

Verifying a deployment

# schema present
docker compose exec clickhouse clickhouse-client \
  --user "$CLICKHOUSE_USER" --password "$CLICKHOUSE_PASSWORD" \
  --query "SHOW TABLES FROM smokeping"

# rows arriving
docker compose exec clickhouse clickhouse-client \
  --user "$CLICKHOUSE_USER" --password "$CLICKHOUSE_PASSWORD" \
  --query "SELECT count(), max(timestamp) FROM smokeping.latency"

# datasource health (expects "Data source is working")
curl -s -u "admin:$GF_SECURITY_ADMIN_PASSWORD" \
  http://localhost:3000/api/datasources/uid/clickhouse/health

Grafana's boot log should show:

grafana-entrypoint: building ClickHouse provisioning tree in /var/lib/grafana/provisioning-active
grafana-entrypoint: ClickHouse dashboard provider activated
grafana-entrypoint: datasources provisioned from /var/lib/grafana/provisioning-active

Switching back to InfluxDB

Set TSDB_TYPE=influxdb and start with the influxdb profile. The ClickHouse provisioning tree is rebuilt only in ClickHouse mode, so nothing needs cleaning up. Data does not migrate between backends.