Cardano Validator Watcher

Cardano Validator Watcher is a Prometheus exporter designed to help you monitor your Cardano validator nodes. It periodically collects essential metrics, providing valuable insights into the performance and health of your infrastructure. These exported metrics can be used to build monitoring dashboards (e.g with Grafana) and configure alerting rules to stay notified when something goes wrong.

Dependencies

This project use the following dependencies:

• BlockFrost. You need to have a account and a subscription.
• cncli to calculate the slot leaders.
• cardano-cli to query additional data from a RPC node.
• A valid RPC node.
• Download the Genesis configuration files and provide the VRF signing key for each monitored pool.

Usage

The watcher needs access to your Cardano node's socket. Two options are available:

Option A — socat sidecar (watcher has direct socket access)

Run socat alongside the watcher to expose the node socket locally:

# With Kubernetes
kubectl port-forward pod/<POD_NAME> 3002 &
socat UNIX-LISTEN:/tmp/cardano.socket,fork,reuseaddr,unlink-early TCP:127.0.0.1:3002

# Without Kubernetes
socat UNIX-LISTEN:/tmp/cardano.socket,fork,reuseaddr,unlink-early TCP:<IP>:<PORT>

Then configure socket-path in the watcher config.

Option B — Go socket proxy (no socat needed in the watcher pod)

If socat is already running on the node pod (exposing the socket over TCP), the watcher can connect directly without a local socat sidecar. Configure node-host and socat-port and the watcher will manage the proxy internally.

Ensure that you have downloaded the Genesis configuration files. You also need to provide the VRF signing key for each monitored pool.

Then, to start the watcher, execute the following command:

./cardano-validator-watcher [flags]

Flags

Flag	Description	Default Value	Required
--config	Path to the configuration file	config.yml	No
--log-level	Log Level	info	No
--http-server-host	Host on which the HTTP server should listen	127.0.0.1	No
--http-server-port	Port on which the HTTP server should listen	8080	No
--network	Cardano network ID	preprod	Yes
--database-path	Path to the local database mainly used by the Cardano client	watcher.db	No
--cardano-config-dir	Path to the directory where Cardano configuration files are stored	/config	No
--cardano-timezone	Timezone to use with cardano-cli	UTC	No
--cardano-socket-path	Path of the socket to communicate with a Cardano node (Option A)	/var/run/cardano.socket	No
--cardano-node-host	Hostname/IP of the cardano-node service (Option B)		No
--cardano-socat-port	TCP port where socat on the node pod exposes the Unix socket (Option B)		No
--blockfrost-project-id	Blockfrost project ID		Yes
--blockfrost-endpoint	Blockfrost API endpoint		Yes
--blockfrost-max-routines	Number of routines used by Blockfrost to perform concurrent actions	10	No
--blockfrost-timeout	Timeout for requests to the Blockfrost API (in seconds)	60	No
--block-watcher-enabled	Enable block watcher	True	No
--block-watcher-refresh-interval	Interval at which the block watcher collects and processes slots (in seconds)	60	No
--pool-watcher-enabled	Enable pool watcher	True	No
--pool-watcher-refresh-interval	Interval at which the pool watcher collects data on monitored pools (in seconds)	60	No
--network-watcher-enabled	Enable network watcher	True	No
--network-watcher-refresh-interval	Interval at which the network watcher collects data related to network (in seconds)	60	No
--status-watcher-refresh-interval	Interval at which the status watcher collects data related to health (in seconds)	15	No

Configuration

The watcher uses a config.yaml file in the current working directory by default. Use the --config flag to specify a different configuration file.

Full Example

pools:
  - instance: "cardano-producer-pool-0"
    id: "pool1abcd1234efgh5678ijklmnopqrstuvwx"
    name: "pool-0"
    key: "config/pool-0.vrf.skey"
  - instance: "cardano-producer-pool-1"
    id: "pool2abcd1234efgh5678ijklmnopqrstuvwx"
    name: "pool-1"
    key: "config/pool-1.vrf.skey"
    exclude: true
    allow-empty-slots: true
network: "mainnet"
block-watcher:
  enabled: true
  refresh-interval: 30
pool-watcher:
  enabled: true
  refresh-interval: 30
network-watcher:
  enabled: true
  refresh-interval: 30
status-watcher:
  enabled: true
  refresh-interval: 15
database:
  path: "watcher.db"
blockfrost:
  project-id: "thisissecret"
  endpoint: "https://cardano-mainnet.blockfrost.io/api/v0"
  max-routines: 10
  timeout: 60
http:
  host: "0.0.0.0"
  port: 8080
cardano:
  config-dir: "config"
  socket-path: "/tmp/cardano.socket"
  timezone: "UTC"

Pools Settings

Field	Description	Example
instance	Name of the instance	"cardano-producer-pool-0"
id	Pool ID	"pool1abcd1234efgh5678ijklmnopqrstuvwx"
name	Name of the pool	"pool-0"
key	Path to the key file	"config/pool-0.vrf.skey"
exclude	Exclude the pool from monitoring	true
allow-empty-slots	Pools is allowed to not have slot leaders	false

Global Settings

Field	Description	Example
network	Name of the network	"mainnet"

network: "mainnet"

Block Watcher Settings

Field	Description	Example
enabled	Enable block watcher	True
refresh-interval	Time, in seconds, between two consecutive collections of block data	60

block-watcher:
  enabled: true
  refresh-interval: 30

Pool Watcher Settings

Field	Description	Example
enabled	Enable pool watcher	True
refresh-interval	Time, in seconds, between two consecutive collections of pool data	60

pool-watcher:
  enabled: true
  refresh-interval: 30

Status Watcher Settings

Field	Description	Example
refresh-interval	Time, in seconds, between two consecutive collections of status data	60

status-watcher:
  refresh-interval: 30

Network Watcher Settings

Field	Description	Example
enabled	Enable network watcher	True
refresh-interval	Time, in seconds, between two consecutive collections of network data	60

network-watcher:
  enabled: true
  refresh-interval: 30

Database Settings

Field	Description	Example
path	Path to the database file	"watcher.db"

database:
  path: "watcher.db"

Blockfrost Settings

Field	Description	Example
project-id	Blockfrost project ID	"thisissecret"
endpoint	Blockfrost API endpoint	"https://cardano-mainnet.blockfrost.io/api/v0"
max-routines	Number of routines used by Blockfrost to perform concurrent actions	10
timeout	Timeout for requests to the Blockfrost API (in seconds)	60

blockfrost:
  project-id: "thisissecret"
  endpoint: "https://cardano-mainnet.blockfrost.io/api/v0"
  max-routines: 10
  timeout: 60

HTTP Server Settings

Field	Description	Example
port	Port on which the HTTP server should listen	8080

http:
  host: "0.0.0.0"
  port: 8080

Cardano Settings

Field	Description	Example
config-dir	Path to the directory where Cardano configuration files are stored	"config"
timezone	Timezone to use with cardano-cli	"UTC"
socket-path	Path of the socket to communicate with a Cardano node (Option A)	"/tmp/cardano.socket"
node-host	Hostname/IP of the cardano-node service (Option B)	"cardano-node-service"
socat-port	TCP port where socat on the node pod exposes the Unix socket (Option B)	3002

# Option A: socat sidecar in the watcher pod
cardano:
  config-dir: "config"
  timezone: "UTC"
  socket-path: "/tmp/cardano.socket"

# Option B: Go-managed proxy, no socat needed in the watcher pod
cardano:
  config-dir: "config"
  timezone: "UTC"
  node-host: "cardano-node-service"
  socat-port: 3002

Metrics

Metric Name	Description	Type	Labels
cardano_validator_watcher_pool_relay_count	Number of relays associated with each pool	GaugeVec	pool_name, pool_id, pool_instance
cardano_validator_watcher_pool_pledge_met	Indicates whether the pool has met its pledge requirements or not (0 or 1)	GaugeVec	pool_name, pool_id, pool_instance
cardano_validator_watcher_pool_saturation_level	The current saturation level of the pool in percent	GaugeVec	pool_name, pool_id, pool_instance
cardano_validator_watcher_pool_drep_registered	Whether the pool owner is registered to a DRep (0 or 1)	GaugeVec	pool_name, pool_id, pool_instance
cardano_validator_watcher_next_epoch_start_time	Start time of the next epoch in seconds	Gauge	-
cardano_validator_watcher_monitored_validators_count	Number of validators monitored by the watcher	Gauge	-
cardano_validator_watcher_missed_blocks	Number of missed blocks in the current epoch	CounterVec	pool_name, pool_id, pool_instance, epoch
cardano_validator_watcher_consecutive_missed_blocks	Number of blocks missed consecutively.	GaugeVec	pool_name, pool_id, pool_instance, epoch
cardano_validator_watcher_validated_blocks	Number of validated blocks in the current epoch	CounterVec	pool_name, pool_id, pool_instance, epoch
cardano_validator_watcher_orphaned_blocks	Number of orphaned blocks in the current epoch	CounterVec	pool_name, pool_id, pool_instance, epoch
cardano_validator_watcher_expected_blocks	Number of expected blocks in the current epoch	CounterVec	pool_name, pool_id, pool_instance, epoch
cardano_validator_watcher_latest_slot_processed_by_block_watcher	Latest slot processed by block watcher	Gauge	-
cardano_validator_watcher_next_slot_leader	Next slot leader for each monitored pool	GaugeVec	pool_name, pool_id, pool_instance, epoch
cardano_validator_watcher_epoch_duration	Duration of an epoch in days	Gauge	-
cardano_validator_watcher_network_epoch	Current epoch number	Gauge	-
cardano_validator_watcher_network_block_height	Latest known block height	Gauge	-
cardano_validator_watcher_network_slot	Latest known slot	Gauge	-
cardano_validator_watcher_network_epoch_slot	Latest known epoch slot	Gauge	-
cardano_validator_watcher_network_network_pools_total	Total number of pools in the network	Gauge	-
cardano_validator_watcher_network_blocks_proposed_current_epoch	Number of blocks proposed in the current epoch by the network	Gauge	-
cardano_validator_watcher_network_active_stake	Total active stake in the network	Gauge	-
cardano_validator_watcher_chain_id	ID of the chain	Gauge	-
cardano_validator_watcher_health_status	Health status of the Cardano validator watcher: 1 = healthy, 0 = unhealthy	Gauge	-