Validator Operations Guide

This comprehensive guide explains how to operate a Sei validator node. We'll cover the complete lifecycle of a validator, from initial setup through ongoing operations and maintenance. Understanding these concepts is crucial for maintaining a reliable and secure validator operation.

Understanding Validator Responsibilities

A validator in the Sei network serves several critical functions. As a validator, you are responsible for:

Participating in consensus by proposing and validating blocks
Maintaining high uptime and performance to avoid slashing
Providing accurate oracle price feeds for asset pricing
Managing delegator relationships and maintaining transparent operations
Participating in governance and network upgrades

Initial Setup

Key Management

The security of your validator begins with proper key management. Your validator requires several distinct keys:

# Validator consensus key - Used for signing blocks
seid tendermint show-validator
 
# Operator key - Used for managing validator operations
seid keys add operator
 
# Oracle key - Used for price feed submissions
seid keys add oracle

These keys serve different purposes and should be managed with appropriate security measures. The consensus key, stored in priv_validator_key.json, is particularly critical as it's used to sign blocks.

Hardware Security Module (HSM) Integration

For production validators, using an HSM is strongly recommended. Here's how to configure an HSM with your validator:

HSM Configuration Steps

# Install required libraries
sudo apt-get install opensc pkcs11-utils
 
# Configure YubiHSM2
yubihsm-connector -d
 
# Generate key in HSM
yubihsm-shell
 
# Configure seid to use HSM
tee "$HOME/.sei/config/priv_validator_config.json" << EOF
{
    "chain_id": "sei-chain",
    "key_type": "yubihsm",
    "state_file": "$HOME/.sei/data/priv_validator_state.json",
    "hsm_serial": "YOUR_HSM_SERIAL",
    "hsm_key_id": "YOUR_KEY_ID"
}
EOF

Validator Registration

Before registering your validator, ensure your node is fully synced with the network. Then create your validator:

seid tx staking create-validator \
    --amount=1000000usei \
    --pubkey=$(seid tendermint show-validator) \
    --moniker="choose_moniker" \
    --chain-id=sei-chain \
    --commission-rate="0.10" \
    --commission-max-rate="0.20" \
    --commission-max-change-rate="0.01" \
    --min-self-delegation="1" \
    --gas="auto" \
    --gas-adjustment="1.5" \
    --gas-prices="0.01usei" \
    --from=operator

The commission parameters deserve careful consideration:

commission-rate: Your initial commission rate
commission-max-rate: An upper limit that can never be exceeded
commission-max-change-rate: Maximum daily commission change

Oracle Price Feeder Setup

As a Sei validator, you must run a price feeder to provide oracle data. This is crucial for network operations.

First, install the price feeder:

cd sei-chain
make install-price-feeder

Create a configuration file for your price feeder:

Price Feeder Configuration

gas_adjustment = 1.5
gas_prices = "0.01usei"
enable_server = true
enable_voter = true
provider_timeout = "500ms"
 
[server]
listen_addr = "0.0.0.0:7171"
read_timeout = "20s"
verbose_cors = true
write_timeout = "20s"
 
[keyring]
backend = "file"
dir = "/home/sei/.sei"
 
[rpc]
grpc_endpoint = "localhost:9090"
rpc_timeout = "500ms"
tmrpc_endpoint = "http://localhost:26657"
 
[telemetry]
enable_hostname = true
enable_hostname_label = true
enable_service_label = true
prometheus_retention = 120
service_name = "price-feeder"
 
[[provider_endpoints]]
name = "binance"
rest = "https://api1.binance.com"
websocket = "stream.binance.com:9443"

Start the price feeder as a service:

sudo tee /etc/systemd/system/price-feeder.service << EOF
[Unit]
Description=Sei Price Feeder
After=network-online.target
 
[Service]
User=$USER
ExecStart=$(which price-feeder) /path/to/config.toml
Restart=always
RestartSec=3
LimitNOFILE=65535
 
[Install]
WantedBy=multi-user.target
EOF
 
sudo systemctl daemon-reload
sudo systemctl enable price-feeder
sudo systemctl start price-feeder

Monitoring and Alerting

Validator-Specific Metrics

Beyond basic node monitoring, validators should track additional metrics:

# Check validator status
seid query staking validator $(seid keys show -a $VALIDATOR_KEY)
 
# Monitor signing status
seid query slashing signing-info $(seid tendermint show-validator)
 
# Check current delegations
seid query staking delegations-to $(seid keys show -a $VALIDATOR_KEY)

Alert Configuration

Set up critical alerts for validator operations. Here are essential Prometheus alert rules:

Validator Alert Rules

groups:
  - name: validator_alerts
    rules:
      - alert: ValidatorMissedBlocks
        expr: increase(tendermint_consensus_validator_missed_blocks[1h]) > 0
        labels:
          severity: critical
        annotations:
          summary: 'Validator missing blocks'
 
      - alert: ValidatorJailed
        expr: tendermint_consensus_validator_status == 0
        labels:
          severity: critical
        annotations:
          summary: 'Validator has been jailed'
 
      - alert: OraclePriceFeedDelay
        expr: time() - sei_oracle_price_timestamp > 300
        labels:
          severity: critical
        annotations:
          summary: 'Oracle price feed delayed'

Security Practices

Network Security

Implement a sentry node architecture to protect your validator:

# Validator node config.toml
[p2p]
pex = false
persistent_peers = "sentry_node_id@sentry_node_ip:26656"
private_peer_ids = ""
addr_book_strict = false
 
# Sentry node config.toml
[p2p]
pex = true
private_peer_ids = "validator_node_id"
addr_book_strict = true

Key Management Practices

Implement secure key backup procedures:

Key Backup Script

#!/bin/bash
# Create encrypted backup of validator keys
BACKUP_DIR="/secure/validator/backup"
DATE=$(date +%Y%m%d)
 
# Backup validator key
tar czf - $HOME/.sei/config/priv_validator_key.json | \
    gpg --symmetric --cipher-algo AES256 \
    -o $BACKUP_DIR/validator_key_$DATE.tar.gz.gpg
 
# Backup keyring
tar czf - $HOME/.sei/keyring-file | \
    gpg --symmetric --cipher-algo AES256 \
    -o $BACKUP_DIR/keyring_$DATE.tar.gz.gpg
 
# Create SHA256 checksums
sha256sum $BACKUP_DIR/*.gpg > $BACKUP_DIR/checksums_$DATE.txt

Maintenance Procedures

Planned Maintenance

When performing planned maintenance:

# Notify delegators (recommended at least 24h in advance)
# Consider posting to:
# - Chain governance forum
# - Social media channels
# - Validator website
 
# Gracefully stop the validator
sudo systemctl stop seid
 
# Perform maintenance tasks
 
# Restart services
sudo systemctl start seid
sudo systemctl start price-feeder

Emergency Procedures

Create an emergency response plan:

Emergency Response Procedures

# 1. If double-signing is detected:
sudo systemctl stop seid
# Investigate priv_validator_state.json
# Contact team and delegators
 
# 2. If node is stuck:
seid status
# Check for consensus failures
journalctl -u seid -n 100
# Attempt safe restart
sudo systemctl restart seid
 
# 3. If oracle feed fails:
systemctl status price-feeder
# Check price-feeder logs
journalctl -u price-feeder -n 100
# Restart if necessary
sudo systemctl restart price-feeder

Governance Participation

As a validator, you have a responsibility to participate in governance. Monitor and vote on proposals:

# List active proposals
seid query gov proposals --status voting_period
 
# Vote on a proposal
seid tx gov vote 1 yes \
    --from operator \
    --chain-id sei-chain \
    --gas auto \
    --gas-prices 0.01usei

Validator Economics

Understanding validator economics is crucial for long-term success:

Commission Rate Strategy: Set competitive rates while ensuring operational sustainability
Delegation Management: Maintain good delegator relationships through transparent communication
Reward Distribution: Rewards are distributed in real-time as blocks are produced
Slashing Risks: Understand and mitigate risks of slashing through proper operation

Recovery Procedures

Validator Recovery

If you need to recover your validator on a new machine:

# 1. Set up new machine with Sei node
# 2. Copy secured backup files
# 3. Restore validator key
gpg -d validator_key_backup.tar.gz.gpg | tar xzf -
# 4. Restore keyring
gpg -d keyring_backup.tar.gz.gpg | tar xzf -
# 5. Start services
sudo systemctl start seid
sudo systemctl start price-feeder

This guide provides a foundation for operating a Sei validator. Remember that validator operation requires constant attention to security, performance, and network participation. Stay engaged with the Sei community and keep updated with network developments.

Technical Reference