Sei Node Operations
This guide covers the detailed operational aspects of running a Sei node, including configuration management, maintenance procedures, and best practices for stable and performant operations.
Configuration Management
Directory Structure
The Sei node configuration is stored in ~/.sei/config/
:
~/.sei/config/
├── app.toml # Application configuration (gas fees, API settings, pruning, etc.)
├── client.toml # CLI and client-related settings
├── config.toml # Core Tendermint settings (network, consensus, and RPC)
├── genesis.json # Chain genesis file, defines initial state
├── node_key.json # Unique node identity key for peer-to-peer (P2P) networking
└── priv_validator_key.json # Validator private signing key (if running as a validator)
Essential Configuration Parameters
Network Settings (config.toml)
[p2p]
# Public IP for other nodes to reach you
external_address = "your-public-ip:26656"
# Local address to listen for incoming P2P connections
laddr = "tcp://0.0.0.0:26656"
# Number of peers allowed
max_num_inbound_peers = 40
max_num_outbound_peers = 20
# Network bandwidth limits to prevent congestion
send_rate = 20480000 # 20MB/s
recv_rate = 20480000 # 20MB/s
[rpc]
# RPC listen address
laddr = "tcp://0.0.0.0:26657"
# Maximum number of simultaneous connections
max_open_connections = 900
# Transaction confirmation timeout
timeout_broadcast_tx_commit = "10s"
Application Settings (app.toml)
# Minimum gas prices [to prevent spam transactions]
minimum-gas-prices = "0.01usei"
[api]
# Enable the API server
enable = true
max-open-connections = 1000
[state-commit]
# Use SeiDB for improved performance
sc-enable = true
[state-store]
# Enable state store for historical queries
ss-enable = true
# Retain 100,000 blocks for queryability
# 0 = "keep all"
ss-keep-recent = 100000
Database Management
Database Types
Sei supports two database backends:
-
SeiDB (Recommended)
- Optimized for performance and sync times
- Reduces resource usage
- Best for all nodes
-
Legacy IAVL DB
- Standard Cosmos SDK database
- More widely tested
Sei-DB Configuration
[state-commit]
sc-enable = true
sc-async-commit-buffer = 100
sc-keep-recent = 1 # Keep only the most recent state for performance
sc-snapshot-interval = 10000 # Take state snapshots every 10,000 blocks
[state-store]
ss-enable = true
ss-backend = "pebbledb" # Default, required
ss-async-write-buffer = 100
ss-keep-recent = 100000 # Keep last 100,000 blocks
ss-prune-interval = 600 # Cleanup interval for pruning
Setting very small [more frequent] pruning intervals may cause issues with automated snapshotting in the event those events collide. Too large [less frequent] pruning intervals means it will take a longer overall time to prune which may cause missed blocks and excessive resync time.
Database Maintenance
The database is typically stable and can be left alone, although some attention may be required:
# Check database size
du -sh ~/.sei/data/
# Confirm correct file paths as per your own setup before proceeding
# Before performing a database wipe, ensure you back up essential files:
# - `node_key.json`: Preserves node identity to maintain peer connections.
# - `priv_validator_key.json`: Critical for validators; losing this key results in double signing risks.
# - `config.toml` & `app.toml`: Retains node-specific configuration.
# - `genesis.json`: Required for correct chain initialization.
# Compact database to optimize storage (only if needed)
find ~/.sei/data/ -mindepth 1 ! -name 'priv_validator_state.json' -delete && rm -rf ~/.sei/wasm
# Backup database
cp -r ~/.sei/data/ ~/sei-backup-$(date +%Y%m%d)/
Service Management
Systemd Commands
# Check service status
systemctl status seid
# Start service
systemctl start seid
# Stop service
systemctl stop seid
# Restart service
systemctl restart seid
# View logs in real time
journalctl -fu seid -o cat
Log Management
Prevent logs from consuming excessive disk space by enabling rotation:
sudo tee /etc/logrotate.d/sei > /dev/null << EOF
/var/log/sei/*.log {
daily
rotate 14
compress
delaycompress
notifempty
create 0640 sei sei
sharedscripts
postrotate
systemctl reload seid
endscript
}
EOF
Update Procedures
Minor Updates
For minor updates that are non-consensus-breaking:
# Stop node
sudo systemctl stop seid
# Update binary
cd sei-chain
git fetch --all
git checkout [new-version]
make install
# Restart node
sudo systemctl restart seid
Major Updates
For major upgrades that introduce state-breaking changes:
- Wait for the designated upgrade block height [this can be seen in the upgrade proposal under 'plan']
- The node will halt automatically.
- Update/replace the binary
- Restart the node.
# After node halts
cd sei-chain
git pull
git checkout [new-version]
make install
sudo systemctl restart seid
Tip - build the upgrade before the halt-height so you can quickly replace it with minimal downtime
Performance Optimization
Performance optimizations can yield different results depending on your system's hardware, workload, and network conditions. Before implementing any changes, research and test them in a controlled environment to ensure they align with your specific configuration and requirements. Always back up important data before making modifications.
Memory Management (sysctl tuning)
Optimizing memory management settings can help improve performance and stability, particularly for high-load nodes. These settings control swap usage and the handling of dirty (unwritten) pages in RAM.
vm.swappiness = 1 # Reduce swapping to disk, ensuring RAM is used efficiently before relying on slower swap space
vm.dirty_background_ratio = 3 # The percentage of system memory that can be filled with "dirty" pages before background writing begins
vm.dirty_ratio = 10 # The maximum percentage of system memory that can be filled with "dirty" pages before a full flush is triggered
vm.dirty_expire_centisecs = 300 # Time (in hundredths of a second) before dirty data is written to disk
vm.dirty_writeback_centisecs = 100 # Frequency (in hundredths of a second) at which the system writes "dirty" pages to disk
Network Stack Optimization
Tuning the network stack can enhance packet processing efficiency and throughput, particularly for nodes handling a large number of peers and high transaction volume.
net.core.somaxconn = 32768 # max connections in queued
net.core.netdev_max_backlog = 32768 # packets allowed in queue before dropping incoming packets
net.ipv4.tcp_max_syn_backlog = 16384 # outstanding SYN requests (half-open connections) allowed before dropping new connections
net.core.rmem_max = 16777216 # receive buffer size for network sockets
net.core.wmem_max = 16777216 # send buffer size for network sockets
Storage Optimization
Optimizing storage settings can significantly reduce write latency and improve database performance, especially for nodes using NVMe SSDs.
echo "none" > /sys/block/nvme0n1/queue/scheduler # disable I/O scheduler to reduce latency
blockdev --setra 4096 /dev/nvme0n1 # readahead value to optimize sequential reads
Backup and Recovery
Regular Backups
Automate backups to avoid data loss:
#!/bin/bash
BACKUP_DIR="/backup/sei"
DATE=$(date +%Y%m%d)
# Stop service
systemctl stop seid
# Create backup
tar czf $BACKUP_DIR/sei-backup-$DATE.tar.gz ~/.sei/
# Restart service
systemctl start seid
Recovery Procedure
Restoring from backup in case of corruption or accidental deletion:
# Stop node
systemctl stop seid
# Remove corrupted data
rm -rf ~/.sei/data/
# Restore from backup
tar xzf sei-backup-[date].tar.gz -C /
# Restart node
systemctl start seid
Troubleshooting
Common Issues and Fixes
-
Sync Problems
- Check available disk space (
df -h
) - Ensure proper peer connections (
seid net info
) - Verify firewall settings (port 26656 open)
- Check available disk space (
-
Performance Issues
- Monitor system resources (
htop
oriotop
) - Check disk I/O performance (
iostat
) - Analyze network traffic (
iftop
)
- Monitor system resources (
-
Database Issues
-
Run database integrity checks using:
seid debug dump-db | grep -i error
If errors are detected, consider restoring from a recent backup.
-
Consider pruning excessive historical data by adjusting
ss-keep-recent
inapp.toml
or running:seid unsafe-reset-all --home=$HOME/.sei --keep-addr-book
Alternatively, manually remove old state snapshots to free up space:
rm -rf ~/.sei/data/snapshots/*
-
Diagnostic Commands
# Check sync status
seid status | jq .SyncInfo
# View peer count
seid net info | jq '.n_peers'
# Check validator status
seid query staking validator $(seid keys show -a $VALIDATOR_KEY)
# View recent blocks
seid query blocks recent
Security Considerations
- Use firewalls and rate-limiting to prevent attacks
- Keep your system and node software updated
- Secure SSH access with key-based authentication
- Protect validator keys with offline storage or hardware security modules (HSMs)
For more in-depth system and configuration guidelines, refer to the [Advanced Configuration and Monitoring Guide](./advanced-config-monitoring).