Guide Running Horcrux MPC for Your Nodes

Horcrux is a multi-party-computation (MPC) signing service for CometBFT (Formerly known as Tendermint) nodes

· cosmos MPC security

In this article/blog we will explain how to run horcrux for your validator according to our experience as well as the introduction of Horcrux for you.

Horcrux

Horcrux is a multi-party-computation (MPC) signing service for CometBFT (Formerly known as Tendermint) nodes.

Why use Horcrux?

Take your validator infrastructure to the next level of security and availability

  • Composed of a cluster of signer nodes in place of the remote signer, enabling High Availability (HA) for block signing through fault tolerance.
  • Secure your validator private key by splitting it across multiple private signer nodes using threshold Ed25519 signatures
  • Add security and availability without sacrificing block sign performance.

Read more detail : https://github.com/strangelove-ventures/horcrux

Single Validator Migration Guide to Horcrux

This is referenced from https://github.com/strangelove-ventures/horcrux/blob/main/docs/migrating.md and then summarized and adopted as we went along.

Using v3.2.2

Please check the latest version when we publish this article https://github.com/strangelove-ventures/horcrux/releases/tag/v3.2.2

We ran this migration example on Bitcanna testnet with 2 sentry and 3 cosigners. please refer to the explanation and preparation below: 👇

Preparation

  1. We assume that you have the bitcanna testnet validator running on 1 VM.
  2. 2 VM for sentry example with specs of 4 CPUs, 16GB RAM, 500GB SSD.
    • These chain daemons should only expose the :26656 (p2p) port to the open internet
    • The daemons will need to expose :1234 (priv validator port) to the horcrux nodes, but not to the open internet
  3. 3 VM for cosigners example with 1 CPU, 1 GB RAM, 20 GB SSD storage running horcrux.
    • cosigner must be connected to all sentry and other cosigners.

Steps for migration

Record the IP or DNS of the signer and sentry

example:

sentry-1: 12.234.56.1
sentry-2: 12.234.56.2

cosigner-1: 12.234.56.7
cosigner-2: 12.234.56.8
cosigner-3: 12.234.56.9

Setup full node

Run a full node for chain bitcanna-dev-1 on each sentry. if u need guide : https://genznodes.dev/resources/install/testnet-bitcanna

  • allow incoming for cosigners:
sudo ufw allow from 12.234.56.7
sudo ufw allow from 12.234.56.8
sudo ufw allow from 12.234.56.9

Prepare horcrux on cosigner

  • enter ssh cosigner and create users example:
sudo adduser cosigner-1
sudo usermod -aG sudo cosigner-1
  • close ssh and enter with new user that we just created
  • install Horcrux
TAG=v3.2.2
curl -L https://github.com/strangelove-ventures/horcrux/releases/download/${TAG}/horcrux_linux-amd64 > horcrux && \
chmod +x ./horcrux &&\
sudo mv ./horcrux /usr/bin/horcrux

check Horcrux

horcrux version

output

{
  "version": "3.2.2",
  "commit": "4fe383e7310f12aa3b2bd4365d37c048419b3912",
  "go_version": "go1.21.4 linux/amd64",
  "cosmos_sdk_version": "v0.50.1",
  "cometbft_version": "v0.38.0"
}
  • Create .horcrux folder
mkdir -p $HOME/.horcrux/state
  • Create service file

enable linger for user

sudo loginctl enable-linger $USER
mkdir $HOME/.config/systemd/user -p
tee $HOME/.config/systemd/user/horcrux.service > /dev/null <<EOF
[Unit]
Description=MPC Signer node
After=network.target

[Service]
Type=simple
ExecStart=$(which horcrux) start
Restart=on-failure
RestartSec=3
LimitNOFILE=4096

[Install]
WantedBy=default.target
EOF

enable service file

systemctl --user daemon-reload
systemctl --user enable horcrux
  • Repeat the above steps and match the cosigner index , example for cosigner-2 user with cosigner-2.

Create config in local machine

horcrux config init --node "tcp://12.234.56.1:1234" --node "tcp://12.234.56.2:1234" --cosigner "tcp://12.234.56.7:2222" --cosigner "tcp://12.234.56.8:2222" --cosigner "tcp://12.234.56.9:2222" --threshold 2 --grpc-timeout 1000ms --raft-timeout 1000ms

Generate cosigner communication encryption keys

horcrux create-ecies-shards --shards 3

this will be create folder

cosigner_1  cosigner_2  cosigner_3

./cosigner_1:
ecies_keys.json

./cosigner_2:
ecies_keys.json

./cosigner_3:
ecies_keys.json

Shard priv_validator_key.json for each chain

CAUTION: The security of any key material is outside the scope of this guide. The suggested procedure here is not necessarily the one you will use. We aim to make this guide easy to understand, not necessarily the most secure. This guide assumes that your local machine is a trusted computer. The tooling here is all written in go and can be compiled and used in an airgapped setup if needed.

This step requires your priv_validator_key.json.

horcrux create-ed25519-shards --chain-id bitcanna-dev-1 --key-file ./priv_validator_key.json --threshold 2 --shards 3
cosigner_1  cosigner_2  cosigner_3

./cosigner_1:
bitcanna-dev-1_shard.json  ecies_keys.json

./cosigner_2:
bitcanna-dev-1_shard.json  ecies_keys.json

./cosigner_3:
bitcanna-dev-1_shard.json  ecies_keys.json

Distribute config file and key shards to each cosigner.

At the end of this step, each of your horcrux nodes should have a ~/.horcrux/{chain-id}shard.json file for each chain-id with the contents matching the appropriate cosigner{id}/{chain-id}shard.json file corresponding to the node number. Additionally, each of your horcrux nodes should have a ~/.horcrux/ecies_keys.json file with the contents matching the appropriate cosigner{id}/ecies_keys.json file corresponding to the node number.

Halt your validator and supply signer state data horcrux nodes.

  • stop your validator node we’re going to skip a few blocks here.
  • we will need the contents of the ~/.bcna/data/priv_validator_state.json file.

example:

{
  "height": "5398498",
  "round": 0,
  "step": 3,
  "signature": "IEOS7EJ8C6ZZxwwXiGeMhoO8mwtgTiq6VPR/F1cpLZuz0ZvUZdsgQjTt0GniAIgosfEjC5izKw4Nvvs3ZIceAw==",
  "signbytes": "6B080211BA8305000000000022480A205D4E1F722F53A3FD9E0D28639D7CE7B588338570EBA5C340687C30609C47BCA41224080112208283B6E16BEA46797F8AD4EE0ACE424AC7A4827202446B2D56E7F4438541B7BD2A0C08E4ACE28B0610CCD0AC830232066A756E6F2D31"
}

modify the file, pay attention to “round”.

{
    "height": "5398498",
    "round": "0",
    "step": 3
}

And put this file into each cosigner precisely in ~/.horcrux/state/bitcanna-dev-1_priv_validator_state.json.

Start Horcrux on each cosigner

systemctl --user start horcrux
journalctl --user -fu horcrux -o cat

Configure sentry

Once the signer cluster has started successfully its time to reconfigure and restart your sentry nodes. On each node enable the priv validator listener and verify config changes with the following commands:

sed -i 's#priv_validator_laddr = ""#priv_validator_laddr = "tcp://0.0.0.0:1234"#g' ~/.bcna/config/config.toml
cat ~/.bcna/config/config.toml | grep priv_validator_laddr

priv_validator_laddr = “tcp://0.0.0.0:1234”

  • Allow incoming for each consigner
sudo ufw allow from 12.234.56.7
sudo ufw allow from 12.234.56.8
sudo ufw allow from 12.234.56.9
sudo ufw reload
  • restart node sentry
systemctl --user restart bcnad
journalctl --user -fu bcnad -o cat

CONGRATS

you have successfully run the horcrux for your validator, please note that we are not responsible for what happens to your node. Also make sure you understand the steps we provide.

reference: