ETCD ID Mismatch in Kubernetes

ETCD is the key-value store used by Kubernetes to keep cluster state. Sometimes you may see the following error in ETCD logs:

2025-11-11 12:13:10.134330 E | rafthttp: request cluster ID mismatch (got fc5dec74b3dcf6fa want 4e38d7b9b94fe83c)

It usually happens when a member of the ETCD cluster has a different ID than what the cluster expects.

Causes

  • Restoring a backup incorrectly.
  • Copying ETCD data directories between nodes.
  • Node reinstallation without cleaning old ETCD state.
  • Cluster misconfiguration after scaling or failover.

How to Fix

Step 1: Identify the problem member

etcdctl member list

Look for a member whose ID does not match the expected cluster state.

Step 2: Remove the problematic member

etcdctl member remove <member-ID>

Step 3: Re-add the member correctly

etcdctl member add <member-name> --peer-urls=https://<peer-IP>:2380

Step 4: Restart ETCD service on the affected node.

systemctl restart etcd

⚠️ Be careful: modifying ETCD cluster members can lead to data loss if done incorrectly. Always backup ETCD before making changes.

Best Practices

  • Always use ETCD snapshots and restore properly.
  • Avoid copying ETCD data directories between nodes.
  • Monitor ETCD logs for early detection of ID mismatches.

Following these steps will help recover ETCD from an ID mismatch and restore cluster stability.