Initiating a failover to your replica cluster

If your GitHub Enterprise Server cluster fails, you can fail over to the replica.

In this article

About failover to your replica cluster

If the data center for your active cluster experiences a failure and you've configured high availability, you can fail over to your replica cluster.

Failing over to your replica cluster promotes it to be your new active cluster, and decouples the new active cluster from the old active cluster. The nodes in your old active cluster are placed in maintenance mode if they are in a healthy enough state for this operation to be performed.

After failover, you will have two standalone clusters without high availability configured. You can reconfigure replication from the new active cluster. For more information, see Configuring high availability replication for a cluster.

Prerequisites

To fail over to replica nodes, you must have configured high availability replication for your cluster. For more information, see Configuring high availability replication for a cluster.

Initiating a failover to your replica cluster

Note

On an instance in a cluster configuration, former primary nodes were able to access the newly promoted nodes after failover. This was fixed in patch release 3.12.2 . For more information, see Release notes.

As a result of this fix, ghe-cluster-failover identifies IPs to block from the old primary cluster and writes them to /data/user/common/cluster-ip-blocklist. After the failover completes, the command runs ghe-cluster-block-ips to block the IPs on the new active cluster.

Additionally, the ghe-cluster-block-ips, ghe-cluster-block-ip, ghe-cluster-unblock-ips, and ghe-cluster-unblock-ip commands were also introduced in these patch releases. With these commands, you can manually control which IPs can access your newly promoted cluster, and avoid the potentially lengthy configuration run associated with running the whole ghe-cluster-failover command. For more information, see Command-line utilities.

  1. SSH into the primary MySQL node in the replica cluster. For more information, see Accessing the administrative shell (SSH).

  2. To begin the failover to the secondary cluster and configure the nodes to respond to requests, run the following command.

    ghe-cluster-failover

  3. After the configuration run finishes, GitHub Enterprise Server displays the following message.

    Finished cluster configuration

  4. Update the DNS record to point to the IP address of the load balancer for your replica cluster. After the TTL period expires, requests will be directed to the replica cluster.

After GitHub Enterprise Server returns you to the prompt and your DNS updates propagate, you've finished failing over. Users can access GitHub Enterprise Server using the usual hostname for your cluster.