Help us improve your experience.

Let us know what you think.

Do you have time for a two-minute survey?

 
ON THIS PAGE
 

Back Up and Restore Contrail Etcd

SUMMARY Learn how to back up and restore the Contrail etcd database.

In release 22.1, Contrail stores its data in the main OpenShift etcd database. When you back up and restore the main etcd database in release 22.1, you implicitly back up and restore Contrail data.

Starting in release 22.2, Contrail stores its data in its own etcd database. Use the procedures in this section to back up and restore the Contrail etcd database.

Back Up the Contrail Etcd Database in Release 22.2

Use this example procedure in release 22.2 to back up the Contrail etcd database. In release 22.2, you run etcdctl commands in the contrail-etcd pods themselves.

  1. Get a list of the contrail-etcd pods.
    Take note of the contrail-etcd pod names and IP addresses. You will refer to these names and IP addresses in the next step.
  2. Back up the etcd database.
    1. Get a shell into one of the contrail-etcd pods.
      For example:where contrail-etcd-xxx is the etcd pod that you want to get a shell into.
    2. Back up the etcd database.
      This example saves the database to /tmp/etcdbackup.db.where <etcd-pod-ip> is the IP address of the pod and the <etcd-port> is the port that etcd is listening on (by default, 12379).
  3. Copy the database to a safe location.
    For example:where contrail-etcd-xxx is the etcd pod where you backed up the database.

Back Up the Contrail Etcd Database in Release 22.3

Use this example procedure in release 22.3 to back up the Contrail etcd database. In release 22.3, you run etcdctl commands on the control plane nodes.

  1. Install etcdctl on all control plane nodes.
    1. Log in to one of the control plane nodes.
      For example:
    2. Download etcd. This example downloads to the /tmp directory.
    3. Untar and move the etcd executable to a directory in your path (for example /usr/local/bin).
    4. Check that you've installed etcd.
    5. Repeat on all the control plane nodes.
  2. Get a list of the contrail-etcd pods.
    Take note of the contrail-etcd pod names, the IP addresses, and the nodes they're running on. You'll need this information in the next few steps.
  3. Copy the etcd certificate and key files from the pods to the control plane nodes.
    We run kubectl on the control plane nodes in this step. We assume you've set up kubeconfig on the control plane nodes in its default location (~/.kube/config).
    1. Pick a contrail-etcd pod (for example, contrail-etcd-0) and log in to the control plane node that's hosting that pod.
    2. Copy the certificate and key files from that contrail-etcd pod to the hosting control plane node.
      In this example, we're copying the certificates and key files from the contrail-etcd-0 pod to local files on this node.This copies the certificate and key files from the contrail-etcd-0 pod to ca.crt, tls.crt, and tls.key in the current directory on this control plane node.
    3. Repeat for each contrail-etcd pod.
  4. Back up the etcd database on one of the control plane nodes. You only need to back up the database on one node.
    1. Log back in to one of the control plane nodes.
    2. Back up the etcd database.
      This example saves the database to /tmp/etcdbackup.db on this control plane node.where <etcd-pod-ip> is the IP address of the pod on this node and the <etcd-port> is the port that etcd is listening on (by default, 12379).
  5. Copy the database to a safe location.

Restore the Contrail Etcd Database in Release 22.2

Use this example procedure in release 22.2 to restore the Contrail etcd database from a snapshot.
  1. Copy the snapshot to all the contrail-etcd pods.
    Repeat for the other contrail-etcd pods.
  2. Restore the snapshot.
    1. Get a shell into one of the contrail-etcd pods.
      For example:where contrail-etcd-xxx is the etcd pod that you want to get a shell into.
    2. Restore the etcd database.
      This creates a <contrail-etcd-xxx>.etcd directory on the pod. where <contrail-etcd-xxx> is the name of the contrail-etcd pod that you're currently in and <contrail-etcd-xxx-ip> is the IP address of that pod. The <contrail-etcd-yyy> and <contrail-etcd-zzz> refer to the other contrail-etcd pods.
    3. Repeat for the other contrail-etcd pods, substituting the --name and --initial-advertise-peer-urls values with the respective contrail-etcd pod name and IP address.
  3. Copy the saved etcd data from the contrail-etcd pods to their respective control plane nodes.
    1. SSH into one of the control plane nodes.
    2. Copy the saved contrail-etcd-xxx.etcd from the respective contrail-etcd pod to the node.
      For example:where contrail-etcd-xxx is the name of the contrail-etcd pod on the control plane node that you logged in to.
    3. Repeat for the other control plane nodes.
  4. Stop the contrail-etcd pods.
    This sets the replicas to 0, which effectively stops the pods.
  5. Replace contrail-etcd data with the data from the snapshot.
    1. SSH into one of the control plane nodes.
    2. Replace the data.
      where contrail-etcd-xxx is the name of the contrail-etcd pod on the control plane node that you logged in to.
    3. Repeat for the other control plane nodes.
  6. Start the contrail-etcd pods.
    This sets the replicas to 3, which effectively starts the pods.
  7. Restart the contrail-system apiserver and controller.
    Delete all the contrail-k8s-apiserver and contrail-k8s-controller pods.These pods will automatically restart.
  8. Restart the vrouters.
    Delete all the contrail-vrouter-masters and contrail-vrouter-nodes pods.These pods will automatically restart.
  9. Check that all pods are in running state.

Restore the Contrail Etcd Database in Release 22.3

Use this example procedure in release 22.3 to restore the Contrail etcd database from a snapshot.
  1. Copy the snapshot you want to restore to all the control plane nodes.
    The steps below assume you've copied the snapshot to /tmp/etcdbackup.db on all the control plane nodes.
  2. Restore the snapshot.
    1. Log in to one of the control plane nodes. In this example, we're logging in to the control plane node that is hosting contrail-etcd-0.
    2. Restore the etcd database to the contrail-etcd-0 pod on this control plane node.
      This creates a contrail-etcd-0.etcd directory on the node. where --name=contrail-etcd-0 specifies that this command is restoring the database to contrail-etcd-0, --initial-cluster=... lists all the contrail-etcd members in the cluster, and --initial-advertise-peer-urls=... refers to the IP address and port number that the contrail-etcd-0 pod is listening on.
    3. Repeat for the other contrail-etcd pods on their respective control plane nodes, substituting the --name and --initial-advertise-peer-urls values with the respective contrail-etcd pod name and IP address.
  3. Stop the contrail-etcd pods.
    This sets the replicas to 0, which effectively stops the pods.
  4. Replace contrail-etcd data with the data from the snapshot.
    1. SSH into one of the control plane nodes.
    2. Replace the data. Recall that the snapshot is stored in the contrail-etcd-<xxx>.etcd directory.
      where contrail-etcd-xxx is the name of the contrail-etcd pod on the control plane node that you logged in to.
    3. Repeat for the other control plane nodes.
  5. Start the contrail-etcd pods.
    This sets the replicas to 3, which effectively starts the pods.
  6. Restart the contrail-system apiserver and controller.
    Delete all the contrail-k8s-apiserver and contrail-k8s-controller pods.These pods will automatically restart.
  7. Restart the vrouters.
    Delete all the contrail-vrouter-masters and contrail-vrouter-nodes pods.These pods will automatically restart.
  8. Check that all pods are in running state.