Skip to main content
Version: 5.21.0

Backup and Restore using Velero

Backing up your Kubernetes cluster resources and persistent volumes has never been easier. We offer a robust backup solution that leverages the Big Bang Velero addon and Amazon Relational Database Service (RDS). Velero can easily be deployed using standard processes and offers all the necessary features to ensure your data is safe and secure.

Our solution includes automatic Rancher Kubernetes Engine Government (RKE2) etcd cluster backups, which are saved locally, before being uploaded to a Simple Storage Service (S3) target. We also utilize Elastic Block Store (EBS) Snapshots to back up persistent volumes, ensuring data is always protected during a disaster.

Velero Backup​

info

For initiating an ad-hoc backup, follow the instructions available in the official Velero documentation.

Backup Storage Location​

After a successful backup operation, a folder will be created in an S3 bucket specific to your cluster. The folder path will resemble the following:

s3://v-xxx-velero-backup/backups/velero-velero-default-timestamp

Within this folder, you will find a compressed JSON file containing a list of the backed-up resources, named per the following pattern:

velero-velero-default-YYYYMMDDHHMMSS-resource-list.json.gz

Monitoring Backup Progress​

Access Velero Pod​

To gain shell access to the Velero Pod, run:

kubectl exec -it -n velero deploy/velero-velero -- sh

List Backups​

To list all available backups, run:

velero backup get

The output provides essential information about each backup, including its status, errors, and warnings. For example:

NAME                                   STATUS      ERRORS   WARNINGS   CREATED                         EXPIRES   STORAGE LOCATION   SELECTOR
velero-velero-default-20220909030046   Completed   0        0          2022-09-09 03:00:46 +0000 UTC   6d        default            <none>

Backup Logs​

To examine the logs for a specific backup, use:

velero backup logs <BACKUP_NAME>

Backup Details​

To get an in-depth review of a particular backup, run:

velero backup describe <BACKUP_NAME>

For even more detailed information, including volume snapshots, add the --details flag:

velero backup describe <BACKUP_NAME> --details
note
  • Scheduled backups will generate backup objects, which can be used as restore points.
  • Ensure to review the logs to confirm your backups are free of errors or warnings.

Velero Restore​

Restore a Deleted Namespace and Child Objects​

Prerequisites​

  • Suspend the targeted resources, if managed by a GitOps tool, such as Flux, to avoid conflicts:
flux suspend hr -n <namespace> <resource-name>
  • Delete the namespace and its associated objects:
kubectl delete namespace <namespace-name>

Steps​

  1. Describe the Restore within the Velero Pod

    To describe the restore within the Velero Pod, execute the following command:

kubectl exec -it -n velero deploy/<velero-deploy-name> -- sh
velero restore describe <restore-name>
tip

You should see the restore phase as InProgress and the number of items that are being restored.

  1. Check Persistent Volume Status

    To check the persistent volume status, execute the following command:

    kubectl get pv

  2. Monitor Restore Logs

    To monitor the restoration process by viewing the Velero logs:

kubectl logs -n velero <velero-pod-name> | grep <pvc-or-resource-name>

  1. Validate the Restore

    To validate that all objects have been restored in the namespace, run:

kubectl get <resource-type> -n <namespace-name>

  1. Post-Restore Actions

    Verify the restoration by checking pod logs and application functionality.

Restore a Deleted ServiceAccount​

Steps​

  1. Delete a ServiceAccount

    To delete a ServiceAccount, execute the following command:

kubectl delete serviceaccount <serviceaccount-name> -n <namespace>

  1. Log into Velero Pod

    To log into Velero Pod, execute the following command:

kubectl exec -it -n velero deploy/<velero-deploy-name> -- sh
  1. Select a Backup to Restore
note

The following commands are executed within the Velero pod.

To list all available backups, run:

velero backup get

  1. Initiate a Restore

    To initiate a restore, execute the following command:

velero restore create --from-backup <backup-name>

  1. Review Restore Logs

    To review the restore logs, execute the following command:

velero restore logs <restore-name>

  1. Monitor Restore Status

    To track progress, execute the following command:

velero restore get