Sep 10, 2020
Kubernetes Migration: How To Move Data Freely Across Clusters
This blog focuses on migrating Kubernetes clusters from one cloud provider to another. We will be migrating our entire data from Google Kubernetes Engine to Azure Kubernetes Service using Velero.
Prerequisite
- A Kubernetes cluster > 1.10
Setup Velero with Restic Integration
Velero consists of a client installed on your local computer and a server that runs in your Kubernetes cluster, like Helm.
Installing Velero Client
You can find the latest release corresponding to your OS and system and download Velero from there:
Extract the tarball (change the version depending on yours) and move the Velero binary to /usr/local/bin
Create a Bucket for Velero on GCP
Velero needs an object storage bucket where it will store the backup. Create a GCS bucket using:
Create a Service Account for Velero
Install Velero Server on GKE and AKS
Use the — use-restic flag on the Velero install command to install restic integration.
After that, you can see a DaemonSet of restic and deployment of Velero in your Kubernetes cluster.
Restic Components
In addition, there are three more Custom Resource Definitions and their associated controllers to provide restic support.
Restic Repository
- Maintain the complete lifecycle for Velero’s restic repositories.
- Restic lifecycle commands such as restic init check and prune are handled by this CRD controller.
PodVolumeBackup
- This CRD backs up the persistent volume based on the annotated pod in selected namespaces.
- This controller executes backup commands on the pod to initialize backups.
PodVolumeRestore
- This controller restores the respective pods that were inside restic backups. And this controller is responsible for the restore commands execution.
Backup an application on GKE
For this blog post, we are considering that Kubernetes already has an application that is using persistent volumes. Or you can install WordPress as an example as explained here.
We will perform GKE Persistent disk migration to Azure Persistent Disk using Velero.
Follow the below steps:
- To back up, the deployment or statefulset checks for the volume name that is mounted to backup that particular persistent volume. For example, here pods need to be annotated with Volume Name “data”.
2. Annotate the pods with the volume names, you’d like to take the backup of and only those volumes will be backed up:
For example,
3. Take a backup of the entire namespace in which the application is running. You can also specify multiple namespaces or skip this flag to backup all namespaces by default. We are going to backup only one namespace in this blog.
4. Monitor the progress of backup:
Once the backup is complete, you can list it using:
You can also check the backup on GCP Portal under Storage.
Select the bucket you created and you should see a similar directory structure:
Restore the application to AKS
Follow the below steps to restore the backup:
- Make sure to have the same StorageClass available in Azure as used by GKE Persistent Volumes. For example, if the Storage Class of the PVs is “persistent-ssd”, create the same on AKS using the template below:
You can monitor the progress of restore:
You can also check on GCP Portal, a new folder “restores” is created under the bucket.
In some time, you should be able to see that the application namespace is back and WordPress and MySQL pods are running again.
Troubleshooting
For any errors/issues related to Velero, you may find below commands helpful for debugging purposes:
Conclusion
The migration of persistent workloads across Kubernetes clusters on different cloud providers is difficult. This became possible by using restic integration with the Velero backup tool. This tool is still said to be in beta quality as mentioned on the official site. I have performed GKE to AKS migration and it went successfully. You can try other combinations of different cloud providers for migrations.
The only drawback of using Velero to migrate data is if your data is too huge, it may take a while to complete the migration. It took me almost a day to migrate a 350 GB disk from GKE to AKS. But, if your data is comparatively less, this should be an efficient and hassle-free way to migrate it.
Originally published at https://www.velotio.com.
