How to recover after deleting the Calico CRDs from a cluster

Issue

Calico uses a number of Custom Resource Definitions (CRDs) in order to store configuration data in Custom Resources. In the event that these CRDs are accidentally deleted from a cluster by a user, the configuration data in these Custom Resources will be deleted, preventing successful programming of pod networking. This article documents how to recreate the CRDs and ensure the configuration data is also re-populated.

Pre-requisites

• A Kubernetes v1.8.x - v1.16.x cluster provisioned by the RKE CLI or Rancher v2.x, running with the Canal or Calico network providers

Resolution

1. Re-create the CRDs

The first step is to re-create the CRDs. The definitions are dependent upon the Kubernetes version running in the cluster, as well as whether the cluster is running the Canal or Calico network provider. Please refer to the matching network provider and Kubernetes version combination below:

Canal Network Provider and Kubernetes version 1.8.x - 1.12.x

Download the canal-calico-crds-k8s-1-8-to-1-12.yaml file and apply this to the cluster: kubectl apply -f canal-calico-crds-k8s-1-8-to-1-12.yaml

Canal Network Provider and Kubernetes version 1.13.x - 1.14.x

Download the canal-calico-crds-k8s-1-13-to-1-14.yaml file and apply this to the cluster: kubectl apply -f canal-calico-crds-k8s-1-13-to-1-14.yaml

Canal Network Provider and Kubernetes version 1.15.x

Download the canal-calico-crds-k8s-1-15.yaml file and apply this to the cluster: kubectl apply -f canal-calico-crds-k8s-1-15.yaml

Canal Network Provider and Kubernetes version 1.16.x

Download the canal-calico-crds-k8s-1-16.yaml file and apply this to the cluster: kubectl apply -f canal-calico-crds-k8s-1-16.yaml

Calico Network Provider and Kubernetes version 1.8.x - 1.12.x

Download the calico-calico-crds-k8s-1-8-to-1-12.yaml file and apply this to the cluster: kubectl apply -f calico-calico-crds-k8s-1-8-to-1-12.yaml

Calico Network Provider and Kubernetes version 1.13.x - 1.14.x

Download the calico-calico-crds-k8s-1-13-to-1-14.yaml file and apply this to the cluster: kubectl apply -f calico-calico-crds-k8s-1-13-to-1-14.yaml

Calico Network Provider and Kubernetes version 1.15.x

Download the calico-calico-crds-k8s-1-15.yaml file and apply this to the cluster: kubectl apply -f calico-calico-crds-k8s-1-15.yaml

Calico Network Provider and Kubernetes version 1.16.x

Download the calico-calico-crds-k8s-1-16.yaml file and apply this to the cluster: kubectl apply -f calico-calico-crds-k8s-1-16.yaml

2. Delete a network pod to trigger re-creation of the Calico custom resources

Delete a network provider pod from a single node in the cluster, per the network provider specific instructions below. This will trigger creation of a new pod on that node, and the initialization of this will create the Calico custom resources containing Calico configuration. After this cluster networking should be fully restored.

Canal Network Provider

Delete one of the canal pods within the kube-system namespace.

Calico Network Provider

Delete one of the calico-node pods within the kube-system namespace.