Cilium CNI Support with
AKO

The Cilium Container Network Interface (CNI) plugin is supported on Kubernetes. Cilium can be configured to use either Cluster Scope mode or Kubernetes Host Scope mode for IPAM, and
AKO
is capable of supporting both.

Viewing IPAM Mode

To see the IPAM mode, check the ipam field in the cilium-config configmap in the kube-system namespace.
In cluster scope mode, the IPAM value is
cluster-pool
.
ipam: cluster-pool
In Kubernetes host scope mode, the IPAM value is
kubernetes
.

Configuring
AKO
to use Cilium CNI

AKO
reads the per-node PodCIDRs to be able to sync the static route configurations. With Cilium CNI, there are two modes to configure the per-node PodCIDRs.

Cluster Scope IPAM mode

By default, Cilium uses the Cluster Scope mode for IPAM. To use Cilium in the cluster scope IPAM mode with
AKO
, the AKOSettings.cniPlugin value in the
AKO
Helm chart
values.yaml
must be set to cilium. For more information, see Configuring AKO and Parameters.
In the cluster scope mode, the podCIDRs range are made available through the CiliumNode (cilium.io/v2.CiliumNode) CRD and
AKO
reads this CRD to determine the Pod CIDR to Node IP mappings. The CiliumNode CRD object is created with the same name as the node name (one per node) and specifies the podCIDRs range in the
spec.ipam.podCIDRs
field.

Kubernetes Host Scope IPAM Mode

In Kubernetes host scope mode, podCIDRs are allocated out of the PodCIDR range associated to each node by Kubernetes. This PodCIDR range is available in the Node
spec.podCIDRs
field. By default, when the
cniPlugin
flag is empty,
AKO
determines the Pod CIDR to Node IP mappings from Node spec.podCIDRs field and configures the static routes accordingly. Hence, the
cniPlugin
flag must be left empty for Kubernetes Host Scope IPAM mode.

Caveats

Starting with version 1.13 Cilium CNI also provides basic SCTP support. However, SCTP support is not enabled by default and needs to be enabled in the Cilium configuration before the SCTP protocol can be used in port definitions.
To enable SCTP support, set the
enable-sctp
field in
cilium-config
configmap to
true
.
enable-sctp: "true"
The Kernel version on the Kubernetes nodes must be 5.2 or higher for Cilium to support SCTP. Otherwise, the Cilium daemon can crash with the following error:
level=fatal msg=”failed to start: daemon creation failed: SCTP support needs kernel 5.2 or newer” subsys=daemon.