AviInfraSetting with Services, Ingress, or Route

Attaching AviInfraSetting to Services

Using Gateway API: Gateway APIs provide interfaces to structure Kubernetes service networking. For more information, see Gateway API.

AKO

supports Gateway APIs through the servicesAPI flag in the values.yaml. For more information on how

AKO

integrates with Gateway API, see Gateway and Gateway Class. The AviInfraSetting resource can be attached to a

Gateway

Class object, through the

.spec.parametersRef

as shown below:

apiVersion: networking.x-k8s.io/v1alpha1
kind: GatewayClass
metadata:
name: avi-gateway-class
spec:
controller: ako.vmware.com/avi-lb
parametersRef:
 group: ako.vmware.com
 kind: AviInfraSetting
 name: my-infrasetting

Using Annotation: If the

servicesAPI

flag is not set to

True

, and

AKO

is not watching over the Gateway APIs, Services of Type

LoadBalancer

can specify the AviInfraSetting using an annotation as shown below:

annotations:
 aviinfrasetting.ako.vmware.com/name: "my-infrasetting"

Attaching

AviInfraSetting

to Ingress

apiVersion: networking.k8s.io/v1
kind: IngressClass
metadata:
name: avi-ingress-class
spec:
controller: ako.vmware.com/avi-lb
parameters:
 apiGroup: ako.vmware.com
 kind: AviInfraSetting
 name: my-infrasetting

Attaching

AviInfraSetting

to Routes

annotations:
  aviinfrasetting.ako.vmware.com/name: "my-infrasetting"

Using Namespace Annotation

annotations:
 aviinfrasetting.ako.vmware.com/name: "my-infrasetting"

AviInfraSetting

Configure Service Engine Group

The AviInfraSetting CRD can be used to configure Service Engine Groups for virtual services created corresponding to Services/Ingresses/ Routes. Ensure the Service Engine Group is created and configured in the

Avi Load Balancer Controller

prior to this CRD creation.

seGroup:
    name: compact-se-group

AKO

tries to configure labels on the SEG specified in the AviInfraSetting resources, which enables static route syncing on the member SEs.

AKO

configures the labels only when the SEGs do not have any other labels configured already. In case

AKO

finds the SEG configured with some different labels, the AviInfraSetting resource would be Rejected.

Once deployed, the member SEs will not reflect any changes related to label additions or updates on the SEGs.

Therefore, label-based static route syncing will not work on already deployed SEs. Ensure that the SEGs have no member SEs deployed before specifying the SEG in the AviInfraSetting resource to properly configure static route syncing.

Configure VIP Networks

The

AviInfraSetting

CRD can be used to configure VIP networks for virtual services created corresponding to Services/Ingresses/Openshift Routes. Ensure that the network is present in the

Avi Load Balancer Controller

prior to this CRD creation.

Starting with

AKO

version 1.10.3,

networkUUID

field has been introduced as part of

vipNetworks

setting to choose appropriate network. User can specify either

networkName

or

networkUUID

while specifying the network.

vipNetworks: - networkName: vip-network-10-10-10-0-24 cidr: 10.10.10.0/24

Or

 network:
   vipNetworks:
     - networkUUID: dvportgroup-2167-cloud-d4b24fc7-a435-408d-af9f-150229a6fea6f
       cidr: 10.10.10.0/24

If there are duplicate network entries at the Controller side,
AKO
recommends to use networkUUID while choosing vipNetwork for new aviinfrasetting object creation.
Multiple network names can be added to the CRD (only in case of AWS cloud). The Avi virtualservices flag will acquire a VIP from each of these specified networks. If Avi fails to allocate a VIP due to IP exhaustion, this results in complete failure of entire request. This is same as vip allocation failures in single VIP.

AKO

1.5.1 updates the schema to provide VIP network information in AviInfraSetting CRD. For further details, see Upgrade AKO.

Configure Pool Placement Networks

AviInfraSetting

CRD can be used to configure Pool Placement Settings on the

AKO

created Pools in order for the Service Engines to place the Pools on appropriate interfaces.

With

AKO

1.10.3,

networkUUID

field has been introduced as part of

nodeNetworks

setting to choose appropriate network. User can specify either

networkName

or

networkUUID

while specifying the network.

     network:
       nodeNetworks:
         - networkName: node-network-10-10-20-0-24
           cidrs:
           - 10.10.20.0/24

     network:
       nodeNetworks:
         - networUUID: dvportgroup-4167-cloud-d4b24fc7-a435-408d-af9f-150229a6fea6f
           cidrs:
           - 10.10.20.0/24

As shown in the examples, each unique network can be specified using either

networkName

or

networkUUID

with appropriate cidrs.

If there are duplicate network entries at the Controller side,

AKO

recommends to use

networkUUID

while choosing

nodeNetworks

for new aviinfrasetting object creation.

If two Kubernetes clusters have overlapping Pod CIDRs, the service engine needs to identify the right gateway for each of the overlapping CIDR groups. This is achieved by specifying the right placement network for the pools that helps the Service Engine place the pools appropriately.

Enable/Disable Route Health Injection

AviInfraSetting CRD is used to activate/deactivate Route Health Injection (RHI) on the virtual services created by

AKO

.

network:
   enableRhi: true

This overrides the global

enableRHI

flag for the virtual services corresponding to the AviInfraSetting.

Enable/Disable Public IP

The AviInfraSetting CRD can be used to activate/deactivate Public IP on the virtual services created by

AKO

.

network:
   enablePublicIP: true

Activating Public IP is only supported for public clouds.

Custom Ports

To customize the ports for L7 Shared or Dedicated virtual services created by

AKO

, users must specify the port details under the

listeners

setting. The ports listed here will be added to the virtual service, along with the default ports 80 and 443, as well as the ports specified in the hostrule. Settings defined in the aviinfrasetting will always precede the default and hostrule configurations.

listeners:
- port: 80
 enableHTTP2: false
 enableSSL: false
- port: 6443
 enableSSL: true
 

One of the ports specified in the setting must have the

enableSSL

field set to

true

. If

enableHTTP2

is set to

true

, HTTP/2 traffic will be supported from the client to the Service Engine.

Configure BGP Peer Labels for BGP Virtual Services

AviInfraSetting CRD can be used to configure BGP Peer labels for BGP virtual services.

AKO

configures the virtual services with the appropriate peer labels, only when

enableRHI

is enabled, either during

AKO

installation through helm chart’s values.yaml or through the

AviInfraSetting

CRD itself. If not set to

true

, the

AviInfraSetting

resource will be marked

Rejected

.

bgpPeerLabels:
   - peer1
   - peer2

Use Dedicated VIP for Ingress

AviInfraSetting CRD can be used to allocate a dedicated vip per Ingress FQDN.

 l7Settings:
   shardSize: DEDICATED

For the subset of ingresses, that refer to an ingress class which in turn refers to an

AviInfraSetting

CRD setting that has

shardSize

as

DEDICATED

, will get VIP per Ingress FQDN.

For passthrough routes/ingresses, setting

l7Settings:shardSize

present in

AviInfrasetting

CRD overrides setting

L7Settings.passthroughShardSize

present in values.yaml.

The value

DEDICATED

is not supported when AviInfra CRD is applied to the passthrough route/ingress.

Configure IPv6

AviInfraSetting CRD can be used to enable IPv6, IPv4, or both IPv4 and IPv6 VIPs on the virtual services created by

AKO

.

 network:
   vipNetworks:
     - networkName: vip-network-10-10-10-0-24
       cidr: 10.10.10.0/24
       v6cidr: 2002::1234:abcd:ffff:c0a8:101/64

Configure T1LR for NSX-T Cloud

AviInfraSetting CRD can be used to configure T1LR. For all the Services and Ingresses that refer to an AviInfraSetting CRD with T1LR configured,

AKO

will use the T1LR defined in the AviInfraSetting while creating objects in

Avi Load Balancer

. For the rest of the resources,

AKO

will use the global T1LR configured in the ako config map. In case of an update in

nsxSettings.t1lr

, the existing virtual services will continue to use the old t1lr value. The updated t1lr value will only apply to the newly created virtual services.

 nsxSettings:
   t1lr: /infra/tier1/tier1_974b13d5-9f68-4be8-8149-a48a5686a3ef

AKO sets up routes in the Avi VRF corresponding to the global T1lr defined in the config map. However, for the T1lr defined in the AviInfraSetting CR, AKO will not setup any routes in the Avi Controller. This will create a connectivity issue between the Service Engine and the pool servers when AKO is deployed in ClusterIP mode. To resolve this connectivity issue, you can manually add routes in VRF, associated with the given T1LR. The connectivity issue will be rewsolved when AKO is deployed in NodePort or NPL mode or using NCP as CNI.