AviInfraSetting with Services, Ingress, or Route

Attaching AviInfraSetting to Services

Using Gateway API: Gateway APIs provide interfaces to structure Kubernetes service networking. For more information, see Gateway API.

AKO

supports Gateway APIs through the servicesAPI flag in the values.yaml. For more information on how

AKO

integrates with Gateway API, see Gateway and Gateway Class. The AviInfraSetting resource can be attached to a

Gateway

Class object, through the

.spec.parametersRef

as shown below:

apiVersion: networking.x-k8s.io/v1alpha1
kind: GatewayClass
metadata:
  name: avi-gateway-class
spec:
  controller: ako.vmware.com/avi-lb
  parametersRef:
    group: ako.vmware.com
    kind: AviInfraSetting
    name: my-infrasetting

Using Annotation: If the

servicesAPI

flag is not set to

True

, and

AKO

is not watching over the Gateway APIs, Services of Type

LoadBalancer

can specify the AviInfraSetting using an annotation as shown below:

annotations:
    aviinfrasetting.ako.vmware.com/name: "my-infrasetting"

Attaching

AviInfraSetting

to Ingress

apiVersion: networking.k8s.io/v1
kind: IngressClass
metadata:
  name: avi-ingress-class
spec:
  controller: ako.vmware.com/avi-lb
  parameters:
    apiGroup: ako.vmware.com
    kind: AviInfraSetting
    name: my-infrasetting

Attaching

AviInfraSetting

to Routes

 annotations:
     aviinfrasetting.ako.vmware.com/name: "my-infrasetting"

AviInfraSetting

Configure Service Engine Group

The AviInfraSetting CRD can be used to configure Service Engine Groups for virtual services created corresponding to Services/Ingresses/ Routes. Ensure the Service Engine Group is created and configured in the

NSX Advanced Load Balancer Controller

prior to this CRD creation.

  seGroup:
       name: compact-se-group

AKO

tries to configure labels on the SEG specified in the AviInfraSetting resources, which enables static route syncing on the member SEs.

AKO

configures the labels only when the SEGs do not have any other labels configured already. In case

AKO

finds the SEG configured with some different labels, the AviInfraSetting resource would be Rejected.

Once deployed, the member SEs will not reflect any changes related to label additions or updates on the SEGs.

Therefore, label-based static route syncing will not work on already deployed SEs. Ensure that the SEGs have no member SEs deployed before specifying the SEG in the AviInfraSetting resource to properly configure static route syncing.

Configure VIP Networks

The

AviInfraSetting

CRD can be used to configure VIP networks for virtual services created corresponding to Services/Ingresses/Openshift Routes. Ensure that the network is present in the

NSX Advanced Load Balancer Controller

prior to this CRD creation.

Starting with

AKO

version 1.10.3,

networkUUID

field has been introduced as part of

vipNetworks

setting to choose appropriate network. User can specify either

networkName

or

networkUUID

while specifying the network.

vipNetworks: - networkName: vip-network-10-10-10-0-24 cidr: 10.10.10.0/24 

Or

    network:
      vipNetworks:
        - networkUUID: dvportgroup-2167-cloud-d4b24fc7-a435-408d-af9f-150229a6fea6f
          cidr: 10.10.10.0/24

If there are duplicate network entries at the Controller side,
AKO
recommends to use networkUUID while choosing vipNetwork for new aviinfrasetting object creation.
Multiple network names can be added to the CRD (only in case of AWS cloud). The Avi virtualservices flag will acquire a VIP from each of these specified networks. If Avi fails to allocate a VIP due to IP exhaustion, this results in complete failure of entire request. This is same as vip allocation failures in single VIP.

AKO

1.5.1 updates the schema to provide VIP network information in AviInfraSetting CRD. For further details, see Upgrade AKO.

Configure Pool Placement Networks

AviInfraSetting

CRD can be used to configure Pool Placement Settings on the

AKO

created Pools in order for the Service Engines to place the Pools on appropriate interfaces.

With

AKO

1.10.3,

networkUUID

field has been introduced as part of

nodeNetworks

setting to choose appropriate network. User can specify either

networkName

or

networkUUID

while specifying the network.

        network:
          nodeNetworks:
            - networkName: node-network-10-10-20-0-24
              cidrs:
              - 10.10.20.0/24

        network:
          nodeNetworks:
            - networUUID: dvportgroup-4167-cloud-d4b24fc7-a435-408d-af9f-150229a6fea6f
              cidrs:
              - 10.10.20.0/24

As shown in the examples, each unique network can be specified using either

networkName

or

networkUUID

with appropriate cidrs.

If there are duplicate network entries at the Controller side,

AKO

recommends to use

networkUUID

while choosing

nodeNetworks

for new aviinfrasetting object creation.

If two Kubernetes clusters have overlapping Pod CIDRs, the service engine needs to identify the right gateway for each of the overlapping CIDR groups. This is achieved by specifying the right placement network for the pools that helps the Service Engine place the pools appropriately.

Enable/Disable Route Health Injection

AviInfraSetting CRD is used to activate/deactivate Route Health Injection (RHI) on the virtual services created by

AKO

.

network:
      enableRhi: true

This overrides the global

enableRHI

flag for the virtual services corresponding to the AviInfraSetting.

Enable/Disable Public IP

The AviInfraSetting CRD can be used to activate/deactivate Public IP on the virtual services created by

AKO

.

   network:
      enablePublicIP: true

Activating Public IP is only supported for public clouds.

Configure BGP Peer Labels for BGP Virtual Services

AviInfraSetting CRD can be used to configure BGP Peer labels for BGP virtual services.

AKO

configures the virtual services with the appropriate peer labels, only when

enableRHI

is enabled, either during

AKO

installation through helm chart’s values.yaml or through the

AviInfraSetting

CRD itself. If not set to

true

, the

AviInfraSetting

resource will be marked

Rejected

.

   bgpPeerLabels:
      - peer1
      - peer2

Use Dedicated VIP for Ingress

AviInfraSetting CRD can be used to allocate a dedicated vip per Ingress FQDN.

For the subset of ingresses, that refer to an ingress class which in turn refers to an

AviInfraSetting

CRD setting that has

shardSize

as

DEDICATED

, will get VIP per Ingress FQDN.

For passthrough routes/ingresses, setting

l7Settings:shardSize

present in

AviInfrasetting

CRD overrides setting

L7Settings.passthroughShardSize

present in values.yaml.

The value

DEDICATED

is not supported when AviInfra CRD is applied to the passthrough route/ingress.

Configure IPv6

AviInfraSetting CRD can be used to enable IPv6, IPv4, or both IPv4 and IPv6 VIPs on the virtual services created by

AKO

.

This feature is currently under Tech-Preview.

    network:
      vipNetworks:
        - networkName: vip-network-10-10-10-0-24
          cidr: 10.10.10.0/24
          v6cidr: 2002::1234:abcd:ffff:c0a8:101/64