You must edit the configuration file

/var/vcap/data/jobs/ncp/xxxxxxxx/config/ncp.ini

(where xxxxxxxx is the BOSH deployment ID). This file will then be copied to

rootfs

and used by NCP every time NCP restarts. The file must be edited on each master node.

The relevant options are in the

nsx_v3

section.

use_native_loadbalancer
- If set to
False
, NCP will not process any Ingress or service of type Loadbalancer updates, regardless of its annotations. This setting applies to the whole TKGI cluster. The default is
True
.
default_ingress_class_nsx
- If set to
True
, NCP becomes the default Ingress controller and will handle both Ingresses annotated with
kubernetes.io/ingress.class: "nsx"
and Ingresses without any annotation. If set to
False
, NCP will only handle Ingresses annotated with
kubernetes.io/ingress.class: "nsx"
. The default is
True
.
Starting with NCP 3.2.1,
default_ingress_class_nsx
is deprecated. NCP will look at the following when resolving the Ingress class:
annotations
ingressClass
objects
If no annotation is specified and
use_native_loadbalancer
is True, then NSX-LB will handle the Ingress. Otherwise, NSX-LB will not handle it.

If you want NCP to assign a floating IP to the NGINX controller pod and update the status of Ingresses with the floating IP, do the following:

In the
k8s
section in
ncp.ini
, set
ingress_mode=nat
.
Add the annotation
ncp/ingress-controller: "True"
to the NGINX Ingress controller pod.

NCP will update the status of Ingresses that have the annotation

kubernetes.io/ingress.class: "nginx"

with the NGINX Ingress controller pod's floating IP. If

default_ingress_class_nsx=False

, NCP will also update the status of Ingresses without the

kubernetes.io/ingress.class

annotation with the NGINX Ingress controller pod's floating IP.

Note: Even if the NGINX Ingress controller pod does not have the annotation

ncp/ingress-controller: "True"

, NCP will update the status of the Ingresses mentioned above to

loadBalancer: {}

. The Ingresses could then be stuck in a loop where the NGINX controller updates the Ingress status to

loadBalancer: {ingress: [{ip: <IP>}]}

and NCP updates the Ingress status to

loadBalancer: {}

. To avoid this situation, perform the following steps:

If the Ingress controller is from https://github.com/kubernetes/ingress-nginx,
On the Ingress controller, change the
ingress-class
to something other than
"nginx"
.
If there is an Ingress with the annotation
kubernetes.io/ingress-class: "nginx"
, change the annotation to a different value.
For more information, see https://kubernetes.github.io/ingress-nginx/user-guide/multiple-ingress.
If the Ingress controller is from https://github.com/nginxinc/kubernetes-ingress,
On the Ingress controller, change the
ingress-class
to something other than
"nginx"
.
If there is an Ingress with the annotation
kubernetes.io/ingress-class: "nginx"
, change the annotation to a different value.
On the Ingress controller pod, set
use-ingress-class-only
to
True
. This will stop this controller from updating Ingresses without the
kubernetes.io/ingress-class
annotation.
For more information, see https://github.com/kubernetes/ingress-nginx/blob/master/docs/user-guide/multiple-ingress.md.

For third-party Ingress controllers deployed in NAT mode, you can modify the

http_ingress_port

and

https_ingress_port

parameters in the

k8s

section to specify custom ports for the NAT rules exposed for the Ingress controller.

Follow this procedure to let NCP handle

nsx

-class Ingresses.

Edit the
nsx_v3
section in
ncp.ini
on each master node.
Set
default_ingress_class_nsx
to
False
.
Leave
use_native_loadbalancer
set to
True
, the default value.
Restart NCP on each master node. This might cause a master failover.
Annotate all the Ingresses that you want NCP to handle with
kubernetes.io/ingress.class: "nsx"
.

Follow this procedure:

No need to edit
ncp.ini
, but ensure that every Ingress is annotated.
Ingresses to be handled by NCP should be annotated with
kubernetes.io/ingress.class: "nsx"
.
Although NCP will handle Ingresses without the
kubernetes.io/ingress.class
annotation, in the case of multiple Ingress controllers, the best practice is to always have the
kubernetes.io/ingress.class
annotation and not to rely on the default Ingress controller behavior.
Ingresses to be handled by third-party Ingress controllers must be annotated with the value required by those Ingress controllers.

Follow this procedure:

Edit the
nsx_v3
section in
ncp.ini
on each master node.
Set
use_native_loadbalancer
to
False
. The value of
default_ingress_class_nsx
is now irrelevant.
Restart NCP on each master node. This might cause a master failover.

Note that NCP will also not handle services of type LoadBalancer