Support for setting setting IPs if kubelet is started with --cloud-provider=external

[3deep5me]

Hello,

We are trying to set up a lab environment with nodes that have both internal and external IPs. For this Cilium Issue, we were able to add a second interface using this guide: Containerlab Manual. This setup works well, but the issue is that the kubelet does not automatically pick up the second IP address by default. Additionally, there is no way to configure it through kubelet arguments.

Why did i place this Issue here?

Normally, the external and internal IPs are set by the cloud provider or cloud controller manager. Would it be possible to implement some simple logic to determine the IP address from the node-interfaces?

[aojea]

The cloud provider kind obtains the ips reported by docker, and only one of each ip family.

Can you expand on the requirement to have all the ips represented in the node object? Why do you need that?

[3deep5me]

I try to create a environment with kind to test this feature cilium/cilium#24403 during development and in the ci. For this feature Cilium uses the ExternalIP in the Node Object.

[aojea]

Fair, in the meantime can you check if you can add it manually by patching the node status, kubectl allows to patch subsresources in beta iirc

[aojea]

Ok, it does not work, both the kubelet (without cloud-provider) or the cloud-provider (in this case cloud-provider-kind) reconciles the addresses after some period :/

/kubectl patch node external-worker --subresource=status --type='json' \urce=status --type='json' \
-p='[{"op": "add", "path": "/status/addresses/-", "value": {"type": "InternalIP", "address": "11.11.11.11"}}]'

after applying the address it shows up

kubectl get node external-worker -o jsonpath='{.status.addresses[*].address}'
192.168.8.6 external-worker 11.11.11.11

but eventually it is removed

$ kubectl get node external-worker -o jsonpath='{.status.addresses[*].address}'
192.168.8.6 external-worker

So, if you just want to test the assignment of node ips, you need to configure the kubelet to use external IPS

• and not run any cloud provider and set the ips manually
• or use the cloud provider kind just for bootstrapping and then remove it and add them manually

[3deep5me]

Thank you for testing! Sadly the kubelet do not have an option to set ExternalIP, see this issue:
kubernetes/kubernetes#42125
But yes I think your option two can work, thank you.

[aojea]

That's what I explained in #284 (comment) , but if you set the kube let to external cloud provider and you shutdown the external cloud provider, nothing will.reconcile the address and what you have manually set will remain