Monitoring metrics do not report external BGP peers #848
Description
The Prometheus metric kube_router_controller_bgp_peers does not include eBGP peers. Some setups, ie. when peering with a ToR switch this is the only type of peer that is used. Monitoring proper function is impossible in these setups.
As the node can be in Ready state even with 0 BGP peers traffic from any pod scheduled on a node without the peer established will be blackholed, leading to service interruptions.
I'd suggest either including eBGP peers in the kube_router_controller_bgp_peers count, or adding a separate metric for it.
It would also make sense to add an option signal the CNI as NonReady until a peer has been established.