Releases: kubernetes-sigs/kubespray
Releases Β· kubernetes-sigs/kubespray
v2.29.0
Urgent Upgrade Notes
(No, really, you MUST read this before you upgrade)
- Action required
/etc/hosts/ is no longer populated with all cluster nodes (#12382, @VannTen) - Action required
Add support forcoredns_affinityto change affinity of coredns deployments, defaulting to the upstream coredns deployment's one.
Thecorednsdeployment's node affinity has been removed, so thecorednspods will no longer be scheduled into control-planes by default. (#11994, @HoKim98) - Action required
Remove support for weave network plugin (#12230, @anshuman-agarwala) - Action required
The tag 'master' is removed, replaced by the tag 'control-plane' (#12228, @VannTen) - Action required
conntrack_modulesis removed; the list of conntrack modules to try to load is instead hardcoded, since there is no reason to have any other values. (#12475, @VannTen) - Action required
drop support for cri-o on ubuntu20. (#12233, @VannTen)
Changes by Kind
Feature
- A new
sysctl_ignoreerrorsvalue has been added, it allows to ignore errors about unknown keys that may be raised by sysctl (#12514, @bidorffOL) - A new configuration option kubelet_static_pod_path has been added which can be used to configure path of static pod manifests OR even to disable staticPodPath setting in kubelet by setting it as empty (STIG recommendation for worker nodes) (#12433, @shaleenbathla)
- Add cilium_install_extra_flags variable (#12262, @tmurakam)
- Add external_openstack_lbaas_member_subnet_id: str (not set by default), to define a specific
member-subnet-idfor the openstack load balancers (#12267, @voondo) - Add support for
containerd_extra_runtime_argsvariable to allow injection of additional runtime configuration options into containerd CRI plugin section. (#12247, @Ujstor) - Add support for
kubeadm_image_repovariable to change kubernetes core image repository (e.g.kube-apiserver,kube-proxy). (#12128, @HoKim98) - Add the possibility to use any values from Cilium Helm Chart (#12375, @cleman95)
- Added Prometheus Operator CRDs installation (#12441, @tico88612)
- Adds support for installing containerd as a static binary.
Bump containerd to 2.1.3, runc to 1.3.0, nerdctl to 2.1.2 (#12377, @yankay) - Bump ansible to 10.7.0 (#11924, @tico88612)
- Calico supports nftable mode (#12255, @tico88612)
- Control plane health check retries for apiserver, scheduler, and controller-manager are now configurable via
control_plane_health_retries(default: 60). (#12452, @aman4433) - Feat: Support certificate validity period config in kubeadm v1beta4 (#12272, @ErikJiang)
- Feat: add support for crio additional mounts (#12561, @mahendra77024)
- Introduce
crio_runtime_switchboolean to allow users to switch the crio runtime by removing pods and stopping crio and kubelet during upgrade ; otherwise crio has problems when trying to work with pods created with the old runtime. (#12008, @mahendra77024) - Introduced
coredns_replicasto alter coredns deployment replicas whenenable_dns_autoscaleris set to false. (#12387, @clwluvw) - Redeploy coredns and nodelocaldns when their configurations change. (#12401, @atobaum)
- Remove --auth-anonymous if kube_api_anonymous_auth is undefined. (#12353, @psychomantys)
- Support Debian 13 Trixie (#12456, @tico88612)
- Support for custom header configuration in containerd registry mirrors via inventory and role variables. Users can now specify headers (e.g., Authorization) for registry mirrors in
hosts.toml. (#12368, @pando85) - Support kubernetes v1.33.1 (#12199, @tmurakam)
- Update cni-plugin to 1.8.0 (#12551, @tmurakam)
- Update load balancers versions to Nginx 1.28.0, Haproxy 3.1.7 (#12178, @guoard)
- Upgrade external snapshot CRD to v0.15.0 (#12308, @tico88612)
- Upgrade multus cni from 4.1.0 to 4.2.2 (#12495, @ThisIsQasim)
- [calico] Update default calico to v3.30.3 (#12523, @tmurakam)
- [flannel] upgrade to 0.26.7 (#12260, @tico88612)
- [ingress-nginx] upgrade controller to version 1.13.3 (#12604, @mzaian)
Design
- Show node to be upgraded/uncordoned in upgrade/uncordon confirmation prompt when using
upgrade_node_confirmorupgrade_node_post_upgrade_confirm(#12399, @MatthiasLohr)
Bug or Regression
- Add argocd_install_checksum: str, to define the checksum of
argocd_install_url(#12266, @voondo) - Add missing addresses in kube-apiserver certificate SAN. (#12413, @hhk7734)
- Bugfix: skip etcd cert extraction if cilium identity uses crd (#12565, @mahendra77024)
- Fix Cilium installation issues (caused by templating syntax errors) when certain non-default features (encryption, etc.) are enabled (#12280, @spantaleev)
- Fix Hubble-Relay peer discovery in clusters using non-default cluster name by properly configuring clusterDomain in Cilium Helm values (#12346, @mertcancam)
- Fix cilium installation role to render cilium_config_extra_vars into helm values (#12335, @atobaum)
- Fix cilium_policy_audit_mode variable (#12569, @guoard)
- Fix error when using
kubeadm_ignore_preflight_errors: ['all'](#12606, @VannTen) - Fix ingress-nginx DaemonSet and Service templates rendering TCP/UDP ports as strings, which prevented correct export of TCP/UDP services via NGINX ingress controller. (#12442, @MahdadGhasemian)
- Fix invalid PodSecurity admission configuration when
kube_pod_security_use_default: false(#12439, @AMacedoP) - Fix scale.yml problems with cached IP facts (#12243, @fox0430)
- Fix the Cilium cluster, which is upgraded from 2.27 to 2.28 will break
Fix helm release re-use message when installing repeatedly (#12254, @tico88612) - Fix the issue of etcd node addition failure caused by incorrect ETCD_INITIAL_CLUSTER configuration. (#12342, @liuxu623)
- Fix(kubeadm): Conditionally add --skip-phases flag for v1.32.0+ (#12351, @ErikJiang)
- Fix: A timeout occurs when running the offline deployment script using Podman. (#11962, @DearJey)
- Fix: When running
./manage-offline-container-images.sh registerwith using Podman, getting the image_id fails and the script is interrupted. (#11961, @DearJey) - Fix: kubeadm secondary nodes use file discovery validation failed (#12132, @tico88612)
- Fixed a looping timeout bug when deleting an entire cluster (#12300, @chadswen)
- Fixed cilium_enable_bgp_control_plane config (#12430, @XuhuiSun95)
- Fixed packages installation on Alma/Rocky Linux when behind a proxy (#12264, @root-expert)
- Fixes a syntax error that made the '_bgp_config' an 'AnsibleUnsafeText' instead of a 'dict', which caused the "Calico | Process BGP Configuration" step to fail (#12258, @mathgaming)
- Make APT updates its package cache before
dist-upgrade(#12465, @guoard) - Nodelocaldns capabilities only use NET_ADMIN, not privileged (#12398, @tico88612)
- [reset] When
flush_iptables: true, set IPv4/IPv6 default policies (INPUT/FORWARD/OUTPUT) to ACCEPT before flushing and delete user-defined chains to ensure a clean, non-locking reset. (#12552, @sasantk)
Other (Cleanup or Flake)
- Change the EOL Debian backports apt package to archive.debian.org (#12434, @tico88612)
- Dnsmasq directories are no longer cleaned up (#12380, @wangsifei99)
- Fix netcheck etcd image tag align with the etcd current version (#12402, @wangsifei99)
- Remove Ubuntu 20.04 support (#12301, @tico88612)
- When using timer based kubeadm certs renewal, only renew if certificates are near expiration (#12194, @panpan0000)
Components
- kubernetes 1.33.5
- etcd 3.5.22
- docker 28.3
- containerd 2.1.4
- cri-o 1.33.4
- cni-plugins 1.8.0
- calico 3.30.3
- cilium 1.18.2
- flannel 0.27.3
- kube-ovn 1.12.21
- kube-router 2.1.1
- multus 4.2.2
- kube-vip 0.8.0
- cert-manager 1.15.3
- coredns 1.12.0
- ingress-nginx 1.13.3
- argocd 2.14.5
- helm 3.18.4
- metallb 0.13.9
- registry 2.8.1
- aws-ebs-csi-plugin 0.5.0
- azure-csi-plugin 1.10.0
- cinder-csi-plugin 1.30.0
- gcp-pd-csi-plugin 1.9.2
- local-path-provisioner 0.0.32
- local-volume-provisioner 2.5.0
- node-feature-discovery 0.16.4
v2.28.1
Changes by Kind
Bug or Regression
- Add argocd_install_checksum: str, to define the checksum of
argocd_install_url(#12467, @k8s-infra-cherrypick-robot) - Add missing addresses in kube-apiserver certificate SAN. (#12505, @k8s-infra-cherrypick-robot)
- Fix Cilium installation issues (caused by templating syntax errors) when certain non-default features (encryption, etc.) are enabled (#12283, @k8s-infra-cherrypick-robot)
- Fix Hubble-Relay peer discovery in clusters using non-default cluster name by properly configuring clusterDomain in Cilium Helm values (#12374, @k8s-infra-cherrypick-robot)
- Fix cilium installation role to render cilium_config_extra_vars into helm values (#12338, @k8s-infra-cherrypick-robot)
- Fix invalid PodSecurity admission configuration when
kube_pod_security_use_default: false(#12478, @k8s-infra-cherrypick-robot) - Fix the Cilium cluster, which is upgraded from 2.27 to 2.28 will break
Fix helm release re-use message when installing repeatedly (#12324, @k8s-infra-cherrypick-robot) - Fix the issue of etcd node addition failure caused by incorrect ETCD_INITIAL_CLUSTER configuration. (#12352, @k8s-infra-cherrypick-robot)
- Fix(kubeadm): Conditionally add --skip-phases flag for v1.32.0+ (#12354, @k8s-infra-cherrypick-robot)
- Fix: When running
./manage-offline-container-images.sh registerwith using Podman, getting the image_id fails and the script is interrupted. (#12316, @k8s-infra-cherrypick-robot) - Fixed cilium_enable_bgp_control_plane config (#12432, @k8s-infra-cherrypick-robot)
- Fixes a syntax error that made the '_bgp_config' an 'AnsibleUnsafeText' instead of a 'dict', which caused the "Calico | Process BGP Configuration" step to fail (#12394, @k8s-infra-cherrypick-robot)
Components
- kubernetes 1.32.8
- etcd 3.5.22
- docker 28.0
- containerd 2.0.6
- cri-o 1.32.0
- cni-plugins 1.4.1
- calico 3.29.5
- cilium 1.17.7
- flannel 0.22.0
- kube-ovn 1.12.21
- kube-router 2.1.1
- multus 4.1.0
- weave 2.8.7
- kube-vip 0.8.0
- cert-manager 1.15.3
- coredns 1.11.3
- ingress-nginx 1.12.1
- argocd 2.14.5
- helm 3.16.4
- metallb 0.13.9
- registry 2.8.1
- aws-ebs-csi-plugin 0.5.0
- azure-csi-plugin 1.10.0
- cinder-csi-plugin 1.30.0
- gcp-pd-csi-plugin 1.9.2
- local-path-provisioner 0.0.24
- local-volume-provisioner 2.5.0
- node-feature-discovery 0.16.4
v2.27.1
Changes by Kind
Feature
- The
external_cloud_providersupportmanualoption lets users install the cloud controller manager themselves. (#11884, @k8s-infra-cherrypick-robot) - Upgrade ingress-nginx to version
v1.12.1to resolve critical vulnerabilities (CVE-2025-1974 and others) and webhook certgen tov1.5.2. (#12080, @tico88612)
Documentation
- Fix sample inventory for the reserved resource (#11922, @k8s-infra-cherrypick-robot)
Failing Test
- Fix CI by exclude the
.ansiblein.ansible-lint
Removectr image pullworkaround for nerdctl (#11956, @k8s-infra-cherrypick-robot)
Bug or Regression
- Add support for control plane reconfiguration on upgrades
Add support for kubeadm-config v1beta4UpgradeConfiguration.applyandUpgradeConfiguration.node
Usekubeadm upgrade nodeduring secondary control plane node upgrades (#12015, @chadswen) - Fix coredns deployment with
coredns_pod_disruption_budget: trueorenable_nodelocaldns_secondary(#11957, @k8s-infra-cherrypick-robot) - Fix: When running
./manage-offline-container-images.sh registerwith using Podman, getting the image_id fails and the script is interrupted. (#12314, @k8s-infra-cherrypick-robot) - Install symlinks parroting as other control plane nodes etcd certificates (and key) on all control plane nodes, to make kubeadm works (#12192, @k8s-infra-cherrypick-robot)
- Make fallback_ip cacheable in facts (#12182, @guoard)
- [calico] Fix kubecontrollersconfigurations list permission (#12039, @k8s-infra-cherrypick-robot)
Components
- kubernetes v1.31.9
- etcd v3.5.21
- docker v26.1
- containerd v1.7.27
- cri-o v1.31.6
- cni-plugins v1.4.1
- calico v3.29.4
- cilium v1.15.9
- flannel v0.22.0
- kube-ovn v1.12.21
- kube-router v2.0.0
- multus v3.8
- weave v2.8.7
- kube-vip v0.8.0
- cert-manager v1.15.3
- coredns v1.11.3
- ingress-nginx v1.12.1
- krew v0.4.4
- argocd v2.11.0
- helm v3.16.4
- metallb v0.13.9
- registry v2.8.1
- cephfs-provisioner v2.1.0-k8s1.11
- rbd-provisioner v2.1.1-k8s1.11
- aws-ebs-csi-plugin v0.5.0
- azure-csi-plugin v1.10.0
- cinder-csi-plugin v1.30.0
- gcp-pd-csi-plugin v1.9.2
- local-path-provisioner v0.0.24
- local-volume-provisioner v2.5.0
- node-feature-discovery v0.16.4
v2.28.0
Announcement
Urgent Upgrade Notes
(No, really, you MUST read this before you upgrade)
- Action required
Krew installation support is removed (#11824, @VannTen) - Action required
You should remove the leading 'v' of all explicit version of components deployed by kubespray (most notablykube_version) (#11890, @VannTen) - Action required
etcd_kubeadm_enabled(was deprecated) is removed. You should remove it from your inventory (#11901, @VannTen) gateway_api_experimental_channelis deprecated, please usegateway_api_channeland setexperimental. (#11763, @tico88612)
Changes by Kind
Feature
- Add Kubernetes 1.32.x hash (#12161, @tmurakam) (#11885, @yankay) (#12003, @mzaian) (#12052, @0ekk)
- Add containerd 2.0.x hash (#11845, @mzaian) (#12011, @mzaian)
- Update runc binary to v1.2.4
Set containerd_limit_open_file_num to 1048576 so it's configurable. (#11845, @mzaian) - Update runc binary to v1.2.5 (#12011, @mzaian)
- Make nerdctl 2.0.3 default (#11913, @mzaian)
- Add deploy_coredns: bool (true by default), to let kubespray deploy or not coredns in kube-system (#12218, @ant31)
- Add option
ubuntu_stop_unattended_upgradesto stop Ubuntu unattended upgrades (#12174, @0ekk) - Add support for
ranges:(startβstop or single start) as an additional way to define Cilium LoadBalancer IP pools, alongside the existingcidrs:field. (#12140, @Kimcheolhui) - Adds the script controb/offline/upload2artifactory.py for offline environments. (#11886, @bbaassssiiee)
- ArgoCD updated to version 2.14.5 to maintain compatibility with Kubernetes version 1.31. (#12041, @farshadasadpour)
- Automatically publish ingress-nginx service address if manual address is not specified and ingress-nginx is not using host network (#11879, @ThisIsQasim)
- Bump node-local-dns (k8s-dns-node-cache) image (#11981, @sathieu)
- Cilium CNI installation replaces Jinja template with Cilium CLI
cilium_agent_custom_argsandcilium_operator_custom_argsare deprecated, please usecilium_agent_extra_argsandcilium_operator_extra_args.
cilium_identity_allocation_modedefault change tocrd.
cilium_enable_host_legacy_routingdefault change tofalse.
Add CIlium hubble export advanced flow log settings (cilium_hubble_export_file_max_backups,cilium_hubble_export_file_max_size_mb,cilium_hubble_export_dynamic_enabledandcilium_hubble_export_dynamic_config_content)
Deprecatedcilium_ipsec_node_encryption, replace it withcilium_encryption_node_encryption(#12101, @tico88612) - Default etcd snapshot count to 10000 (#11997, @ErikJiang)
- Enable_dual_stack_networks deprecated, refact network stack with separate ipv4 and ipv6 (#11953, @borislitv)
- Ensure metrics port exists for nodelocaldns/nodelocaldns-second daemonsets (#11998, @Rickkwa)
- Fix cilium network plugin config issue deploying cilium 1.17 (#11986, @pedro-peter)
- For RHEL hosts, checking for subscription status timeout after
rh_subscription_check_timeout(default to 3 minutes) (#12115, @VannTen) - Gateway API can be brought forward before the CNI installation. (#12189, @tico88612)
- Improve ntp package conflict handling (#12212, @ErikJiang)
- Increase the control plane memory requirement to 2GB (#11864, @yankay)
- Network: Fix calico-kube-controller can't list the tiers resources (#12169, @cyclinder)
- Setting up a Docker image service for offline installation on a Mac (#11960, @diguage)
- Support containerd registry mirror certificate configuration (#11857, @KubeKyrie)
- Support kube-proxy nftables mode (#12060, @yankay)
- Terraform upcloud: Add possibility to setup cluster using nodes with no public IPs (#11696, @Xartos)
- Terraform: Added support for UpCloud routers and gateways (#11386, @Xartos)
- The
external_cloud_providersupportmanualoption lets users install the cloud controller manager themselves. (#11883, @tico88612) - Tolerations of cilium-operator deployments can be defined using the cilium_operator_tolerations group_var (#12200, @felipe88alves)
- Update default crio capabilities to allow rancher to start (#11989, @jvkassi)
- Update CI test from AlmaLinux8 to AlmaLinux9 (#11889, @yankay)
- Update kube-vip to v0.8.9 (#11983, @sathieu)
- Upgrade OpenStack Cloud Controller Manager to v1.32.0 (#12121, @tico88612)
- Upgrade ingress-nginx to version
v1.12.1to resolve critical vulnerabilities (CVE-2025-1974 and others) and webhook certgen tov1.5.2. (#12075, @farshadasadpour) - Upgrade kube-router to 2.1.1 (#12066, @VannTen)
- Upgrade load balancers image version to Nginx 1.27, Haproxy 3.1. (#11928, @guoard)
- Upgrade the default Docker version to 28.0 (#12070, @tico88612)
- Users can now configure
hubble-export-file-max-backupsandhubble-export-file-max-size-mbthrough the Kubespray inventory. (#12072, @ErmolenkoMaxim) - [calico] Update default calico to v3.29.2 (#12012, @mzaian)
- [kubernetes/control-plane] Added support for structured AuthorizationConfiguration files. (#11852, @chadswen)
Documentation
- Fix documentation for offline usage by adding the 'v' prefix in download urls (#12166, @tmurakam)
- Fix path to facts.yml in node facts refresh section (#12177, @guoard)
- Fix sample inventory for the reserved resource (#11895, @anshuman-agarwala)
- No longer reserve outdated cephfs-provisioner installation and documentation (#12113, @tico88612)
- No longer reserve outdated rbd-provisioner installation and documentation (#12114, @tico88612)
- Our CRI-O default capabilities remove
NET_RAWandSYS_CHROOT. (#12018, @tico88612)
Failing Test
- Add
dns_autoscaler_affinityand remove in-place values. (#12165, @tico88612) - Fix CI by exclude the
.ansiblein.ansible-lint
Removectr image pullworkaround for nerdctl (#11948, @yankay)
Bug or Regression
- Add support for control plane reconfiguration on upgrades
Add support for kubeadm-config v1beta4UpgradeConfiguration.applyandUpgradeConfiguration.node
Usekubeadm upgrade nodeduring secondary control plane node upgrades (#12015, @chadswen) - Enable NRI by default on containerd (following containerd defaults) (#12152, @ShinyaIshitobi)
- File download.url's are masked unless the extra var unsafe_show_logs is true. (#11959, @bbaassssiiee)
- Fix a bug where
kubeadm_certificate_keywas not defined if control plane nodes were not in correct order (#11875, @Xartos) - Fix a bug where custom TCP/UDP ports were not exposed by the ingress-nginx-controller container and service. (#11850, @commx)
- Fix broken calico Typha template when using both
calico_ipam_host_localandtypha_secure(#11917, @c-romeo) - Fix broken dhclient hooks when using resolvconf (#11946, @kyrbrbik)
- Fix control plane pods deletion with proper shell quoting (#11943, @iptizer)
- Fix coredns deployment with
coredns_pod_disruption_budget: trueorenable_nodelocaldns_secondary(#11952, @RaulButuc) - Fix hubble-ui deployment to not renders tls volume when the
cilium_hubble_tls_generateoption not configured. (#12143, @atobaum) - Fix scale.yml problems with cached IP facts (#12020, @0ekk)
- Fix: Using the ./manage-offline-container-images.sh register command does not create a new container but registers the image in the existing container registry. (#11964, @DearJey)
- Fix: arm64 checksums for youki and kata-containers (#12173, @ErikJiang)
- Fix: missing 'v' prefix in offline image tags (#12086, @ErikJiang)
- Fix: prevent kubeadm to override coredns configuration/deployment on upgrade (#12028, @sathieu)
- Fixed an issue where the second and subsequent parameters in
kubelet_cpu_manager_policy_optionswere ignored due to incorrect indentation. (#12123, @HoKim98) - Fixed kube-vip to use
kube-vip/kube-vip-iptablesimage instead ofkube-vip/kube-vipwhenlb_fwdmethodorkube_vip_lb_fwdmethodis set tomasquerade(#12145, @aviral-agarwal) - Install symlinks parroting as other control plane nodes etcd certificates (and key) on all control plane nodes, to make kubeadm works (#12181, @VannTen)
- Kubelet-csr-approver moves to regular application installation (#12141, @tico88612)
- New Boolean default variable
leave_etc_backup_files: true, set tofalsefor uncluttered /etc directory on target nodes. (#11937, @bbaassssiiee) - [calico] Fix kubecontrollersconfigurations list permission (#12035, @darkobas2)
Other (Cleanup or Flake)
- Binary checksums are no longer overridable from inventories or host facts (#12234, @VannTen)
- Calico-node pods no longer have a cpu limit by default (#11914, @VannTen)
- Enhance safety and validation mechanisms in the node removal process (#12085, @farshadasadpour)
- Heketi playbook (contrib) is removed. (#12091, @VannTen)
- Kubectl bash completion and alias available for Suse operation systems family (#11860, @noama-nv)
- Kubespray-defaults role is renamed to kubespray_defaults (#12202, @VannTen)
- Remove contrib/kvm-setup and contrib/mitogen. (#12093, @VannTen)
- Rename role bootstrap-os to bootstrap_os (#12203, @VannTen)
- Update KUBESPRAY_VERSION for v2.27.0 (#11854, @yankay)
- Update containerd.options key name (#12170, @flpanbin)
- Upgrade CI for openSuse 15.6 (#12074, @yankay)
Component versions
- kubernetes 1.32.5
- etcd 3.5.16
- docker 28.0
- containerd 2.0.5
- cri-o 1.32.0
- cni-plugins 1.4.1
- calico 3.29.3
- cilium 1.17.3
- flannel 0.22.0
- kube-ovn 1.12.21
- kube-router 2.1.1
- multus 4.1.0
- weave 2.8.7
- kube-vip 0.8.0
- cert-man...
v2.27.0
Urgent Upgrade Notes
No, really, you MUST read this before you upgrade
- Action required
Changekubeadm_patchesformat to use an array of inline patch instead of patch files.
See the example for new format. (#11521, @VannTen) - Action required
Removes the generation of static tokens for every node in the cluster whenkube_token_auth: true(#11567, @VannTen) - Action required
Thekubelet_node_{config_extra_args,custom_flags}are removed. Usekubelet_{config_extra_args,custom_flags}in<your_inventory>/group_vars/kube_node.yml.
The{kube,system}_master_{cpu,memory,ephemeral-storage,pid}are removed. Use the{kube,system}_{cpu,memory,ephemeral-storage,pid}variables in<your_inventory>/group_vars/kube_control_plane.yml.kubelet_custom_flags` can no longer be a string, an array is required. (#10643, @VannTen) - Action required
k8s_clustergroup is now automatically defined, it can be removed from your inventory if you're not using it for group_vars (#11559, @VannTen) - Action required
kubeadm_ignore_preflight_errorsis introduced to ignore specific preflight checks from kubeadm. The previous was effectivelyall, so some errors might surface during upgrade, in which cases, users should add the ones they choose to ignore to that variable. (#11710, @VannTen)
Container-Managers
- Containerd upgrade to 1.7.24 (#11833, @tico88612)
Add hashes containerd 1.7.23 (#11642, @robertvolkmann)
Add hashes containerd 1.7.22 default (#11554, @foobaar) - Runc upgrade to v1.2.3 (#11833, @tico88612)
Runc add version v1.1.14 (#11574, @janosbabik)
API Change
- If you use CRI-O and want to keep
runcas your container default runtime when you upgrade cluster, you must setrunc_enable: trueandcrio_default_runtime: "runc".
Make CRI-O's default runtime configurable
CRI-O v1.31 default runtime change to crun
Crun upgrade to 1.17
Skopeo upgrade to v1.16.1 (#11601, @tico88612)
Feature
- Make Kubernetes v1.31.4 default
Add hashes for Kubernetes 1.31.4, 1.30.8 and 1.29.12 (#11828, @tico88612)
Add hashes for Kubernetes 1.31.3, 1.30.7 and 1.29.11 (#11737, @tico88612)
Add hashes for Kubernetes 1.31.2, 1.30.6 and 1.29.10 (#11662, @robertvolkmann)
Add hashes for Kubernetes 1.31.1 and 1.31.0 (#11533, @philipsabri)
Add hashes for kubernetes 1.29.8, 1.29.9, 1.30.5 (#11581, @DirkTheDaring) - Add CI for openeuler 24.03
Add CI Image for openeuler 24.03, 22.03 (#11689, @yankay) - Add ResourceQuota AdmissionController plugin Configuration (#11814, @chadswen)
- Add a new CRI-O
crio_rootvariable (#11692, @toliger) - Add external Oracle cloud infrastructure cloud controller manager (#11378, @tico88612)
- Add optional support for Host Firewall and PolicyAuditMode features in Cilium (#11230, @ledroide)
- Add support Fedora 39/40 (#11573, @tico88612)
- Add support to use existing fips with terraform OpenStack (#11558, @anders-elastisys)
- Add the support of network isolation configuration in Multus. (#11605, @Sispheor)
- Added support for using ntpsec (#11665, @davidumea)
- Adds
ingress_nginx_service_annotationsvariable to allow setting annotations for ingress-nginx controller service (#11544, @ThisIsQasim) - Adds
nodelocaldns_additional_configsvariable (#11657, @0x4c6565) - Allow disabling cilium hubble-ui using
cilium_enable_hubble_uivariable (#10939, @pedro-peter) - Allow to skip network configuration by setting kube_network_plugin value to
none(#11844, @ant31) - Configuration can now be supplied to
ImagePolicyWebhookandPodNodeSelectoradmission plugins (#11471, @VannTen) - Feat(calico): add support for numAllowedLocalASNumbers on bgppeers per node definition (#11570, @mirwan)
- Feat: Kubeadm config API support v1beta4 (#11674, @tico88612)
- Iproute is installed before gathering facts (needed for getting
ansible_default_ipv4) (#11816, @0ekk) - Partial Support of Cilium v1.16+ - kube-proxy replacement var changes
Add optional support for configuring BGP Control Plane, IP Load Balancer Pools , Legacy BGP Peer Config v1 and BGP Config v2 features in Cilium (#11620, @logicsys) - [cilium] Make cilium 1.15.9 default (#11593, @foobaar)
- Make cri-dockerd log level configurable (#11646, @mirwan)
- Remove support Fedora 37/38 (#11600, @tico88612)
- Reset operation: remove /var/log/containers and disable service auto-boot, make sure that multi-user.target.wants is deleted. (#11501, @leeonfu)
- Support Configuring EncryptionAlgorithm in Kubeadm v1beta4 (#11757, @ErikJiang)
- Update crictl to version v1.31.1 for Kubernetes 1.31
Update crictl to version v1.30.1 for Kubernetes 1.30 (#11661, @robertvolkmann) - Update multus to v4.1.0 (#11434, @ThisIsQasim)
- Upgrade CoreDNS version to v1.11.3 (#11653, @tico88612)
- Upgrade OpenStack Cloud Controller Manager to v1.31.1 (#11738, @tico88612)
- Upgrade pause container to 3.10 (#11695, @tico88612)
- [calico] Update default calico to v3.29.1 (#11798, @mzaian)
- [cert-manager] upgrade to v1.15.3 (#11668, @tico88612)
- [cri-o] Switch binaries to libexecdir
Update youki version to 0.4.1 to fix ci. (#11584, @yankay) - [etcd] Default version to 3.5.16 for 1.28, 1.29, 1.30, 1.31 (#11572, @janosbabik)
- [helm] Upgrade to v3.16.4, add 3.16.x checksum (#11832, @tico88612)
- [ingress-nginx] upgrade controller to version 1.12.0 (#11846, @mzaian)
- [need notice] update containerd
max_container_log_line_sizedefault value to 16384 (#11585, @KubeKyrie) - [nerdctl] Default version to 1.7.7 (#11575, @janosbabik)
Documentation
- No longer support in-tree cloud provider, please delete or write
externalto thecloud_providervariable. (#11633, @tico88612) - Remove inventory_builder scripts and contrib/dind (#11748, @VannTen)
- Update
dns-stack.mdreference indocs/ansible/vars.md(#11745, @emmanuel-ferdman)
Failing Test
- Fix CI: elastx cleanup security groups (#11411, @yankay)
- Update rockylinux9 base image to 9.5 (#11744, @yankay)
Bug or Regression
- Action required
Running kubespray with --limit without cached facts is no longer supported. Improves the scaling for large clusters. (#11598, @VannTen) - Always copy cert generation script to first etcd to pick up fixes on existing clusters (#11612, @VannTen)
- Fix Cilium agent permission can't read loadbalancerippools and secrets (#11466, @foobaar)
- Fix calico dual stack installation when using
ipandip6. (#11770, @VannTen) - Fix collection usage for calico and other configuration depending on .sh and .conf files in Kubespray (#11707, @VannTen)
- Fix format of kubeadm-config v1beta4 (#11709, @VannTen)
- Fix kube-vip container securityContext (#11647, @KubeKyrie)
- Fix openEuler system packages installation (#11688, @VannTen)
- Fix pretty-printing (in kubectl) of nodelocaldns and coredns configmap when using
dns_upstream_forward_extra_optswith an empty value option. (#11694, @VannTen) - Fix spurious failure with 'localhost' when using
scale.yml --limit <some nodes>(#11817, @VannTen) - Fix task naming in bootstrap-os (#11714, @ErikJiang)
- Fix terraform.py on python >=3.12 (#11773, @enrico9034)
- Fix the check for cached data when using --limit (#11693, @VannTen)
- Fix the usage of --limit when using legacy groups (#11577, @VannTen)
- Fix usage of admission plugins configuration. (#11779, @VannTen)
- Fix using the default network manager in reset.yml (#11678, @KubeKyrie)
- Fix: cannot stop & remove all cri containers via remove_node.yml (#11631, @tico88612)
- Fixed: VSphere CSI and CPI drivers and are now retrieved from registry.k8s.io instead of gcr.io, as they have been deleted from the latter. Only a few recent versions are available in the new repository; if you have pinned
vsphere_csi_controller,vsphere_csi_driver_image_tagorvsphere_syncer_image_tagto a version older thanv3.1.2, please check if that version is available from the new repository. The same goes forexternal_vsphere_cloud_controller_image_tagwhich can no longer belatest, and should align with the running version of Kubernetes. It now defaults tov1.31.0. (#11564, @luringens) - HA etcd cluster keeps quorum during upgrades. (#11677, @VannTen)
- Kubeadm images (kube-controller-manager,kube-scheduler,kube-apiserver,kube-proxy) are properly downloaded, including when using the download cache. (#11741, @VannTen)
- Make sure kubespray-defaults can be executed successfully by executing bootstrap-os first (#11441, @huangkevin404)
- Make upcloud csi_driver use the correct pull secret (#11597, @VannTen)
- Modifies Helm parameters
waitandatomicto be set tofalsewhen usingkube_network_plugin=cnito prevent deployment issues withkubelet-csr-approver. (#11704, @M-JavadHeydarpour) - Remove invalid extraArgs entry and update template file reference (#11703, @agravgaard)
- Update calico-nopde template and remove flexvol-driver initContainer (#11634, @KubeKyrie)
- Use correct version for community.general collection (#11724, @VannTen)
Other (Cleanup or Flake)
- Cleanup older terminology, replace "master" with "control plane" (#11394, @bogd)
- Drop support for Kubernetes 1.28.x minimum version now is 1.29.x
Drop support for CRI-O 1.28.x minimum version now is 1.29.x (#11609, @yankay) - Fix
roles/download/tasks/download_file.ymltask name typo (#11684, @dmncmn) - Optimize CA cert hash calculation with community.crypto (#11758, @ErikJiang)
- Remove
pip install .support and rpm spec file (#11760, @VannTen) - Replace deprecated unarchive.copy with unarchive.remote_src (#11207, @Payback159)
- Update KUBESPRAY_VERSION for v2.26.0 (#11511, @yankay)
containerd_use_config_pathis removed as kubespray now always use containerdconfig_pathconfiguration. (#11755, @VannTen)
v2.25.1
Changes by Kind
Deprecation / Removal
- Deprecating support for Centos7 and Weave ; they are not tested anymore (#11428, @k8s-infra-cherrypick-robot)
- Remove Debian 10 support. (#11429, @k8s-infra-cherrypick-robot)ra-cherrypick-robot)
Feature
- Default to Kubernetes v1.29.10 (#11666, @enneitex)
- Add hashes for Kubernetes v1.29.9 (#11599, @robertvolkmann), Add hashes for Kubernetes v1.29.7 (#11408, @mzaian), Add hashes for Kubernetes v1.29.6 (#11355, @tico88612)
- Default to etcd v3.5.16 (#11576, @janosbabik)
- [ingress-nginx] upgrade controller to version 1.11.2 (#11464, @k8s-infra-cherrypick-robot)
- Allow to run kubespray with an empty kube_node group, to provision only the control plane (#11249, @k8s-infra-cherrypick-robot)
- CentOS 7 yum repo baseurl update (#11365, @k8s-infra-cherrypick-robot)
- Check CentOS-Base.repo exists for CentOS 7 (#11438, @k8s-infra-cherrypick-robot)
- User has a possibility to fix nodePort of ingress-nginx service with property in addons.yaml (#11339, @k8s-infra-cherrypick-robot)
Applications
- [helm] Upgrade to v3.14.4 (#11599, @robertvolkmann)
Network
- [calico] Update default calico to v3.27.4
[calico] Fix high cpu load due to XDP program in iptables (#11476, @mzaian)
Container-Managers
- [containerd] Default to v1.7.22
[nerdctl] Upgrade to 1.7.7
[runc] Upgrade to v1.1.14 (#11576, @janosbabik)
Bug or Regression
- Always copy cert generation script to first etcd to pick up fixes on existing clusters (#11616, @k8s-infra-cherrypick-robot)
- Ingress-nginx-controller admission service is automatically created when
ingress_nginx_webhook_enabled: true(#11332, @k8s-infra-cherrypick-robot) - Fix: cannot stop & remove all cri containers via remove_node.yml (#11638, @k8s-infra-cherrypick-robot)
v2.24.3
Changes by Kind
API Change
- Default to kubernetes v1.28.14
Default to etcd v3.5.16
Default to containerd v1.7.22
Default to cri-o v1.28.10
Default to nerdctl 1.7.7
Default to runc v1.1.14 (#11516, @VannTen)
Feature
- User has a possibility to fix nodePort of ingress-nginx service with property in addons.yaml (#11361, @k8s-infra-cherrypick-robot)
- Check CentOS-Base.repo exists for CentOS 7 (#11546, @tico88612)
Other (Cleanup or Flake)
v2.26.0
Deprecation / Removal
- Deprecating support for Centos7; they are not tested anymore (#11344, @ant31)
- Remove Debian 10 support. (#11347, @tico88612)
- Remove the
kubeadm_versionwhich is always equal tokube_version(#11473, @VannTen) - Drop support for Kubernetes 1.27.x minimum version now is 1.28.x (#11221, @mzaian)
- if you were previously only setting
serializeImagePulls: falseto have unlimited parallel pulls, you will need to setkubelet_max_parallel_images_pullsto a suitable value instead (#11094, @tu1h)
Feature / Major Changes
- Make kubernetes v1.30.4 default (#11455, @kokyhm)
- Add hashes for Kubernetes v1.30.3 default (#11391, @tico88612), Add hashes for Kubernetes v1.30.2 default (#11343, @tmurakam), Add hashes for Kubernetes 1.30.0, 1.30.1 and 1.30.2 (#11261, @tmurakam), Add hashes for kubernetes 1.29.7, 1.28.[11-12] (#11407, @mzaian)
- Add option
ubuntu_kernel_unattended_upgrades_disabledto control unattended-upgrades for Linux kernel and all packages start with linux- on Ubuntu (#11296, @tu1h) - Added option to configure dependencies for kubelet.service (#11297, @ledroide)
- Adds the possibility to add extra arguments to the various containers in the cinder-csi plugin.(#11169, @Payback159)
- Allow to run kubespray with an empty kube_node group, to provision only the control plane (#11248, @VannTen)
- CentOS 7 yum repo baseurl update (#11360, @tico88612)
- Check CentOS-Base.repo exists for CentOS 7 (#11402, @tu1h)
- Check if peers is defined when peering with routers (#11259, @ehsan310)
- OpenStack Cloud Controller Manager upgrade to 1.30.0 (#11358, @tico88612)
- Rename
systemdmodule tosystemd_service(#11396, @tu1h) - User has the ability to configure calico-kube-controllers log level (#11335, @mirwan)
- User has the ability to configure local_volume_provisioner log level (#11336, @mirwan)
- User has the ability to configure netchecker components log levels (#11334, @mirwan)
- You can now disable installing OS dependencies using system's package manager by skipping
system-packagestag. (#10872, @hedayat) kubelet_max_parallel_image_pullsrepresents the maximum number of image pulls in parallel (#11094, @tu1h)- Update reset task to support Tencent OS (
reset_restart_network_service_name) (#11459, @KubeKyrie) - Add conditional checking on ubuntu kernel unattended_upgrades disabling (#11479, @tu1h)
Applications
- Bump Cinder CSI Plugin to v1.30.0 (#11374, @tico88612)
- Bump upcloud csi driver to v1.1.0 in order to enable csi volume snapshots. (#11303, @Elias-elastisys)
- User has a possibility to fix nodePort of ingress-nginx service with property in addons.yaml (#11310, @mochizuki875)
- Update kube-vip to v0.8.0 (#11156, @jisnardo)
- [cert-manager] upgrade to v1.14.7 (#11341, @tico88612)
- [cert-manager] add support v1.13.6 (#11279, @tico88612)
- [ingress-nginx] upgrade controller to version 1.11.2 (#11463, @mzaian)
- [helm] Upgrade to v3.15.4, add 3.15.x, and drop 3.13.x (#11486, @yankay)
- Add support for LB in UpCloud private zone (#11260, @davidumea)
- Bump UpCloud terraform module to v5.6.0
UpCloud servers specify server groups to be apart of, eliminates manual rescheduling. (#11311, @robinAwallace) - Update node-feature-discovery to v0.16.4 (#11250, @mzaian)
- Allow for configuring etcd progress notify interval and default set to 5s (#11499, @liuxu623)
- Support Gateway API CRDs install (#11376, @tico88612)
- Increase ansible timeout to 300 (#11354, @rptaylor)
Network
- [calico] Change calico default version to v3.28.1, add v3.28.0 and checksum , Update calico apiserver deployment to use new readiness probe (#11234, @ehsan310)
- [calico] add calico support v3.27.4 to fix high cpu load due to XDP program in iptables (#11476, @ehsan310)
- Add cilium_hubble_event_buffer_capacity & cilium_hubble_event_queue_size vars (#10943, @pedro-peter)
- [network] bump cni version to v1.4.0 (#10698, @cyclinder)
- Change weave CNI to community version and upgrade to the latest version (2.8.7) (#11228, @tico88612)
- [kube-ovn] update to v1.12.21 (#11445, @oilbeater)
Container-Managers
- [containerd] Make containerd 1.7.21 default (#11478, @yankay)
- [containerd] added debug config variables (#11080, @spnngl)
- [containerd] fixes wrong templating for tracing config (#11372, @ugur99)
[runc] Upgrade to v1.1.13 (#11413, @mzaian) - Update docker cli version 26.1.2 (#11291, @ErikJiang)
Documentation
Bug or Regression
- Delete
/etc/NetworkManager/conf.d/dns.confon reset. (#11440, @HoKim98) - Fix Hetzner kubernetes group names (#11232, @jmaccabee13)
- Fix: skip multus when not defined (#10934, @darkobas2)
- Ingress-nginx-controller admission service is automatically created when
ingress_nginx_webhook_enabled: true(#11309, @mochizuki875) - Provide missing advertise-address flag to kube-apiserver (#11387, @derselbst)
- Update reset task to support Kylin OS (
reset_restart_network_service_name) (#11406, @KubeKyrie) - Updated indentation in cni-kube-ovn.yml.j2 (L658) (#11357, @sanshah1211)
- Fix CI with fail docker pull in gitlab runner by change DOCKER_HOST (#11315, @yankay)
- Fix etcd not starting up when using a custom access address (#11388, @derselbst)
- Fix the Auto Bump PR is blocked by the label
do-not-merge/release-note-label-neededby adding dependabotrelease-note-nonelabel. (#11256, @yankay) - Fix kube_reserved so it only controls kubeReservedCgroup . (#11367, @rptaylor)
- Disables reconfiguring the cluster during upgrade (remove --config option from kubeadm upgrade apply) (#11352, @tmurakam)
- Fix error in boostrap-os when git does not handle symlinks (#11508, @VannTen)
- Fix static kube-apiserver advertise address based on first control plane (#11457, @Seljuke)
- Fix incorrect member matching when removing etcd nodes (#11488, @ErikJiang)
- Fix double pop of access_ip (#11435, @rptaylor)
- Fix use super-admin.conf for kube-vip on first master when it exists to support initial k8s v1.29+ installation with kube-vip enabled (#11422, @Seljuke)
Other (Cleanup or Flake)
- Contrib playbooks are no longer included in the ansible kubespray collection (#11239, @VannTen)
- Reduced required python packages in requirements.txt (#11199, @itayporezky)
- Fix openstack cleanup by change the delete security_group order (#11299, @yankay)
- RHEL 7, Centos 7 and derivatives are no longer supported. (#11246, @VannTen)
- Use TasksMask=infinity on ostree systems for docker systemd service (#11493, @VannTen)
Supported Components
- Core
- kubernetes v1.30.4
- etcd v3.5.12
- docker v26.1
- containerd v1.7.21
- cri-o v1.30.3 (experimental: see CRI-O Note. Only on fedora, ubuntu and centos based OS)
- Network Plugin
- cni-plugins v1.2.0
- calico v3.28.1
- cilium v1.15.4
- flannel v0.22.0
- kube-ovn v1.12.21
- kube-router v2.0.0
- multus v3.8
- weave v2.8.7
- kube-vip v0.8.0
- Application
- cert-manager v1.14.7
- coredns v1.11.1
- ingress-nginx v1.11.2
- krew v0.4.4
- argocd v2.11.0
- helm v3.15.4
- metallb v0.13.9
- registry v2.8.1
- Storage Plugin
- cephfs-provisioner v2.1.0-k8s1.11
- rbd-provisioner v2.1.1-k8s1.11
- aws-ebs-csi-plugin v0.5.0
- azure-csi-plugin v1.10.0
- cinder-csi-plugin v1.30.0
- gcp-pd-csi-plugin v1.9.2
- local-path-provisioner v0.0.24
- local-volume-provisioner v2.5.0
- node-feature-discovery v0.16.4
Known issues
- Upgrade of clusters with external etcd can be problematic (in particular long lived clusters, as this is not reproducible on cluster created by v2.25.1) ; see #11500 (comment) and the previous discussion for details and a workaround
Notes
- Deprecating support for Centos7
- The Ansible version has been upgrade to 9.8.0
- Change weave CNI to community version https://github.com/weaveworks/weave
Maintainers
Great respect for joining maintainers π
v2.24.2
Changes by Kind
Feature
- Make kubernetes v1.28.10 default (#11269, @mzaian)
- Revert 'Support CoreDNS use host network & config CoreDNS port' (#10617, @liuxu623)
- User has a possibility to modify Service type with "ingress_nginx_service_type" property in addons. (#11330, @mochizuki875)
Bug or Regression
- Ingress-nginx-controller admission service is automatically created when
ingress_nginx_webhook_enabled: true(#11331, @mochizuki875) - Fix CentOS 7 yum repo baseurl update (#11364, @tico88612 )
Other (Cleanup or Flake)
v2.25.0
Deprecation / Removal
- Remove support for Kubernetes 1.26.x (move min version to 1.27.x) (#10817, @KubeKyrie)
- Remove documentation for removed in-tree openstack provider (#10889, @LarssonOliver)
Feature / Major Changes
- A check is introduced to fail the playbook if cgroups are not enabled on the node (#11165, @franznemeth)
- Add Calico v3.27.3 and make it default (#11141, @pomland-94)
- Add extra_vars support to vagrant setup (#10932, @VannTen)
- Add kube-vip LeaderElection variables
vip_leaseduration, vip_renewdeadline, vip_retryperiodoptions for kube-vip (#11021, @KubeKyrie) - Add new option
remove_anonymous_accessto prevent granting RBAC permissions to anonymous users. (#11016, @nicolas-goudry) - Add scheduler plugins support (
scheduler_plugins_enabledenable or disable the installation scheduler plugins /scheduler_plugins_enabled_pluginsdescribe the enabled plugins /scheduler_plugins_diabled_pluginsdescribe the disabled plugins /scheduler_plugins_plugin_configset the custom config for enabled plugins) (#10747, @tu1h) - Added a config option to filter ntp interfaces (#11066, @Pavan-Gunda)
- Adding egress IPv6 for node-local-dns queries (
k8s_allowed_egress_ipv6_ips) (#10396, @raviranjanelastisys) - Bump docker version for kylin linux (#11203, @ErikJiang)
- Bump docker version for openeuler linux (#11206, @ErikJiang)
- Update almalinux-8 base image to 8.9 (#10918, @VannTen)
- Bumping checksums and various versions (#10999, @MrFreezeex)
- Containerd: allow to configure fallback server (#10988, @sathieu)
- Docker upgrade from 24.0 to 26.1 (#11198, @tico88612)
- Download hash script: auto discover versions (#10849, @VannTen)
- Enable configuring mountOptions, reclaimPolicy and volumeBindingMode for cinder-csi StorageClasses. (#10450, @Payback159)
- Make containerd v1.7.15 default (#11083, @Payback159)
- Make kubernetes v1.28.6 default (#10810, @mzaian)
- Make kubernetes v1.29.1 default
Remove SecCompDefault feature gate from hardening configuration for kubernetes 1.29 (#10820, @tmurakam) - Make kubernetes v1.29.2 default (#10919, @mzaian)
- Make kubernetes v1.29.3 default (#11035, @mzaian)
- Make kubernetes v1.29.4 default (#11108, @mzaian)
- Make kubernetes v1.29.5 default (#11196, @mzaian)
- Metallb: added metallb_namespace variable to parameterize namespace (#11136, @oik741)
- OpenStack Cloud Controller Manager upgrade to 1.28.2 (#11174, @tico88612)
- Opensuse deployment is now tested in CI. (#11159, @VannTen)
- Add
selinux-ngrepo in Amazon Linux to installcontainer-selinux(#11182, @yankay) - Add CI Image for Ubuntu 24.04 (#11167, @yankay)
- Allows .vagrant folder location to be configured (#10718, @kri5)
- Prevent nodelocaldns to be OOM-killed (#11056, @sathieu)
- Support Node Feature Discovery (#10861, @yankay)
- Support Ubuntu 24.04 (#11132, @tico88612)
- Support following k8s version selection pause image (#10756, @my-git9)
- The variable
old_dns_domains(list) can be used for backward compatibility when changingdns_domain(#10630, @VannTen) - Update external huawei cloud controller to 0.26.6 (#10824, @dabeck)
- Update external huawei cloud controller to 0.26.8 (#11172, @dabeck)
- Update kube-vip to v0.8.0 (#11156, @jisnardo)
- Update metrics server to v0.7.0 (#10856, @mzaian)
- Updated ingress controller version to 1.9.6 (#10868, @kundan2707)
- User has a possibility to modify Service type with "ingress_nginx_service_type" property in addons. (#10925, @chrxmvtik)
- [Terraform-openstack] Added possibility to build an octavia loadbalancer for the Kubernetes Api. (#10924, @jaszil)
- [containerd] added distributed tracing config variables for containerd (
containerd_tracing_enabled,containerd_tracing_endpoint,containerd_tracing_protocol,containerd_tracing_sampling_ratio,containerd_tracing_service_name); it is disabled by default. (#11103, @ugur99) - [etcd] Default version to 3.5.12 for k8s 1.27 , 1.28 , 1.29 (#11036, @mzaian)
- Minimum ansible-core version is now 2.16.4 (#10984, @VannTen)
- Remove the archived debian apt repository when installing docker-engine (#11088, @yankay)
- Change
dependbotinterval to weekly (#11189, @yankay) - Allow specifying CPU Manager Policy options through kubelet_cpu_manager_policy_options (#11023, @derselbst)
- [kube-apiserver] added distributed tracing config variables for kube-apiserver (
kube_apiserver_tracing,kube_apiserver_tracing_endpoint,kube_apiserver_tracing_sampling_rate_per_million); it is disabled by default.
[kubelet] added distributed tracing config variables for kubelet (kubelet_tracing,kubelet_tracing_endpoint,kubelet_tracing_sampling_rate_per_million); it is disabled by default. (#10795, @ugur99)
Applications
- [argocd] update argocd to v2.11.0 (#11193, @mzaian)
- [helm] Upgrade to v3.14.2 (#10967, @cleman95)
- Bump coredns version to 1.11.1 (#10719, @batazor)
- Support CoreDNS use host network & config CoreDNS port (#10617, @liuxu623)
- Fix secondary coredns missing var (#10821, @VannTen)
- Revert "support CoreDNS use host network and config dns port (#10617)" (#11185, @VannTen)
dns_mode: coredns_dualis now tested in CI. (#10903, @VannTen)
Network
- Adds support for cilium v1.15
- Adds the option to install calico 3.27.3 (#11059, @danielfrg)
- [calico] Update default calico to v3.27.2 (#10960, @mzaian)
Container-Managers
- crictl stop container grace period, cri_stop_containers_grace_period: 0 (#10651, @krembu)
- Update the docker default version to 24.0 (#10873, @yankay)
- [Containerd] Enable by default
discard_unpacked_layersto save some space (see containerd/containerd#6295) (#10905, @VannTen) - [Nerdctl] Upgrade to version 1.7.4 (#10968, @cleman95)
- [containerd] Make containerd 1.7.13 default
[runc] Upgrade to v1.1.12 (#10862, @KubeKyrie) - [containerd] Make containerd 1.7.16 default (#11142, @mzaian)
API Change
- Make proxy protocol in Upcloud LB configurable (#10971, @davidumea)
Design
- Merge stop and remove systemd service task in reset/tasks/main.yml (#10902, @kimsehwan96)
Documentation
- Add documentation for configuring nat outgoing ipv6 (#10866, @anders-elastisys)
- Add new OpenStack Cloud for terraform (#10910, @DragomirAlin)
- BREAKING CHANGE: This script is introduced to facilitate living documentation and its administration. This leads to a restructuring in the documentation at https://kubespray.io/#/ to simplify the automatic creation of links, as the structure in the sidebar changes. (#11128, @Payback159)
- Change a task name
Ensure kube-bench parameters are setintoEnsure kubelet expected parameters are setinroles/kubernetes/preinstall/tasks/0080-system-configurations.ymlfor a clearer understanding of its operation (#11171, @kimsehwan96) - Do not disable SELinux surreptitiously (#10920, @rptaylor)
- Doc clarification: skipping patches releases is OK (#10850, @VannTen)
- Docs: vagrant-libvirt is tested in CI (#10847, @VannTen)
- Explicit private/public nature of *ip vars (#10904, @VannTen)
- Fix typo in vagrant.md (#10836, @kundan2707)
- Fix typo mistake in roles/kubernetes/control-plane/tasks/define-first-kube-control.yml (#10835, @kimsehwan96)
- Fixed typos in inventory/sample/group_vars/k8s_cluster (#10911, @arahmangulov)
- Kubespray used as a collection will have the correct collection version (#10727, @VannTen)
- Make large-deployments.md link to downloads.md (#10840, @spantaleev)
- Removed not needed graduated feature gates. (#10448, @Smidra)
- Update upgrades.md with serial=1 for rolling updates (#10837, @titansmc)
- Variable cilium_ipsec_key must be base64 encoded (#10781, @ledroide)
Bug or Regression
- Added an optional variable (
cni_bin_owner) to allow the user to set a different owner for/opt/cni/bin/and it's contents. (#10929, @Rickkwa) - Change the position of the containerd_extra_args parameter to enhance its universality. (#11013, @qcu266)
- Configure crio container runtime to use kube reserved cgroup (#11028, @pedro-peter)
- Don't overwrite changes to openstack allowed_address_pairs #10760 (#10760, @rptaylor)
- Download cache directory permissions are no longer reset recursively (#10900, @VannTen)
- Fix ClusterRole for Calico >=v1.26.x with Calico API Server installed (#11089, @RaSerge)
- Fix ansible parameter ssh_args in ansible.cfg file not work (#10981, @joy717)
- Fix boostrap for Amazon Linux (#11139, @VannTen)
- Fix crio registries config file when using slashes in the registry path (#11030, @pedro-peter)
- Fix file loss during download (#10779, @ErikJiang)
- Fix kubespray-defaults: Check for boostrap-os FQCN (#11073, @KubeKyrie)
- Fix local path provisioner image repo in sample inventory. (#11180, @tico88612)
- Fix logical error when checking for boostrap-os (#10867, @VannTen)
- Fix lsattr command error when kubelet has symbolic link (#11074, @KubeKyrie)
- Fix network manage service of Debian 12 (#11058, @KubeKyrie)
- Fix nginx controller leader election RBAC (#10913, @VannTen)
- Fix python regex matching problem when finding docker packages (#11075, @KubeKyrie)
- Fix waiting for MetalLB controller (#10858, @flxbwr)
- Fix(kubernetes): taint nodes on cluster upgrade (#10705, @maxime1907)
- Fix: config hostname as string type in kubeadmConfig rendering (#10997, @ErikJiang)
- Fixes running
recover-control-plane.ymlwith offline broken etcd nodes. (#10660, @yuha0) - Revert OCCM standard dnsPolicy to ClusterFirst to fix #10914 which was introduced with #10618 and make dnsPolicy configurable to furthermore support #10618 (#11168, @Payback159)
- Force update helm repo if ...