Skip to content

kube-proxy fails to expose NodePort because of r/o filesystem #155

@neuhalje

Description

@neuhalje

The containerised kube-proxy fails to expose services with NodePort because it cannot lock /run/xtables.lock (open /run/xtables.lock: read-only file system).
.

Version used

sudo atomic images list
...
>  registry.fedoraproject.org/f27/kubernetes-proxy     latest   68406693c322   2017-12-10 17:26   237.16 MB      ostree

Service definition

Given the following yaml:

apiVersion: apps/v1beta1
kind: Deployment
metadata:
  name: nginx-deployment
spec:
  selector:
    matchLabels:
      app: nginx
  replicas: 2
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx:1.8
        ports:
        - containerPort: 80

---
kind: Service
apiVersion: v1
metadata:
  name: my-nginx
spec:
  type: NodePort
  selector:
    app: nginx
  ports:
  - protocol: TCP
    port: 80
    targetPort: 80

kubectl

kubectl get pods

NAME                                READY     STATUS    RESTARTS   AGE
nginx-deployment-3718365652-1ht6q   1/1       Running   1          1h
nginx-deployment-3718365652-9g2jq   1/1       Running   1          1h
kubectl get service my-nginx

NAME       TYPE       CLUSTER-IP      EXTERNAL-IP   PORT(S)        AGE
my-nginx   NodePort   10.254.106.83   <none>        80:32315/TCP   1h

Expected behaviour

  • curl http://172.20.61.51:32315 should return the nginx page.

Observed behaviour

The port is not exposed.

sudo netstat -tulpen | grep proxy

tcp        0      0 127.0.0.1:10249         0.0.0.0:*               LISTEN      994        25072      797/kube-proxy
tcp6       0      0 :::10256                :::*                    LISTEN      994        24025      797/kube-proxy

Although I can connect to the ports of the container:

curl http://127.0.0.1:10249/

404 page not found

journaltctl -xe -u kube-proxy.service returns the following errors:

Failed to start in resource-only container "/kube-proxy": mkdir /sys/fs/cgroup/cpuset/kube-proxy: read-only file system
...
Failed to execute iptables-restore: failed to open iptables lock /run/xtables.lock: open /run/xtables.lock: read-only file system

# the last line is repeated every 30 seconds

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions