kubernetes
📝 This article is a translation of the original Japanese post. View original
Falling Into an Unexpected Trap with ConfigMap volumeMounts
- Version tested: kubernetes 1.20.11
You sometimes use a ConfigMap via volumeMounts in a Deployment and the like, as below.
yaml-1
1 spec:
2 containers:
3 - image: httpd
4 name: httpd
5 volumeMounts:
6 - name: configmap-volume
7 mountPath: /var/tmp/configmap
8 volumes:
9 - name: configmap-volume
10 configMap:
11 name: configmap-sample-data
Regarding the volume above, normally you cannot apply duplicates with the same name.
yaml-2
1 spec:
2 containers:
3 - image: httpd
4 name: httpd
5 volumeMounts:
6 - name: configmap-volume
7 mountPath: /var/tmp/configmap
8 volumes:
9 - name: configmap-volume
10 configMap:
11 name: configmap-sample-data
12 - name: configmap-volume
13 configMap:
14 name: configmap-sample-data
1The Deployment "test" is invalid: spec.template.spec.volumes[1].name: Duplicate value: "configmap-volume"
However, if you apply yaml-1 and then apply yaml-2, the volume with the duplicate name gets applied (!!!)
After applying, two configmap-volume entries have been generated in last-applied-configuration.
Then applying yaml-1 produces a Not found error.
1# this part is normal
2$ kubectl apply -f ~/Desktop/yaml-1.yml
3deployment.apps/test created
4
5# this one gets accepted
6$ kubectl apply -f ~/Desktop/yaml-2.yml
7deployment.apps/test configured
8
9# this becomes Not found
10$ kubectl apply -f ~/Desktop/yaml-1.yml
11The Deployment "test" is invalid:
12* spec.template.spec.containers[0].volumeMounts[0].name: Not found: "configmap-volume"
I skimmed the upstream Kubernetes issues, but it’s unclear whether this is by design or a bug (it depends on how volumes, being defined as an array, handles additions after apply).
As an investigation technique, after the first apply, run kubectl get deployment test -oyaml and expand the last-applied-configuration json inside the deployment to check it.
As is often said, Kubernetes is a declarative system, but the master copy of the declarative configuration lives in etcd, and it can be viewed via last-applied-configuration.
This is a case you might run into while experimenting with kustomize or helm, and I got well and truly stuck in it.
Verifying on minikube
You can reproduce it as follows.
1$ minikube kubectl -- apply -f <(cat <<EOF
2apiVersion: v1
3kind: Namespace
4metadata:
5 name: test
6---
7apiVersion: v1
8kind: ConfigMap
9metadata:
10 name: configmap-sample-data
11 namespace: test
12data:
13 test: "test"
14---
15apiVersion: apps/v1
16kind: Deployment
17metadata:
18 name: nginx-deployment
19 labels:
20 app: nginx
21 namespace: test
22spec:
23 replicas: 1
24 selector:
25 matchLabels:
26 app: nginx
27 template:
28 metadata:
29 labels:
30 app: nginx
31 spec:
32 containers:
33 - name: nginx
34 image: nginx:latest
35 ports:
36 - containerPort: 80
37 volumeMounts:
38 - name: configmap-volume
39 mountPath: /var/tmp/configmap
40 volumes:
41 - name: configmap-volume
42 configMap:
43 name: configmap-sample-data
44EOF
45)
46namespace/test created
47configmap/configmap-sample-data created
48deployment.apps/nginx-deployment created
49
50
51$ minikube kubectl -- apply -f <(cat <<EOF
52apiVersion: apps/v1
53kind: Deployment
54metadata:
55 name: nginx-deployment
56 labels:
57 app: nginx
58 namespace: test
59spec:
60 replicas: 1
61 selector:
62 matchLabels:
63 app: nginx
64 template:
65 metadata:
66 labels:
67 app: nginx
68 spec:
69 containers:
70 - name: nginx
71 image: nginx:latest
72 ports:
73 - containerPort: 80
74 volumeMounts:
75 - name: configmap-volume
76 mountPath: /var/tmp/configmap
77 volumes:
78 - name: configmap-volume
79 configMap:
80 name: configmap-sample-data
81 - name: configmap-volume # this one actually gets applied
82 configMap:
83 name: configmap-sample-data
84EOF
85)
86deployment.apps/nginx-deployment configured
87
88$ minikube kubectl -- -ntest get deploy nginx-deployment -oyaml
89apiVersion: apps/v1
90kind: Deployment
91metadata:
92 annotations:
93 deployment.kubernetes.io/revision: "1"
94 kubectl.kubernetes.io/last-applied-configuration: |
95 {"apiVersion":"apps/v1","kind":"Deployment","metadata":{"annotations":{},"labels":{"app":"nginx"},"name":"nginx-deployment","namespace":"test"},"spec":{"replicas":1,"selector":{"matchLabels":{"app":"nginx"}},"template":{"metadata":{"labels":{"app":"nginx"}},"spec":{"containers":[{"image":"nginx:latest","name":"nginx","ports":[{"containerPort":80}],"volumeMounts":[{"mountPath":"/var/tmp/configmap","name":"configmap-volume"}]}],"volumes":[{"configMap":{"name":"configmap-sample-data"},"name":"configmap-volume"},{"configMap":{"name":"configmap-sample-data"},"name":"configmap-volume"}]}}}}
96 creationTimestamp: "2021-11-20T14:27:43Z"
97 generation: 2
98 labels:
99 app: nginx
100...
101
102# let us look at last-applied-configuration
103{
104 "apiVersion": "apps/v1",
105 "kind": "Deployment",
106 "metadata": {
107 "annotations": {},
108 "labels": {
109 "app": "nginx"
110 },
111 "name": "nginx-deployment",
112 "namespace": "test"
113 },
114 "spec": {
115 "replicas": 1,
116 "selector": {
117 "matchLabels": {
118 "app": "nginx"
119 }
120 },
121 "template": {
122 "metadata": {
123 "labels": {
124 "app": "nginx"
125 }
126 },
127 "spec": {
128 "containers": [
129 {
130 "image": "nginx:latest",
131 "name": "nginx",
132 "ports": [
133 {
134 "containerPort": 80
135 }
136 ],
137 "volumeMounts": [
138 {
139 "mountPath": "/var/tmp/configmap",
140 "name": "configmap-volume"
141 }
142 ]
143 }
144 ],
145 "volumes": [
146 {
147 "configMap": {
148 "name": "configmap-sample-data"
149 },
150 "name": "configmap-volume" # <-- two entries with the same name can exist...
151 },
152 {
153 "configMap": {
154 "name": "configmap-sample-data"
155 },
156 "name": "configmap-volume" # <-- two entries with the same name can exist..
157 }
158 ]
159 }
160 }
161 }
162}
163
164
165# this one fails
166$ minikube kubectl -- apply -f <(cat <<EOF
167apiVersion: apps/v1
168kind: Deployment
169metadata:
170 name: nginx-deployment
171 labels:
172 app: nginx
173 namespace: test
174spec:
175 replicas: 1
176 selector:
177 matchLabels:
178 app: nginx
179 template:
180 metadata:
181 labels:
182 app: nginx
183 spec:
184 containers:
185 - name: nginx
186 image: nginx:latest
187 ports:
188 - containerPort: 80
189 volumeMounts:
190 - name: configmap-volume
191 mountPath: /var/tmp/configmap
192 volumes:
193 - name: configmap-volume
194 configMap:
195 name: configmap-sample-data
196EOF
197)
198The Deployment "nginx-deployment" is invalid: spec.template.spec.containers[0].volumeMounts[0].name: Not found: "configmap-volume"