ConfigMap の volumeMounts で思わぬ沼にはまってしまう
検証したバージョン: kubernetes 1.20.11
下記のように、Deployment などで ConfigMap を volumeMounts して利用することがある。
yaml-1
spec:
containers:
- image: httpd
name: httpd
volumeMounts:
- name: configmap-volume
mountPath: /var/tmp/configmap
volumes:
- name: configmap-volume
configMap:
name: configmap-sample-data
上記の volume なのだが、通常は同名で重複して apply することはできない。
yaml-2
spec:
containers:
- image: httpd
name: httpd
volumeMounts:
- name: configmap-volume
mountPath: /var/tmp/configmap
volumes:
- name: configmap-volume
configMap:
name: configmap-sample-data
- name: configmap-volume
configMap:
name: configmap-sample-data
The Deployment "test" is invalid: spec.template.spec.volumes[1].name: Duplicate value: "configmap-volume"
しかし、yaml-1 を apply した後、yaml-2 を apply すると、同名の volume を apply できてしまう (!!!)
apply すると last-applied-configuration としては configmap-volume が 2つ生成されている。
その後、yaml-1 を apply すると Not found のエラーが発生する
# ここは普通
$ kubectl apply -f ~/Desktop/yaml-1.yml
deployment.apps/test created
# ここが反映できてしまう
$ kubectl apply -f ~/Desktop/yaml-2.yml
deployment.apps/test configured
# Not found になる
$ kubectl apply -f ~/Desktop/yaml-1.yml
The Deployment "test" is invalid:
* spec.template.spec.containers[0].volumeMounts[0].name: Not found: "configmap-volume"
Kubernetes 本家の issue をざっと見たが、仕様なのか、あるいはバグなのか不明（volume が配列で定義されていて、apply 後の追加をどう処理しているか次第）
調査の仕方として、1回目の apply を行ったあとに kubectl get deployment test -oyaml で deployment の中の last-applied-configuration の json を展開して確認する。
よく言われるように、kubernetes は宣言的なシステムというが、宣言的な設定のマスターは etcd の中にあり、それは last-applied-configuration で閲覧することができる。
kustomize や helm で色々試していると出くわすかもしれないケースで、沼にハマってしまった。
minikube での検証
下記の方法で再現が行える。
$ minikube kubectl -- apply -f <(cat <<EOF
apiVersion: v1
kind: Namespace
metadata:
name: test
---
apiVersion: v1
kind: ConfigMap
metadata:
name: configmap-sample-data
namespace: test
data:
test: "test"
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-deployment
labels:
app: nginx
namespace: test
spec:
replicas: 1
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx:latest
ports:
- containerPort: 80
volumeMounts:
- name: configmap-volume
mountPath: /var/tmp/configmap
volumes:
- name: configmap-volume
configMap:
name: configmap-sample-data
EOF
)
namespace/test created
configmap/configmap-sample-data created
deployment.apps/nginx-deployment created
$ minikube kubectl -- apply -f <(cat <<EOF
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-deployment
labels:
app: nginx
namespace: test
spec:
replicas: 1
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx:latest
ports:
- containerPort: 80
volumeMounts:
- name: configmap-volume
mountPath: /var/tmp/configmap
volumes:
- name: configmap-volume
configMap:
name: configmap-sample-data
- name: configmap-volume # これが適用できてしまう
configMap:
name: configmap-sample-data
EOF
)
deployment.apps/nginx-deployment configured
$ minikube kubectl -- -ntest get deploy nginx-deployment -oyaml
apiVersion: apps/v1
kind: Deployment
metadata:
annotations:
deployment.kubernetes.io/revision: "1"
kubectl.kubernetes.io/last-applied-configuration: |
{"apiVersion":"apps/v1","kind":"Deployment","metadata":{"annotations":{},"labels":{"app":"nginx"},"name":"nginx-deployment","namespace":"test"},"spec":{"replicas":1,"selector":{"matchLabels":{"app":"nginx"}},"template":{"metadata":{"labels":{"app":"nginx"}},"spec":{"containers":[{"image":"nginx:latest","name":"nginx","ports":[{"containerPort":80}],"volumeMounts":[{"mountPath":"/var/tmp/configmap","name":"configmap-volume"}]}],"volumes":[{"configMap":{"name":"configmap-sample-data"},"name":"configmap-volume"},{"configMap":{"name":"configmap-sample-data"},"name":"configmap-volume"}]}}}}
creationTimestamp: "2021-11-20T14:27:43Z"
generation: 2
labels:
app: nginx
...
# last-applied-configuration を見てみる
{
"apiVersion": "apps/v1",
"kind": "Deployment",
"metadata": {
"annotations": {},
"labels": {
"app": "nginx"
},
"name": "nginx-deployment",
"namespace": "test"
},
"spec": {
"replicas": 1,
"selector": {
"matchLabels": {
"app": "nginx"
}
},
"template": {
"metadata": {
"labels": {
"app": "nginx"
}
},
"spec": {
"containers": [
{
"image": "nginx:latest",
"name": "nginx",
"ports": [
{
"containerPort": 80
}
],
"volumeMounts": [
{
"mountPath": "/var/tmp/configmap",
"name": "configmap-volume"
}
]
}
],
"volumes": [
{
"configMap": {
"name": "configmap-sample-data"
},
"name": "configmap-volume" # <-- 同名が２つ存在できる...
},
{
"configMap": {
"name": "configmap-sample-data"
},
"name": "configmap-volume" # <-- 同名が２つ存在できる..
}
]
}
}
}
}
# こちらが失敗する
$ minikube kubectl -- apply -f <(cat <<EOF
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-deployment
labels:
app: nginx
namespace: test
spec:
replicas: 1
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx:latest
ports:
- containerPort: 80
volumeMounts:
- name: configmap-volume
mountPath: /var/tmp/configmap
volumes:
- name: configmap-volume
configMap:
name: configmap-sample-data
EOF
)
The Deployment "nginx-deployment" is invalid: spec.template.spec.containers[0].volumeMounts[0].name: Not found: "configmap-volume"