kubernetes, archlinux
archLinux で minikube を試す
pacman で最新の minikube を試すことができると知って、さっそくインストールしてみた。
minikube は k8s を開発目的でシングルノードに立てられるツールだ。ドキュメントは充実しているため、公式ドキュメント を見ること。
結論
- pacman で minikube をインストールできる
- minikube の driver は none でよい
インストール
1# pacman の持っているパッケージ情報を更新
2archlinux% sudo pacman -Sy
3:: Synchronizing package databases...
4 core 136.0 KiB 223 KiB/s 00:01 [############################################] 100%
5 extra 1650.3 KiB 2.59 MiB/s 00:01 [############################################] 100%
6 community 4.9 MiB 11.7 MiB/s 00:00 [############################################] 100%
7
8archlinux% sudo pacman -Sy minikube
9:: Synchronizing package databases...
10 core is up to date
11 extra is up to date
12 community is up to date
13resolving dependencies...
14looking for conflicting packages...
15
16Packages (1) minikube-1.9.0-1
17
18Total Download Size: 10.80 MiB
19Total Installed Size: 47.67 MiB
20
21:: Proceed with installation? [Y/n] Y
22:: Retrieving packages...
23 minikube-1.9.0-1-x86_64 10.8 MiB 4.62 MiB/s 00:02 [############################################] 100%
24(1/1) checking keys in keyring [############################################] 100%
25(1/1) checking package integrity [############################################] 100%
26(1/1) loading package files [############################################] 100%
27(1/1) checking for file conflicts [############################################] 100%
28(1/1) checking available disk space [############################################] 100%
29:: Processing package changes...
30(1/1) installing minikube [############################################] 100%
31Optional dependencies for minikube
32 kubectl: to manage the cluster
33 virtualbox: to use --vm-driver=virtualbox
34:: Running post-transaction hooks...
35(1/1) Arming ConditionNeedsUpdate...
36archlinux% minikube start
37* minikube 1.9.1 is available! Download it: https://github.com/kubernetes/minikube/releases/tag/v1.9.1
38* To disable this notice, run: 'minikube config set WantUpdateNotification false'
39
40! minikube v1.9.0 on Arch
41X Unable to determine a default driver to use. Try specifying --driver, or see https://minikube.sigs.k8s.io/docs/start/
Driver Setup を見る必要がある。
docker driver を利用してみる (後述の理由で不要)
docker だと手軽に利用できるので、docker driver を選ぶ。
1archlinux% minikube start --driver=docker
2* minikube v1.9.0 on Arch
3* Using the docker driver based on user configuration
4
5! 'docker' driver reported an issue: exec: "docker": executable file not found in $PATH
6* Suggestion: Docker is required.
7* Documentation: https://minikube.sigs.k8s.io/docs/reference/drivers/docker/
8
9X docker does not appear to be installed
docker をインストールしていなかった。インストールする
1archlinux% sudo pacman -Sy docker --noconfirm
2
3:: Synchronizing package databases...
4 core is up to date
5 extra is up to date
6 community is up to date
7resolving dependencies...
8looking for conflicting packages...
9
10Packages (5) bridge-utils-1.6-4 containerd-1.3.3-1 libtool-2.4.6+42+gb88cebd5-11 runc-1.0.0rc10-2 docker-1:19.03.8-2
11
12Total Download Size: 92.43 MiB
13Total Installed Size: 399.09 MiB
14
15:: Proceed with installation? [Y/n]
16:: Retrieving packages...
17 bridge-utils-1.6-4-x86_64 15.5 KiB 75.0 KiB/s 00:00 [##############################################################] 100%
18 libtool-2.4.6+42+gb88cebd5-11-x86_64 407.4 KiB 483 KiB/s 00:01 [##############################################################] 100%
19 runc-1.0.0rc10-2-x86_64 2.5 MiB 5.81 MiB/s 00:00 [##############################################################] 100%
20 containerd-1.3.3-1-x86_64 22.4 MiB 9.44 MiB/s 00:02 [##############################################################] 100%
21 docker-1:19.03.8-2-x86_64 67.0 MiB 9.71 MiB/s 00:07 [##############################################################] 100%
22(5/5) checking keys in keyring [##############################################################] 100%
23(5/5) checking package integrity [##############################################################] 100%
24(5/5) loading package files [##############################################################] 100%
25(5/5) checking for file conflicts [##############################################################] 100%
26(5/5) checking available disk space [##############################################################] 100%
27:: Processing package changes...
28(1/5) installing bridge-utils [##############################################################] 100%
29(2/5) installing libtool [##############################################################] 100%
30(3/5) installing runc [##############################################################] 100%
31(4/5) installing containerd [##############################################################] 100%
32(5/5) installing docker [##############################################################] 100%
33Optional dependencies for docker
34 btrfs-progs: btrfs backend support
35 pigz: parallel gzip compressor support
36:: Running post-transaction hooks...
37(1/4) Creating system user accounts...
38(2/4) Reloading system manager configuration...
39(3/4) Reloading device manager configuration...
40(4/4) Arming ConditionNeedsUpdate...
これで minikube 起動できるか確認。
1archlinux% minikube start --driver=docker
2
3* minikube v1.9.0 on Arch
4* Using the docker driver based on user configuration
5
6! 'docker' driver reported an issue: exit status 1
7* Suggestion: Docker is not running or is responding too slow. Try: restarting docker desktop.
8
9* Pulling base image ...
10* Downloading Kubernetes v1.18.0 preload ...
11 > preloaded-images-k8s-v2-v1.18.0-docker-overlay2-amd64.tar.lz4: 542.91 MiB
12E0405 05:20:09.825972 1511 cache.go:114] Error downloading kic artifacts: error loading image: Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
13* Creating Kubernetes in docker container with (CPUs=2) (0 available), Memory=3900MB (0MB available) ...
14! StartHost failed, but will try again: creating host: create: creating: create kic node: creating volume for minikube container: output Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
15: exit status 1
16* docker "minikube" container is missing, will recreate.
17* Creating Kubernetes in docker container with (CPUs=2) (0 available), Memory=3900MB (0MB available) ...
18* StartHost failed again: recreate: creating host: create: creating: create kic node: creating volume for minikube container: output Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
19: exit status 1
20 - Run: "minikube delete", then "minikube start --alsologtostderr -v=1" to try again with more logging
21*
22X Unable to start VM after repeated tries. Please try {{'minikube delete' if possible: recreate: creating host: create: creating: create kic node: creating volume for minikube container: output Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
23: exit status 1
24*
25* minikube is exiting due to an error. If the above message is not useful, open an issue:
26 - https://github.com/kubernetes/minikube/issues/new/choose
Is the docker daemon running?
との通り、docker を enable して起動しておく必要がある。systemd の状態を確認する。
1archlinux% sudo systemctl status docker
2
3* docker.service - Docker Application Container Engine
4 Loaded: loaded (/usr/lib/systemd/system/docker.service; disabled; vendor preset: disabled)
5 Active: inactive (dead)
6TriggeredBy: * docker.socket
7 Docs: https://docs.docker.com
dead (´・ω・｀)
1archlinux% sudo systemctl start docker
2archlinux% sudo systemctl enable docker
3Created symlink /etc/systemd/system/multi-user.target.wants/docker.service -> /usr/lib/systemd/system/docker.service.
docker を起動して、サーバー起動時に自動で立ち上がるようにする。さて、ようやく minikube が起動できるか…
1archlinux% minikube start --driver=docker
2
3* minikube v1.9.0 on Arch
4* Using the docker driver based on existing profile
5
6! 'docker' driver reported an issue: exit status 1
7* Suggestion: Docker is not running or is responding too slow. Try: restarting docker desktop.
8
9* Pulling base image ...
10E0405 05:35:22.655536 2820 cache.go:114] Error downloading kic artifacts: error loading image: Got permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock: Post "http://%2Fvar%2Frun%2Fdocker.sock/v1.24/images/load?quiet=0": dial unix /var/run/docker.sock: connect: permission denied
11* docker "minikube" container is missing, will recreate.
12* Creating Kubernetes in docker container with (CPUs=2) (0 available), Memory=3900MB (0MB available) ...
13! StartHost failed, but will try again: recreate: creating host: create: creating: create kic node: creating volume for minikube container: output Got permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock: Post "http://%2Fvar%2Frun%2Fdocker.sock/v1.40/volumes/create": dial unix /var/run/docker.sock: connect: permission denied
permission denied だと… (´・ω・｀)
考えてみたら、driver は KVM, VirtualBox, Docker, None というように選択できる。minikube 自体を起動するホストをどうするかということだろう。
ということは、docker driver を利用する場合は docker で Linux & minikube を起動する必要があるのではないか？
それであれば、driver None を使って archLinux から直接起動したほうが、都合は良い。
driver none で試してみる
docker は別用途で使うので置いておくとして、minikube を driver none で起動する。
1archlinux% sudo minikube start --driver=none
2* minikube v1.9.0 on Arch
3* Using the none driver based on user configuration
4X The none driver requires conntrack to be installed for kubernetes version 1.18.0
conntrack を求められている。現在だと、Arch Linux - conntrack-tools 1.4.6-2 (x86_64) をインストールすれば良さそうだ。
conntrack とは
connection tracking の略称。Linux kernel が論理的なネットワーク接続や通信をトラッキングする仕組みとして、Netfilter - Wikipedia がある。 Netfilter の情報にアクセスして現在のネットワーク接続の情報を取得できるのが、conntrack。
コネクションが大量に発生するような本番環境では、conntrack の実行がパフォーマンスの劣化を起こすことがあるみたいだ。これは面白い。
When Linux conntrack is no longer your friend | Project Calico
Calico のような静的ファイアウォールは conntrack の情報を利用しているようだ。
conntrack-tools をインストール
1archlinux% sudo pacman -S --noconfirm conntrack-tools
2resolving dependencies...
3looking for conflicting packages...
4
5Packages (4) libnetfilter_cthelper-1.0.0-3 libnetfilter_cttimeout-1.0.0-5 libnetfilter_queue-1.0.3-1 conntrack-tools-1.4.6-2
6
7Total Download Size: 0.17 MiB
8Total Installed Size: 0.57 MiB
9
10:: Proceed with installation? [Y/n]
11:: Retrieving packages...
12 libnetfilter_cttimeout-1.0.0-5-x86_64 7.3 KiB 0.00 B/s 00:00 [##############################################################] 100%
13 libnetfilter_cthelper-1.0.0-3-x86_64 6.8 KiB 0.00 B/s 00:00 [##############################################################] 100%
14 libnetfilter_queue-1.0.3-1-x86_64 14.3 KiB 0.00 B/s 00:00 [##############################################################] 100%
15 conntrack-tools-1.4.6-2-x86_64 144.8 KiB 353 KiB/s 00:00 [##############################################################] 100%
16(4/4) checking keys in keyring [##############################################################] 100%
17(4/4) checking package integrity [##############################################################] 100%
18(4/4) loading package files [##############################################################] 100%
19(4/4) checking for file conflicts [##############################################################] 100%
20(4/4) checking available disk space [##############################################################] 100%
21:: Processing package changes...
22(1/4) installing libnetfilter_cttimeout [##############################################################] 100%
23(2/4) installing libnetfilter_cthelper [##############################################################] 100%
24(3/4) installing libnetfilter_queue [##############################################################] 100%
25(4/4) installing conntrack-tools [##############################################################] 100%
26:: Running post-transaction hooks...
27(1/2) Reloading system manager configuration...
28(2/2) Arming ConditionNeedsUpdate...
conntrack が依存する形で、netfilter が入っていることがわかる。
ようやく minikube を立ち上げる
1archlinux% sudo minikube start --driver=none
2* minikube v1.9.0 on Arch
3* Using the none driver based on user configuration
4* Running on localhost (CPUs=8, Memory=15879MB, Disk=464105MB) ...
5* OS release is Arch Linux
6! Node may be unable to resolve external DNS records
7* Preparing Kubernetes v1.18.0 on Docker 19.03.8-ce ...
8 > kubectl.sha256: 65 B / 65 B [--------------------------] 100.00% ? p/s 0s
9 > kubeadm.sha256: 65 B / 65 B [--------------------------] 100.00% ? p/s 0s
10 > kubelet.sha256: 65 B / 65 B [--------------------------] 100.00% ? p/s 0s
11 > kubectl: 41.98 MiB / 41.98 MiB [---------------] 100.00% 42.98 MiB p/s 1s
12 > kubeadm: 37.96 MiB / 37.96 MiB [---------------] 100.00% 19.48 MiB p/s 2s
13 > kubelet: 108.01 MiB / 108.01 MiB [-------------] 100.00% 45.30 MiB p/s 3s
14* Enabling addons: default-storageclass, storage-provisioner
15! Enabling 'default-storageclass' returned an error: running callbacks: [sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.18.0/kubectl apply -f /etc/kubernetes/addons/storageclass.yaml: exec: already started
16stdout:
17
18stderr:
19]
20! Enabling 'storage-provisioner' returned an error: running callbacks: [sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.18.0/kubectl apply -f /etc/kubernetes/addons/storage-provisioner.yaml: exec: already started
21stdout:
22
23stderr:
24]
25* Configuring local host environment ...
26*
27! The 'none' driver provides limited isolation and may reduce system security and reliability.
28! For more information, see:
29 - https://minikube.sigs.k8s.io/docs/reference/drivers/none/
30*
31! kubectl and minikube configuration will be stored in /root
32! To use kubectl or minikube commands as your own user, you may need to relocate them. For example, to overwrite your own settings, run:
33*
34 - sudo mv /root/.kube /root/.minikube $HOME
35 - sudo chown -R $USER $HOME/.kube $HOME/.minikube
36*
37* This can also be done automatically by setting the env var CHANGE_MINIKUBE_NONE_USER=true
38* Done! kubectl is now configured to use "minikube"
39* For best results, install kubectl: https://kubernetes.io/docs/tasks/tools/install-kubectl/
最後に Done! と出ていれば起動している。
1archlinux% sudo minikube status
2
3host: Running
4kubelet: Running
5apiserver: Running
6kubeconfig: Configured
ローカルのポートの使用状況を確認する。
1archlinux% sudo ss -ntpl
2State Recv-Q Send-Q Local Address:Port Peer Address:Port Process
3LISTEN 0 4096 127.0.0.1:10248 0.0.0.0:* users:(("kubelet",pid=31016,fd=32))
4LISTEN 0 4096 127.0.0.1:10249 0.0.0.0:* users:(("kube-proxy",pid=31655,fd=11))
5LISTEN 0 128 0.0.0.0:26313 0.0.0.0:* users:(("sshd",pid=374,fd=3))
6LISTEN 0 4096 192.168.XX.XX:2379 0.0.0.0:* users:(("etcd",pid=30615,fd=6))
7LISTEN 0 4096 127.0.0.1:2379 0.0.0.0:* users:(("etcd",pid=30615,fd=5))
8LISTEN 0 4096 192.168.XX.XX:2380 0.0.0.0:* users:(("etcd",pid=30615,fd=3))
9LISTEN 0 4096 127.0.0.1:2381 0.0.0.0:* users:(("etcd",pid=30615,fd=10))
10LISTEN 0 4096 127.0.0.1:10257 0.0.0.0:* users:(("kube-controller",pid=30609,fd=6))
11LISTEN 0 4096 127.0.0.1:10259 0.0.0.0:* users:(("kube-scheduler",pid=30611,fd=6))
12LISTEN 0 4096 127.0.0.1:40855 0.0.0.0:* users:(("kubelet",pid=31016,fd=8))
13LISTEN 0 128 [::]:26313 [::]:* users:(("sshd",pid=374,fd=4))
14LISTEN 0 4096 *:10250 *:* users:(("kubelet",pid=31016,fd=39))
15LISTEN 0 4096 *:10251 *:* users:(("kube-scheduler",pid=30611,fd=5))
16LISTEN 0 4096 *:10252 *:* users:(("kube-controller",pid=30609,fd=5))
17LISTEN 0 4096 *:10256 *:* users:(("kube-proxy",pid=31655,fd=10))
18LISTEN 0 4096 *:8443 *:* users:(("kube-apiserver",pid=30629,fd=5))
kubectl は /var/lib/minikube/binaries/v1.18.0/kubectl のようなパスにあるため、必要に応じて PATH に追加する。
NG集
1archlinux% sudo minikube start --driver=none
2* minikube v1.9.0 on Arch
3* Using the none driver based on user configuration
4*
5X Failed to save config: failed to acquire lock for /root/.minikube/profiles/minikube/config.json: {Name:mk270d1b5db5965f2dc9e9e25770a63417031943 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}: unable to open /tmp/juju-mk270d1b5db5965f2dc9e9e25770a63417031943: permission denied
6*
7* minikube is exiting due to an error. If the above message is not useful, open an issue:
8 - https://github.com/kubernetes/minikube/issues/new/choose
sudo 抜きで起動したり、driver docker で起動したりをしていた影響で、/tmp 配下にゴミファイルが残ってしまって、それが原因で minikube が起動できないということがあった。
1archlinux% sudo rm /tmp/minikube.*
2archlinux% sudo rm /tmp/juju-mk*
初回構築であれば上記のファイルは /tmp で削除してしまって問題ない。また、 minikube delete を使って driver docker で起動した minikube を一度クリーンにする必要があった。