archlinux で minikube を試す
pacman で最新の minikube を試すことができると知って、さっそくインストールしてみた。
minikube は k8s を開発目的でシングルノードに立てられるツールだ。ドキュメントは充実しているため、公式ドキュメント を見ること。
結論
- pacman で minikube をインストールできる
- minikube の driver は none でよい
インストール
# pacman の持っているパッケージ情報を更新
archlinux% sudo pacman -Sy
:: Synchronizing package databases...
core 136.0 KiB 223 KiB/s 00:01 [############################################] 100%
extra 1650.3 KiB 2.59 MiB/s 00:01 [############################################] 100%
community 4.9 MiB 11.7 MiB/s 00:00 [############################################] 100%
archlinux% sudo pacman -Sy minikube
:: Synchronizing package databases...
core is up to date
extra is up to date
community is up to date
resolving dependencies...
looking for conflicting packages...
Packages (1) minikube-1.9.0-1
Total Download Size: 10.80 MiB
Total Installed Size: 47.67 MiB
:: Proceed with installation? [Y/n] Y
:: Retrieving packages...
minikube-1.9.0-1-x86_64 10.8 MiB 4.62 MiB/s 00:02 [############################################] 100%
(1/1) checking keys in keyring [############################################] 100%
(1/1) checking package integrity [############################################] 100%
(1/1) loading package files [############################################] 100%
(1/1) checking for file conflicts [############################################] 100%
(1/1) checking available disk space [############################################] 100%
:: Processing package changes...
(1/1) installing minikube [############################################] 100%
Optional dependencies for minikube
kubectl: to manage the cluster
virtualbox: to use --vm-driver=virtualbox
:: Running post-transaction hooks...
(1/1) Arming ConditionNeedsUpdate...
archlinux% minikube start
* minikube 1.9.1 is available! Download it: https://github.com/kubernetes/minikube/releases/tag/v1.9.1
* To disable this notice, run: 'minikube config set WantUpdateNotification false'
! minikube v1.9.0 on Arch
X Unable to determine a default driver to use. Try specifying --driver, or see https://minikube.sigs.k8s.io/docs/start/
Driver Setup を見る必要がある。
docker driver を利用してみる (後述の理由で不要)
docker だと手軽に利用できるので、docker driver を選ぶ。
archlinux% minikube start --driver=docker
* minikube v1.9.0 on Arch
* Using the docker driver based on user configuration
! 'docker' driver reported an issue: exec: "docker": executable file not found in $PATH
* Suggestion: Docker is required.
* Documentation: https://minikube.sigs.k8s.io/docs/reference/drivers/docker/
X docker does not appear to be installed
docker をインストールしていなかった。インストールする
archlinux% sudo pacman -Sy docker --noconfirm
:: Synchronizing package databases...
core is up to date
extra is up to date
community is up to date
resolving dependencies...
looking for conflicting packages...
Packages (5) bridge-utils-1.6-4 containerd-1.3.3-1 libtool-2.4.6+42+gb88cebd5-11 runc-1.0.0rc10-2 docker-1:19.03.8-2
Total Download Size: 92.43 MiB
Total Installed Size: 399.09 MiB
:: Proceed with installation? [Y/n]
:: Retrieving packages...
bridge-utils-1.6-4-x86_64 15.5 KiB 75.0 KiB/s 00:00 [##############################################################] 100%
libtool-2.4.6+42+gb88cebd5-11-x86_64 407.4 KiB 483 KiB/s 00:01 [##############################################################] 100%
runc-1.0.0rc10-2-x86_64 2.5 MiB 5.81 MiB/s 00:00 [##############################################################] 100%
containerd-1.3.3-1-x86_64 22.4 MiB 9.44 MiB/s 00:02 [##############################################################] 100%
docker-1:19.03.8-2-x86_64 67.0 MiB 9.71 MiB/s 00:07 [##############################################################] 100%
(5/5) checking keys in keyring [##############################################################] 100%
(5/5) checking package integrity [##############################################################] 100%
(5/5) loading package files [##############################################################] 100%
(5/5) checking for file conflicts [##############################################################] 100%
(5/5) checking available disk space [##############################################################] 100%
:: Processing package changes...
(1/5) installing bridge-utils [##############################################################] 100%
(2/5) installing libtool [##############################################################] 100%
(3/5) installing runc [##############################################################] 100%
(4/5) installing containerd [##############################################################] 100%
(5/5) installing docker [##############################################################] 100%
Optional dependencies for docker
btrfs-progs: btrfs backend support
pigz: parallel gzip compressor support
:: Running post-transaction hooks...
(1/4) Creating system user accounts...
(2/4) Reloading system manager configuration...
(3/4) Reloading device manager configuration...
(4/4) Arming ConditionNeedsUpdate...
これで minikube 起動できるか確認。
archlinux% minikube start --driver=docker
* minikube v1.9.0 on Arch
* Using the docker driver based on user configuration
! 'docker' driver reported an issue: exit status 1
* Suggestion: Docker is not running or is responding too slow. Try: restarting docker desktop.
* Pulling base image ...
* Downloading Kubernetes v1.18.0 preload ...
> preloaded-images-k8s-v2-v1.18.0-docker-overlay2-amd64.tar.lz4: 542.91 MiB
E0405 05:20:09.825972 1511 cache.go:114] Error downloading kic artifacts: error loading image: Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
* Creating Kubernetes in docker container with (CPUs=2) (0 available), Memory=3900MB (0MB available) ...
! StartHost failed, but will try again: creating host: create: creating: create kic node: creating volume for minikube container: output Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
: exit status 1
* docker "minikube" container is missing, will recreate.
* Creating Kubernetes in docker container with (CPUs=2) (0 available), Memory=3900MB (0MB available) ...
* StartHost failed again: recreate: creating host: create: creating: create kic node: creating volume for minikube container: output Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
: exit status 1
- Run: "minikube delete", then "minikube start --alsologtostderr -v=1" to try again with more logging
*
X Unable to start VM after repeated tries. Please try {{'minikube delete' if possible: recreate: creating host: create: creating: create kic node: creating volume for minikube container: output Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
: exit status 1
*
* minikube is exiting due to an error. If the above message is not useful, open an issue:
- https://github.com/kubernetes/minikube/issues/new/choose
Is the docker daemon running?
との通り、docker を enable して起動しておく必要がある。systemd の状態を確認する。
archlinux% sudo systemctl status docker
* docker.service - Docker Application Container Engine
Loaded: loaded (/usr/lib/systemd/system/docker.service; disabled; vendor preset: disabled)
Active: inactive (dead)
TriggeredBy: * docker.socket
Docs: https://docs.docker.com
dead (´・ω・｀)
archlinux% sudo systemctl start docker
archlinux% sudo systemctl enable docker
Created symlink /etc/systemd/system/multi-user.target.wants/docker.service -> /usr/lib/systemd/system/docker.service.
docker を起動して、サーバー起動時に自動で立ち上がるようにする。さて、ようやく minikube が起動できるか…
archlinux% minikube start --driver=docker
* minikube v1.9.0 on Arch
* Using the docker driver based on existing profile
! 'docker' driver reported an issue: exit status 1
* Suggestion: Docker is not running or is responding too slow. Try: restarting docker desktop.
* Pulling base image ...
E0405 05:35:22.655536 2820 cache.go:114] Error downloading kic artifacts: error loading image: Got permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock: Post "http://%2Fvar%2Frun%2Fdocker.sock/v1.24/images/load?quiet=0": dial unix /var/run/docker.sock: connect: permission denied
* docker "minikube" container is missing, will recreate.
* Creating Kubernetes in docker container with (CPUs=2) (0 available), Memory=3900MB (0MB available) ...
! StartHost failed, but will try again: recreate: creating host: create: creating: create kic node: creating volume for minikube container: output Got permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock: Post "http://%2Fvar%2Frun%2Fdocker.sock/v1.40/volumes/create": dial unix /var/run/docker.sock: connect: permission denied
permission denied だと… (´・ω・｀)
考えてみたら、driver は KVM, VirtualBox, Docker, None というように選択できる。minikube 自体を起動するホストをどうするかということだろう。
ということは、docker driver を利用する場合は docker で linux & minikube を起動する必要があるのではないか？
それであれば、driver None で直接 archlinux から起動してしまったほうが都合が良い。
driver none で試してみる
docker はそれはそれで使うので置いておくとして、minikube を driver none で起動する。
archlinux% sudo minikube start --driver=none
* minikube v1.9.0 on Arch
* Using the none driver based on user configuration
X The none driver requires conntrack to be installed for kubernetes version 1.18.0
conntrack を求められている。現在だと、Arch Linux - conntrack-tools 1.4.6-2 (x86_64) をインストールすれば良さそうだ。
conntrack とは
connection tracking の略称。Linux kernel が論理的なネットワーク接続や通信をトラッキングする仕組みとして、Netfilter - Wikipedia がある。 Netfilter の情報にアクセスして現在のネットワーク接続の情報を取得できるのが、conntrack。
コネクションが大量に発生するような本番環境では、conntrack の実行がパフォーマンスの劣化を起こすことがあるみたいだ。これは面白い。
When Linux conntrack is no longer your friend | Project Calico
Calico のような静的ファイアウォールは conntrack の情報を利用しているようだ。
conntrack-tools をインストール
archlinux% sudo pacman -S --noconfirm conntrack-tools
resolving dependencies...
looking for conflicting packages...
Packages (4) libnetfilter_cthelper-1.0.0-3 libnetfilter_cttimeout-1.0.0-5 libnetfilter_queue-1.0.3-1 conntrack-tools-1.4.6-2
Total Download Size: 0.17 MiB
Total Installed Size: 0.57 MiB
:: Proceed with installation? [Y/n]
:: Retrieving packages...
libnetfilter_cttimeout-1.0.0-5-x86_64 7.3 KiB 0.00 B/s 00:00 [##############################################################] 100%
libnetfilter_cthelper-1.0.0-3-x86_64 6.8 KiB 0.00 B/s 00:00 [##############################################################] 100%
libnetfilter_queue-1.0.3-1-x86_64 14.3 KiB 0.00 B/s 00:00 [##############################################################] 100%
conntrack-tools-1.4.6-2-x86_64 144.8 KiB 353 KiB/s 00:00 [##############################################################] 100%
(4/4) checking keys in keyring [##############################################################] 100%
(4/4) checking package integrity [##############################################################] 100%
(4/4) loading package files [##############################################################] 100%
(4/4) checking for file conflicts [##############################################################] 100%
(4/4) checking available disk space [##############################################################] 100%
:: Processing package changes...
(1/4) installing libnetfilter_cttimeout [##############################################################] 100%
(2/4) installing libnetfilter_cthelper [##############################################################] 100%
(3/4) installing libnetfilter_queue [##############################################################] 100%
(4/4) installing conntrack-tools [##############################################################] 100%
:: Running post-transaction hooks...
(1/2) Reloading system manager configuration...
(2/2) Arming ConditionNeedsUpdate...
conntrack が依存する形で、netfilter が入っていることがわかる。
ようやく minikube を立ち上げる
archlinux% sudo minikube start --driver=none
* minikube v1.9.0 on Arch
* Using the none driver based on user configuration
* Running on localhost (CPUs=8, Memory=15879MB, Disk=464105MB) ...
* OS release is Arch Linux
! Node may be unable to resolve external DNS records
* Preparing Kubernetes v1.18.0 on Docker 19.03.8-ce ...
> kubectl.sha256: 65 B / 65 B [--------------------------] 100.00% ? p/s 0s
> kubeadm.sha256: 65 B / 65 B [--------------------------] 100.00% ? p/s 0s
> kubelet.sha256: 65 B / 65 B [--------------------------] 100.00% ? p/s 0s
> kubectl: 41.98 MiB / 41.98 MiB [---------------] 100.00% 42.98 MiB p/s 1s
> kubeadm: 37.96 MiB / 37.96 MiB [---------------] 100.00% 19.48 MiB p/s 2s
> kubelet: 108.01 MiB / 108.01 MiB [-------------] 100.00% 45.30 MiB p/s 3s
* Enabling addons: default-storageclass, storage-provisioner
! Enabling 'default-storageclass' returned an error: running callbacks: [sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.18.0/kubectl apply -f /etc/kubernetes/addons/storageclass.yaml: exec: already started
stdout:
stderr:
]
! Enabling 'storage-provisioner' returned an error: running callbacks: [sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.18.0/kubectl apply -f /etc/kubernetes/addons/storage-provisioner.yaml: exec: already started
stdout:
stderr:
]
* Configuring local host environment ...
*
! The 'none' driver provides limited isolation and may reduce system security and reliability.
! For more information, see:
- https://minikube.sigs.k8s.io/docs/reference/drivers/none/
*
! kubectl and minikube configuration will be stored in /root
! To use kubectl or minikube commands as your own user, you may need to relocate them. For example, to overwrite your own settings, run:
*
- sudo mv /root/.kube /root/.minikube $HOME
- sudo chown -R $USER $HOME/.kube $HOME/.minikube
*
* This can also be done automatically by setting the env var CHANGE_MINIKUBE_NONE_USER=true
* Done! kubectl is now configured to use "minikube"
* For best results, install kubectl: https://kubernetes.io/docs/tasks/tools/install-kubectl/
最後に Done! と出ていれば起動している。
archlinux% sudo minikube status
host: Running
kubelet: Running
apiserver: Running
kubeconfig: Configured
ローカルのポートの使用状況を確認する。
archlinux% sudo ss -ntpl
State Recv-Q Send-Q Local Address:Port Peer Address:Port Process
LISTEN 0 4096 127.0.0.1:10248 0.0.0.0:* users:(("kubelet",pid=31016,fd=32))
LISTEN 0 4096 127.0.0.1:10249 0.0.0.0:* users:(("kube-proxy",pid=31655,fd=11))
LISTEN 0 128 0.0.0.0:26313 0.0.0.0:* users:(("sshd",pid=374,fd=3))
LISTEN 0 4096 192.168.XX.XX:2379 0.0.0.0:* users:(("etcd",pid=30615,fd=6))
LISTEN 0 4096 127.0.0.1:2379 0.0.0.0:* users:(("etcd",pid=30615,fd=5))
LISTEN 0 4096 192.168.XX.XX:2380 0.0.0.0:* users:(("etcd",pid=30615,fd=3))
LISTEN 0 4096 127.0.0.1:2381 0.0.0.0:* users:(("etcd",pid=30615,fd=10))
LISTEN 0 4096 127.0.0.1:10257 0.0.0.0:* users:(("kube-controller",pid=30609,fd=6))
LISTEN 0 4096 127.0.0.1:10259 0.0.0.0:* users:(("kube-scheduler",pid=30611,fd=6))
LISTEN 0 4096 127.0.0.1:40855 0.0.0.0:* users:(("kubelet",pid=31016,fd=8))
LISTEN 0 128 [::]:26313 [::]:* users:(("sshd",pid=374,fd=4))
LISTEN 0 4096 *:10250 *:* users:(("kubelet",pid=31016,fd=39))
LISTEN 0 4096 *:10251 *:* users:(("kube-scheduler",pid=30611,fd=5))
LISTEN 0 4096 *:10252 *:* users:(("kube-controller",pid=30609,fd=5))
LISTEN 0 4096 *:10256 *:* users:(("kube-proxy",pid=31655,fd=10))
LISTEN 0 4096 *:8443 *:* users:(("kube-apiserver",pid=30629,fd=5))
kubectl は /var/lib/minikube/binaries/v1.18.0/kubectl のようなパスにあるため、必要に応じて PATH に追加する。
NG集
archlinux% sudo minikube start --driver=none
* minikube v1.9.0 on Arch
* Using the none driver based on user configuration
*
X Failed to save config: failed to acquire lock for /root/.minikube/profiles/minikube/config.json: {Name:mk270d1b5db5965f2dc9e9e25770a63417031943 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}: unable to open /tmp/juju-mk270d1b5db5965f2dc9e9e25770a63417031943: permission denied
*
* minikube is exiting due to an error. If the above message is not useful, open an issue:
- https://github.com/kubernetes/minikube/issues/new/choose
sudo 抜きで起動したり、driver docker で起動したりをしていた影響で、/tmp 配下にゴミファイルが残ってしまって、それが原因で minikube が起動できないということがあった。
archlinux% sudo rm /tmp/minikube.*
archlinux% sudo rm /tmp/juju-mk*
初回構築であれば上記のファイルは /tmp で削除してしまって問題ない。また、 minikube delete を使って driver docker で起動した minikube を一度クリーンにする必要があった。