archlinux で minikube を試す

自宅サーバーでお手軽 k8s


Posted by Akira Masuda on Sat, Apr 4, 2020
Tags k8s, archlinux

archlinux で minikube を試す

pacman で最新の minikube を試すことができると知って、さっそくインストールしてみた。

minikube は k8s を開発目的でシングルノードに立てられるツールである。ドキュメントは充実しているため、公式ドキュメント を見ること。

結論

  • pacman で minikube をインストールできる
  • minikube の driver は none でよい

インストール

# pacman の持っているパッケージ情報を更新
archlinux% sudo pacman -Sy
:: Synchronizing package databases...
 core                                              136.0 KiB   223 KiB/s 00:01 [############################################] 100%
 extra                                            1650.3 KiB  2.59 MiB/s 00:01 [############################################] 100%
 community                                           4.9 MiB  11.7 MiB/s 00:00 [############################################] 100%

archlinux%  sudo pacman -Sy minikube
:: Synchronizing package databases...
 core is up to date
 extra is up to date
 community is up to date
resolving dependencies...
looking for conflicting packages...

Packages (1) minikube-1.9.0-1

Total Download Size:   10.80 MiB
Total Installed Size:  47.67 MiB

:: Proceed with installation? [Y/n] Y
:: Retrieving packages...
 minikube-1.9.0-1-x86_64                            10.8 MiB  4.62 MiB/s 00:02 [############################################] 100%
(1/1) checking keys in keyring                                                 [############################################] 100%
(1/1) checking package integrity                                               [############################################] 100%
(1/1) loading package files                                                    [############################################] 100%
(1/1) checking for file conflicts                                              [############################################] 100%
(1/1) checking available disk space                                            [############################################] 100%
:: Processing package changes...
(1/1) installing minikube                                                      [############################################] 100%
Optional dependencies for minikube
    kubectl: to manage the cluster
    virtualbox: to use --vm-driver=virtualbox
:: Running post-transaction hooks...
(1/1) Arming ConditionNeedsUpdate...
archlinux% minikube start
* minikube 1.9.1 is available! Download it: https://github.com/kubernetes/minikube/releases/tag/v1.9.1
* To disable this notice, run: 'minikube config set WantUpdateNotification false'

! minikube v1.9.0 on Arch
X Unable to determine a default driver to use. Try specifying --driver, or see https://minikube.sigs.k8s.io/docs/start/

Driver Setup を見る必要がある。

docker driver を利用してみる (後述の理由で不要)

docker だと手軽に利用できるので、docker driver を選ぶ。

archlinux% minikube start --driver=docker
* minikube v1.9.0 on Arch
* Using the docker driver based on user configuration

! 'docker' driver reported an issue: exec: "docker": executable file not found in $PATH
* Suggestion: Docker is required.
* Documentation: https://minikube.sigs.k8s.io/docs/reference/drivers/docker/

X docker does not appear to be installed

docker をインストールしていなかった。インストールする

archlinux% sudo pacman -Sy docker --noconfirm

:: Synchronizing package databases...
 core is up to date
 extra is up to date
 community is up to date
resolving dependencies...
looking for conflicting packages...

Packages (5) bridge-utils-1.6-4  containerd-1.3.3-1  libtool-2.4.6+42+gb88cebd5-11  runc-1.0.0rc10-2  docker-1:19.03.8-2

Total Download Size:    92.43 MiB
Total Installed Size:  399.09 MiB

:: Proceed with installation? [Y/n]
:: Retrieving packages...
 bridge-utils-1.6-4-x86_64                                                   15.5 KiB  75.0 KiB/s 00:00 [##############################################################] 100%
 libtool-2.4.6+42+gb88cebd5-11-x86_64                                       407.4 KiB   483 KiB/s 00:01 [##############################################################] 100%
 runc-1.0.0rc10-2-x86_64                                                      2.5 MiB  5.81 MiB/s 00:00 [##############################################################] 100%
 containerd-1.3.3-1-x86_64                                                   22.4 MiB  9.44 MiB/s 00:02 [##############################################################] 100%
 docker-1:19.03.8-2-x86_64                                                   67.0 MiB  9.71 MiB/s 00:07 [##############################################################] 100%
(5/5) checking keys in keyring                                                                          [##############################################################] 100%
(5/5) checking package integrity                                                                        [##############################################################] 100%
(5/5) loading package files                                                                             [##############################################################] 100%
(5/5) checking for file conflicts                                                                       [##############################################################] 100%
(5/5) checking available disk space                                                                     [##############################################################] 100%
:: Processing package changes...
(1/5) installing bridge-utils                                                                           [##############################################################] 100%
(2/5) installing libtool                                                                                [##############################################################] 100%
(3/5) installing runc                                                                                   [##############################################################] 100%
(4/5) installing containerd                                                                             [##############################################################] 100%
(5/5) installing docker                                                                                 [##############################################################] 100%
Optional dependencies for docker
    btrfs-progs: btrfs backend support
    pigz: parallel gzip compressor support
:: Running post-transaction hooks...
(1/4) Creating system user accounts...
(2/4) Reloading system manager configuration...
(3/4) Reloading device manager configuration...
(4/4) Arming ConditionNeedsUpdate...

これで minikube 起動できるか確認。

archlinux% minikube start --driver=docker

* minikube v1.9.0 on Arch
* Using the docker driver based on user configuration

! 'docker' driver reported an issue: exit status 1
* Suggestion: Docker is not running or is responding too slow. Try: restarting docker desktop.

* Pulling base image ...
* Downloading Kubernetes v1.18.0 preload ...
    > preloaded-images-k8s-v2-v1.18.0-docker-overlay2-amd64.tar.lz4: 542.91 MiB
E0405 05:20:09.825972    1511 cache.go:114] Error downloading kic artifacts:  error loading image: Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
* Creating Kubernetes in docker container with (CPUs=2) (0 available), Memory=3900MB (0MB available) ...
! StartHost failed, but will try again: creating host: create: creating: create kic node: creating volume for minikube container: output Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
: exit status 1
* docker "minikube" container is missing, will recreate.
* Creating Kubernetes in docker container with (CPUs=2) (0 available), Memory=3900MB (0MB available) ...
* StartHost failed again: recreate: creating host: create: creating: create kic node: creating volume for minikube container: output Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
: exit status 1
  - Run: "minikube delete", then "minikube start --alsologtostderr -v=1" to try again with more logging
*
X Unable to start VM after repeated tries. Please try {{'minikube delete' if possible: recreate: creating host: create: creating: create kic node: creating volume for minikube container: output Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
: exit status 1
*
* minikube is exiting due to an error. If the above message is not useful, open an issue:
  - https://github.com/kubernetes/minikube/issues/new/choose

Is the docker daemon running?

との通り、docker を enable して起動しておく必要がある。systemd の状態を確認する。

archlinux% sudo systemctl status docker

* docker.service - Docker Application Container Engine
     Loaded: loaded (/usr/lib/systemd/system/docker.service; disabled; vendor preset: disabled)
     Active: inactive (dead)
TriggeredBy: * docker.socket
       Docs: https://docs.docker.com

dead (´・ω・`)

archlinux% sudo systemctl start docker
archlinux% sudo systemctl enable docker
Created symlink /etc/systemd/system/multi-user.target.wants/docker.service -> /usr/lib/systemd/system/docker.service.

docker を起動して、サーバー起動時に自動で立ち上がるようにする。さて、ようやく minikube が起動できるか…

archlinux% minikube start --driver=docker

* minikube v1.9.0 on Arch
* Using the docker driver based on existing profile

! 'docker' driver reported an issue: exit status 1
* Suggestion: Docker is not running or is responding too slow. Try: restarting docker desktop.

* Pulling base image ...
E0405 05:35:22.655536    2820 cache.go:114] Error downloading kic artifacts:  error loading image: Got permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock: Post "http://%2Fvar%2Frun%2Fdocker.sock/v1.24/images/load?quiet=0": dial unix /var/run/docker.sock: connect: permission denied
* docker "minikube" container is missing, will recreate.
* Creating Kubernetes in docker container with (CPUs=2) (0 available), Memory=3900MB (0MB available) ...
! StartHost failed, but will try again: recreate: creating host: create: creating: create kic node: creating volume for minikube container: output Got permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock: Post "http://%2Fvar%2Frun%2Fdocker.sock/v1.40/volumes/create": dial unix /var/run/docker.sock: connect: permission denied

permission denied だと… (´・ω・`)

考えてみたら、driver は KVM, VirtualBox, Docker, None というように選択できる。minikube 自体を起動するホストをどうするかということだろう。

ということは、docker driver を利用する場合は docker で linux & minikube を起動する必要があるのではないか?

それであれば、driver None で直接 archlinux から起動してしまったほうが都合が良い。

driver none で試してみる

docker はそれはそれで使うので置いておくとして、minikube を driver none で起動する。

archlinux% sudo minikube start --driver=none
* minikube v1.9.0 on Arch
* Using the none driver based on user configuration
X The none driver requires conntrack to be installed for kubernetes version 1.18.0

conntrack を求められている。現在だと、Arch Linux - conntrack-tools 1.4.6-2 (x86_64) をインストールすれば良さそうだ。

conntrack とは

connection tracking の略称。Linux kernel が論理的なネットワーク接続や通信をトラッキングする仕組みとして、Netfilter - Wikipedia がある。 Netfilter の情報にアクセスして現在のネットワーク接続の情報を取得できるのが、conntrack。

コネクションが大量に発生するような本番環境では、conntrack の実行がパフォーマンスの劣化を起こすことがあるみたいだ。これは面白い。

When Linux conntrack is no longer your friend | Project Calico

Calico のような静的ファイアウォールは conntrack の情報を利用しているようだ。

conntrack-tools をインストール

archlinux% sudo pacman -S --noconfirm conntrack-tools
resolving dependencies...
looking for conflicting packages...

Packages (4) libnetfilter_cthelper-1.0.0-3  libnetfilter_cttimeout-1.0.0-5  libnetfilter_queue-1.0.3-1  conntrack-tools-1.4.6-2

Total Download Size:   0.17 MiB
Total Installed Size:  0.57 MiB

:: Proceed with installation? [Y/n]
:: Retrieving packages...
 libnetfilter_cttimeout-1.0.0-5-x86_64                                        7.3 KiB  0.00   B/s 00:00 [##############################################################] 100%
 libnetfilter_cthelper-1.0.0-3-x86_64                                         6.8 KiB  0.00   B/s 00:00 [##############################################################] 100%
 libnetfilter_queue-1.0.3-1-x86_64                                           14.3 KiB  0.00   B/s 00:00 [##############################################################] 100%
 conntrack-tools-1.4.6-2-x86_64                                             144.8 KiB   353 KiB/s 00:00 [##############################################################] 100%
(4/4) checking keys in keyring                                                                          [##############################################################] 100%
(4/4) checking package integrity                                                                        [##############################################################] 100%
(4/4) loading package files                                                                             [##############################################################] 100%
(4/4) checking for file conflicts                                                                       [##############################################################] 100%
(4/4) checking available disk space                                                                     [##############################################################] 100%
:: Processing package changes...
(1/4) installing libnetfilter_cttimeout                                                                 [##############################################################] 100%
(2/4) installing libnetfilter_cthelper                                                                  [##############################################################] 100%
(3/4) installing libnetfilter_queue                                                                     [##############################################################] 100%
(4/4) installing conntrack-tools                                                                        [##############################################################] 100%
:: Running post-transaction hooks...
(1/2) Reloading system manager configuration...
(2/2) Arming ConditionNeedsUpdate...

conntrack が依存する形で、netfilter が入っていることがわかる。

ようやく minikube を立ち上げる

archlinux% sudo minikube start --driver=none
* minikube v1.9.0 on Arch
* Using the none driver based on user configuration
* Running on localhost (CPUs=8, Memory=15879MB, Disk=464105MB) ...
* OS release is Arch Linux
! Node may be unable to resolve external DNS records
* Preparing Kubernetes v1.18.0 on Docker 19.03.8-ce ...
    > kubectl.sha256: 65 B / 65 B [--------------------------] 100.00% ? p/s 0s
    > kubeadm.sha256: 65 B / 65 B [--------------------------] 100.00% ? p/s 0s
    > kubelet.sha256: 65 B / 65 B [--------------------------] 100.00% ? p/s 0s
    > kubectl: 41.98 MiB / 41.98 MiB [---------------] 100.00% 42.98 MiB p/s 1s
    > kubeadm: 37.96 MiB / 37.96 MiB [---------------] 100.00% 19.48 MiB p/s 2s
    > kubelet: 108.01 MiB / 108.01 MiB [-------------] 100.00% 45.30 MiB p/s 3s
* Enabling addons: default-storageclass, storage-provisioner
! Enabling 'default-storageclass' returned an error: running callbacks: [sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.18.0/kubectl apply -f /etc/kubernetes/addons/storageclass.yaml: exec: already started
stdout:

stderr:
]
! Enabling 'storage-provisioner' returned an error: running callbacks: [sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.18.0/kubectl apply -f /etc/kubernetes/addons/storage-provisioner.yaml: exec: already started
stdout:

stderr:
]
* Configuring local host environment ...
*
! The 'none' driver provides limited isolation and may reduce system security and reliability.
! For more information, see:
  - https://minikube.sigs.k8s.io/docs/reference/drivers/none/
*
! kubectl and minikube configuration will be stored in /root
! To use kubectl or minikube commands as your own user, you may need to relocate them. For example, to overwrite your own settings, run:
*
  - sudo mv /root/.kube /root/.minikube $HOME
  - sudo chown -R $USER $HOME/.kube $HOME/.minikube
*
* This can also be done automatically by setting the env var CHANGE_MINIKUBE_NONE_USER=true
* Done! kubectl is now configured to use "minikube"
* For best results, install kubectl: https://kubernetes.io/docs/tasks/tools/install-kubectl/

最後に Done! と出ていれば起動している。

archlinux% sudo minikube status

host: Running
kubelet: Running
apiserver: Running
kubeconfig: Configured

ローカルのポートの使用状況を確認する。

archlinux% sudo ss -ntpl
State           Recv-Q          Send-Q                   Local Address:Port                    Peer Address:Port         Process
LISTEN          0               4096                         127.0.0.1:10248                        0.0.0.0:*             users:(("kubelet",pid=31016,fd=32))
LISTEN          0               4096                         127.0.0.1:10249                        0.0.0.0:*             users:(("kube-proxy",pid=31655,fd=11))
LISTEN          0               128                            0.0.0.0:26313                        0.0.0.0:*             users:(("sshd",pid=374,fd=3))
LISTEN          0               4096                      192.168.XX.XX:2379                         0.0.0.0:*             users:(("etcd",pid=30615,fd=6))
LISTEN          0               4096                         127.0.0.1:2379                         0.0.0.0:*             users:(("etcd",pid=30615,fd=5))
LISTEN          0               4096                      192.168.XX.XX:2380                         0.0.0.0:*             users:(("etcd",pid=30615,fd=3))
LISTEN          0               4096                         127.0.0.1:2381                         0.0.0.0:*             users:(("etcd",pid=30615,fd=10))
LISTEN          0               4096                         127.0.0.1:10257                        0.0.0.0:*             users:(("kube-controller",pid=30609,fd=6))
LISTEN          0               4096                         127.0.0.1:10259                        0.0.0.0:*             users:(("kube-scheduler",pid=30611,fd=6))
LISTEN          0               4096                         127.0.0.1:40855                        0.0.0.0:*             users:(("kubelet",pid=31016,fd=8))
LISTEN          0               128                               [::]:26313                           [::]:*             users:(("sshd",pid=374,fd=4))
LISTEN          0               4096                                 *:10250                              *:*             users:(("kubelet",pid=31016,fd=39))
LISTEN          0               4096                                 *:10251                              *:*             users:(("kube-scheduler",pid=30611,fd=5))
LISTEN          0               4096                                 *:10252                              *:*             users:(("kube-controller",pid=30609,fd=5))
LISTEN          0               4096                                 *:10256                              *:*             users:(("kube-proxy",pid=31655,fd=10))
LISTEN          0               4096                                 *:8443                               *:*             users:(("kube-apiserver",pid=30629,fd=5))

kubectl は /var/lib/minikube/binaries/v1.18.0/kubectl のようなパスにあるため、必要に応じて PATH に追加する。

NG集

archlinux% sudo minikube start --driver=none
* minikube v1.9.0 on Arch
* Using the none driver based on user configuration
*
X Failed to save config: failed to acquire lock for /root/.minikube/profiles/minikube/config.json: {Name:mk270d1b5db5965f2dc9e9e25770a63417031943 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}: unable to open /tmp/juju-mk270d1b5db5965f2dc9e9e25770a63417031943: permission denied
*
* minikube is exiting due to an error. If the above message is not useful, open an issue:
  - https://github.com/kubernetes/minikube/issues/new/choose

sudo 抜きで起動したり、driver docker で起動したりをしていた影響で、/tmp 配下にゴミファイルが残ってしまって、それが原因で minikube が起動できないということがあった。

archlinux% sudo rm /tmp/minikube.*
archlinux% sudo rm /tmp/juju-mk*

初回構築であれば上記のファイルは /tmp で削除してしまって問題ない。また、 minikube delete を使って driver docker で起動した minikube を一度クリーンにする必要があった。

comments powered by Disqus