Docker and Docker Compose Notes

Posted on Wed, Mar 20, 2024
Tags docker, container, docker-compose
📝 This article is a translation of the original Japanese post. View original

Collected Notes on Docker

About the docker Command

Operations on Images and Containers

Renaming with docker tag

There are situations where an image ends up as a hash after a build and you want to give it a clearer name. Looks like tag is the only way?

How Docker images work - Qiita

1
2

docker build .
docker tag 09134185310 myapp:latest

Copying a file out of a docker image

1
2
3

docker create --name temp-image some-image
docker cp temp-image:/some/dir/file.tmp file.tmp
docker rm temp-image

Running Containers

docker run = docker create & docker start

run creates a container and starts it. At that point, using -d to detach leaves the process running in the background. -i brings the container down when you exit from it.

Docker Documentation

Adding privileged makes it a privileged container that can do anything

1

docker run --it alpine:latest --privileged

docker exec runs a command in a running docker process

docker exec -it <container name> lets you get into a process currently running under docker (provided a shell is on the PATH).

docker ps -a shows everything including stopped processes

1
2
3
4
5
6
7
8

sudo docker ps -a

CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS                     PORTS               NAMES
71e0cf6a50a6        101779243274        "/bin/bash"         5 weeks ago         Exited (137) 2 weeks ago                       angry_fermat
d5577c8aca3f        bbe77145eb18        "/bin/bash"         5 weeks ago         Exited (137) 2 weeks ago                       eager_carson
b8c0b1a734a6        101779243274        "bash"              6 weeks ago         Up 8 days                                      interesting_shtern
3ce1af5ce44c        bbe77145eb18        "/bin/bash"         8 weeks ago         Exited (137) 2 weeks ago                       hardcore_torvalds
0456af55e878        bbe77145eb18        "/bin/bash"         2 months ago        Exited (137) 2 weeks ago                       lucid_swanson

Cleaning Up Images and Containers

Deleting images whose TAG is none

1

docker image prune

This appears to be available from docker 1.25. Before that you had to use several commands, as below.

1

docker rmi $(docker images -f "dangling=true" -q)

Deleting docker images in a for loop

1

arr=("image_id1" "image_id2" "image_id3"); for i in "${arr[@]}" ; do sudo docker rmi -f $i ; done

Deleting containers that have exited and stopped

1

docker rm $(docker ps -aq)

deletes the stopped containers.

$(docker ps -aq) also targets running ones, but docker rm does not delete running containers.

Bringing the container down when you exit docker run

1

docker run --rm -it --name="default" alpine /bin/sh

Adding --rm makes the container come down when you exit the command. With this, you can experiment freely, and once you docker commit or save you have a Docker image, so all that’s left is to delete it.

Commands I Use Often from the Help Output

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58

# subcommands I personally rarely use are commented out
$ docker -h

Common Commands:
  run         Create and run a new container from an image
  exec        Execute a command in a running container
  ps          List containers
  build       Build an image from a Dockerfile
  pull        Download an image from a registry
# push        Upload an image to a registry
  images      List images
# login       Log in to a registry
# logout      Log out from a registry
# search      Search Docker Hub for images
  version     Show the Docker version information
  info        Display system-wide information

Management Commands:
# builder     Manage builds
  container   Manage containers
# context     Manage contexts
  image       Manage images
# manifest    Manage Docker image manifests and manifest lists
  network     Manage networks
# plugin      Manage plugins
  system      Manage Docker
# trust       Manage trust on Docker images
  volume      Manage volumes

Commands:
  attach      Attach local standard input, output, and error streams to a running container
# commit      Create a new image from a container's changes
  cp          Copy files/folders between a container and the local filesystem
# create      Create a new container
# diff        Inspect changes to files or directories on a container's filesystem
# events      Get real time events from the server
# export      Export a container's filesystem as a tar archive
# history     Show the history of an image
  import      Import the contents from a tarball to create a filesystem image
# inspect     Return low-level information on Docker objects
  kill        Kill one or more running containers
  load        Load an image from a tar archive or STDIN
  logs        Fetch the logs of a container
# pause       Pause all processes within one or more containers
  port        List port mappings or a specific mapping for the container
  rename      Rename a container
  restart     Restart one or more containers
  rm          Remove one or more containers
  rmi         Remove one or more images
  save        Save one or more images to a tar archive (streamed to STDOUT by default)
# start       Start one or more stopped containers
# stats       Display a live stream of container(s) resource usage statistics
# stop        Stop one or more running containers
  tag         Create a tag TARGET_IMAGE that refers to SOURCE_IMAGE
  top         Display the running processes of a container
# unpause     Unpause all processes within one or more containers
# update      Update configuration of one or more containers
# wait        Block until one or more containers stop, then print their exit codes

docker compose

About the docker compose Command

1
2
3
4
5
6
7
8

# start
docker compose up

# when you use volumes they are not deleted automatically. The command below deletes non-running containers and volumes too
docker system prune --volumes

# stop containers started with docker compose, and delete unneeded volumes along with them
docker compose down -v

docker-compose.yml

restart

The behavior when a container crashes. With restart, it always tries to bring it back up.

user

As described in Compose file version 3 reference | Docker Documentation, this corresponds to USER in docker run. And it lets you specify a user’s UID or username.

depends_on

Lets you define startup order. On shutdown the order is reversed. In particular, when web depends on db, it’s often written as depends_on: [ db ].

Incidentally, when you want to reach db from web, specifying something like db:5432 resolves the name.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

services:
  db:
    image: postgres:15
    environment:
      POSTGRES_DB: dev
      POSTGRES_USER: postgres
      POSTGRES_PASSWORD: postgres
    ports:
      - 5432:5432

  django:
    build: .
    command: python manage.py runserver 0.0.0.0:8000
    environment:
      # the host name is db, as started by docker
      DATABASE_URL: "postgres://postgres:postgres@db:5432/dev"
      SECRET_KEY: "sample_secret_key"
    volumes:
      - .:/djangoapp
    ports:
      - "8000:8000"
    depends_on:
      - db

How Docker Works

Docker Networking

Docker networking is a common place to get stuck.

Docker compose creates one network per application. Once the services on each container join the default network, they can be reached from other containers on the same network. They also become discoverable by host name and container name.

network driver

For Driver you can specify bridge or overlay. The default is bridge.

What is a bridge network?

It says it “corresponds to docker0,” but what does that mean…

Use bridge networks | Docker Documentation

1

A bridge network is a Link Layer device which forwards traffic between network segments. A bridge can be a hardware device or a software device running within a host machine's kernel.

So it’s a Link layer device that forwards communication between network segments. Docker uses a software bridge, and containers connected within a defined bridge network can communicate with each other.

In bridge mode, containers using the same docker daemon end up on the same bridge network

If you want to communicate between different daemons, apparently you either set up routing via the OS or use an overlay network.

What is a bridge connection | IT terminology dictionary

Use bridge networks | Docker Documentation

1

The default bridge network is considered a legacy detail of Docker and is not recommended for production use. Configuring it is a manual operation, and it has technical shortcomings.

So the default bridge network is described as legacy.

daemon.json holds the configuration of the bridge network the docker daemon uses.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11

To configure the default bridge network, you specify options in daemon.json. Here is an example daemon.json with several options specified. Only specify the settings you need to customize.

{
  "bip": "192.168.1.5/24",
  "fixed-cidr": "192.168.1.5/25",
  "fixed-cidr-v6": "2001:db8::/64",
  "mtu": 1500,
  "default-gateway": "10.20.1.1",
  "default-gateway-v6": "2001:db8:abcd::89",
  "dns": ["10.20.1.2","10.20.1.3"]
}

It appears to use the range 192.168.1.0 - 192.168.1.255.

Communication over a user-defined bridge

1

Containers connected to the same user-defined bridge network automatically expose all ports to each other, and no ports to the outside world. This allows containerized applications to communicate with each other easily, without accidentally opening access to the outside world.

On the same bridge network all ports can communicate with each other, but from the outside world no ports appear open.

How to Write a Dockerfile

Reference Material

ADD Adds a Layer, but COPY Does Not

Shrinking the File Size of a docker Image

Go’s src contains vendor, so the size gets large. But only /go/bin is used, so /go/src is entirely unnecessary (- 300MB).

1
2
3
4
5
6
7
8

[root@7a4baa5582f7 /]# du -sh /go/bin/* | sort -h
3.7M    /go/bin/yaml-patch
13M     /go/bin/om
28M     /go/bin/bosh-cli
[root@7a4baa5582f7 /]# du -sh /go/src/* | sort -h
2.2M    /go/src/gopkg.in
31M     /go/src/golang.org
299M    /go/src/github.com

/usr/local/go exists because the Go language was installed. However, Go isn’t executed after the image build, so it becomes unnecessary (since the Go binary has been produced) (- 300 MB).

1
2
3
4
5
6

[root@7a4baa5582f7 /]# du -sh /usr/local/go/* | sort -h
6.5M    /usr/local/go/api
12M     /usr/local/go/test
31M     /usr/local/go/bin
75M     /usr/local/go/src
214M    /usr/local/go/pkg

The contents of /usr/lib are Python and jvm, which are needed for basic operation, so they can’t be deleted.

1
2
3
4

5.7M    /usr/lib/udev
9.1M    /usr/lib/systemd
155M    /usr/lib/python2.7
166M    /usr/lib/jvm

The yum cache is unnecessary too (- 80MB).

1
2
3

[root@7a4baa5582f7 /]# du -sh /var/cache/* | sort -h
16K     /var/cache/ldconfig
81M     /var/cache/yum

Using multi stage build So It Builds Reliably from Anywhere

Multi stage builds are very handy. Just building the dockerfile assembles everything you need.

  1. The make build approach <- can speed things up, e.g. downloading binaries, but depends on the environment running make
  2. multi stage build <- setting up the environment is verbose, but all you need is docker build

Using ARG in a multi stage build

1
2
3
4
5
6

ARG JDK_VERSION="15"
ARG PLANTUML_VERSION="1.2020.2"
FROM openjdk:${JDK_VERSION}-jdk-alpine

RUN apk add --update-cache graphviz wget && \
    wget "http://downloads.sourceforge.net/project/plantuml/$PLANTUML_VERSION/plantuml.$PLANTUML_VERSION.jar"

This code doesn’t work: ARG is only recognized up to the first FROM. The correct form is as follows.

1
2
3
4
5
6

ARG JDK_VERSION="15"
FROM openjdk:${JDK_VERSION}-jdk-alpine

ARG PLANTUML_VERSION="1.2020.2"
RUN apk add --update-cache graphviz wget && \
    wget "http://downloads.sourceforge.net/project/plantuml/$PLANTUML_VERSION/plantuml.$PLANTUML_VERSION.jar"

Defining .dockerignore Lets You Restrict Which Files Go into the docker Image

Writing

1
2

.git
**/secret.yaml

excludes them from any folder. Note that the format differs somewhat from .gitignore

In this case, if you don’t exclude the .git folder, secrets could be restored with git reset, so be careful (though the real question is why a secret is in there at all).

Share

See also

Edit on GitHub