Collected Notes on Docker
About the docker Command
Operations on Images and Containers
Renaming with docker tag
There are situations where an image ends up as a hash after a build and you want to give it a clearer name. Looks like tag is the only way?
How Docker images work - Qiita
Copying a file out of a docker image
1docker create --name temp-image some-image
2docker cp temp-image:/some/dir/file.tmp file.tmp
3docker rm temp-image
Running Containers
docker run = docker create & docker start
run creates a container and starts it. At that point, using -d to detach leaves the process running in the background. -i brings the container down when you exit from it.
Adding privileged makes it a privileged container that can do anything
1docker run --it alpine:latest --privileged
docker exec runs a command in a running docker process
docker exec -it <container name> lets you get into a process currently running under docker (provided a shell is on the PATH).
docker ps -a shows everything including stopped processes
1sudo docker ps -a
2
3CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
471e0cf6a50a6 101779243274 "/bin/bash" 5 weeks ago Exited (137) 2 weeks ago angry_fermat
5d5577c8aca3f bbe77145eb18 "/bin/bash" 5 weeks ago Exited (137) 2 weeks ago eager_carson
6b8c0b1a734a6 101779243274 "bash" 6 weeks ago Up 8 days interesting_shtern
73ce1af5ce44c bbe77145eb18 "/bin/bash" 8 weeks ago Exited (137) 2 weeks ago hardcore_torvalds
80456af55e878 bbe77145eb18 "/bin/bash" 2 months ago Exited (137) 2 weeks ago lucid_swanson
Cleaning Up Images and Containers
Deleting images whose TAG is none
1docker image prune
This appears to be available from docker 1.25. Before that you had to use several commands, as below.
1docker rmi $(docker images -f "dangling=true" -q)
Deleting docker images in a for loop
1arr=("image_id1" "image_id2" "image_id3"); for i in "${arr[@]}" ; do sudo docker rmi -f $i ; done
Deleting containers that have exited and stopped
1docker rm $(docker ps -aq)
deletes the stopped containers.
$(docker ps -aq) also targets running ones, but docker rm does not delete running containers.
Bringing the container down when you exit docker run
1docker run --rm -it --name="default" alpine /bin/sh
Adding --rm makes the container come down when you exit the command.
With this, you can experiment freely, and once you docker commit or save you have a Docker image, so all that’s left is to delete it.
Commands I Use Often from the Help Output
1# subcommands I personally rarely use are commented out
2$ docker -h
3
4Common Commands:
5 run Create and run a new container from an image
6 exec Execute a command in a running container
7 ps List containers
8 build Build an image from a Dockerfile
9 pull Download an image from a registry
10# push Upload an image to a registry
11 images List images
12# login Log in to a registry
13# logout Log out from a registry
14# search Search Docker Hub for images
15 version Show the Docker version information
16 info Display system-wide information
17
18Management Commands:
19# builder Manage builds
20 container Manage containers
21# context Manage contexts
22 image Manage images
23# manifest Manage Docker image manifests and manifest lists
24 network Manage networks
25# plugin Manage plugins
26 system Manage Docker
27# trust Manage trust on Docker images
28 volume Manage volumes
29
30Commands:
31 attach Attach local standard input, output, and error streams to a running container
32# commit Create a new image from a container's changes
33 cp Copy files/folders between a container and the local filesystem
34# create Create a new container
35# diff Inspect changes to files or directories on a container's filesystem
36# events Get real time events from the server
37# export Export a container's filesystem as a tar archive
38# history Show the history of an image
39 import Import the contents from a tarball to create a filesystem image
40# inspect Return low-level information on Docker objects
41 kill Kill one or more running containers
42 load Load an image from a tar archive or STDIN
43 logs Fetch the logs of a container
44# pause Pause all processes within one or more containers
45 port List port mappings or a specific mapping for the container
46 rename Rename a container
47 restart Restart one or more containers
48 rm Remove one or more containers
49 rmi Remove one or more images
50 save Save one or more images to a tar archive (streamed to STDOUT by default)
51# start Start one or more stopped containers
52# stats Display a live stream of container(s) resource usage statistics
53# stop Stop one or more running containers
54 tag Create a tag TARGET_IMAGE that refers to SOURCE_IMAGE
55 top Display the running processes of a container
56# unpause Unpause all processes within one or more containers
57# update Update configuration of one or more containers
58# wait Block until one or more containers stop, then print their exit codes
docker compose
About the docker compose Command
1# start
2docker compose up
3
4# when you use volumes they are not deleted automatically. The command below deletes non-running containers and volumes too
5docker system prune --volumes
6
7# stop containers started with docker compose, and delete unneeded volumes along with them
8docker compose down -v
docker-compose.yml
restart
The behavior when a container crashes. With restart, it always tries to bring it back up.
user
As described in Compose file version 3 reference | Docker Documentation, this corresponds to USER in docker run. And it lets you specify a user’s UID or username.
depends_on
Lets you define startup order. On shutdown the order is reversed.
In particular, when web depends on db, it’s often written as depends_on: [ db ].
Incidentally, when you want to reach db from web, specifying something like db:5432 resolves the name.
1services:
2 db:
3 image: postgres:15
4 environment:
5 POSTGRES_DB: dev
6 POSTGRES_USER: postgres
7 POSTGRES_PASSWORD: postgres
8 ports:
9 - 5432:5432
10
11 django:
12 build: .
13 command: python manage.py runserver 0.0.0.0:8000
14 environment:
15 # the host name is db, as started by docker
16 DATABASE_URL: "postgres://postgres:postgres@db:5432/dev"
17 SECRET_KEY: "sample_secret_key"
18 volumes:
19 - .:/djangoapp
20 ports:
21 - "8000:8000"
22 depends_on:
23 - db
How Docker Works
Docker Networking
Docker networking is a common place to get stuck.
- Networking in Compose — Docker-docs-ja 17.06.Beta documentation
- Understanding the whole picture of Docker, part 2 - Qiita
Docker compose creates one network per application. Once the services on each container join the default network, they can be reached from other containers on the same network. They also become discoverable by host name and container name.
network driver
For Driver you can specify bridge or overlay. The default is bridge.
What is a bridge network?
It says it “corresponds to docker0,” but what does that mean…
Use bridge networks | Docker Documentation
1A bridge network is a Link Layer device which forwards traffic between network segments. A bridge can be a hardware device or a software device running within a host machine's kernel.
So it’s a Link layer device that forwards communication between network segments. Docker uses a software bridge, and containers connected within a defined bridge network can communicate with each other.
In bridge mode, containers using the same docker daemon end up on the same bridge network
If you want to communicate between different daemons, apparently you either set up routing via the OS or use an overlay network.
What is a bridge connection | IT terminology dictionary
Use bridge networks | Docker Documentation
1The default bridge network is considered a legacy detail of Docker and is not recommended for production use. Configuring it is a manual operation, and it has technical shortcomings.
So the default bridge network is described as legacy.
daemon.json holds the configuration of the bridge network the docker daemon uses.
1To configure the default bridge network, you specify options in daemon.json. Here is an example daemon.json with several options specified. Only specify the settings you need to customize.
2
3{
4 "bip": "192.168.1.5/24",
5 "fixed-cidr": "192.168.1.5/25",
6 "fixed-cidr-v6": "2001:db8::/64",
7 "mtu": 1500,
8 "default-gateway": "10.20.1.1",
9 "default-gateway-v6": "2001:db8:abcd::89",
10 "dns": ["10.20.1.2","10.20.1.3"]
11}
It appears to use the range 192.168.1.0 - 192.168.1.255.
Communication over a user-defined bridge
1Containers connected to the same user-defined bridge network automatically expose all ports to each other, and no ports to the outside world. This allows containerized applications to communicate with each other easily, without accidentally opening access to the outside world.
On the same bridge network all ports can communicate with each other, but from the outside world no ports appear open.
How to Write a Dockerfile
Reference Material
- Best practices for writing Dockerfiles | Docker Documentation
- Security best practices Top 20 Dockerfile best practices | Sysdig
ADD Adds a Layer, but COPY Does Not
Shrinking the File Size of a docker Image
Go’s src contains vendor, so the size gets large. But only /go/bin is used, so /go/src is entirely unnecessary (- 300MB).
1[root@7a4baa5582f7 /]# du -sh /go/bin/* | sort -h
23.7M /go/bin/yaml-patch
313M /go/bin/om
428M /go/bin/bosh-cli
5[root@7a4baa5582f7 /]# du -sh /go/src/* | sort -h
62.2M /go/src/gopkg.in
731M /go/src/golang.org
8299M /go/src/github.com
/usr/local/go exists because the Go language was installed. However, Go isn’t executed after the image build, so it becomes unnecessary (since the Go binary has been produced) (- 300 MB).
1[root@7a4baa5582f7 /]# du -sh /usr/local/go/* | sort -h
26.5M /usr/local/go/api
312M /usr/local/go/test
431M /usr/local/go/bin
575M /usr/local/go/src
6214M /usr/local/go/pkg
The contents of /usr/lib are Python and jvm, which are needed for basic operation, so they can’t be deleted.
The yum cache is unnecessary too (- 80MB).
Using multi stage build So It Builds Reliably from Anywhere
Multi stage builds are very handy. Just building the dockerfile assembles everything you need.
- The make build approach <- can speed things up, e.g. downloading binaries, but depends on the environment running make
- multi stage build <- setting up the environment is verbose, but all you need is docker build
Using ARG in a multi stage build
1ARG JDK_VERSION="15"
2ARG PLANTUML_VERSION="1.2020.2"
3FROM openjdk:${JDK_VERSION}-jdk-alpine
4
5RUN apk add --update-cache graphviz wget && \
6 wget "http://downloads.sourceforge.net/project/plantuml/$PLANTUML_VERSION/plantuml.$PLANTUML_VERSION.jar"
This code doesn’t work: ARG is only recognized up to the first FROM. The correct form is as follows.
1ARG JDK_VERSION="15"
2FROM openjdk:${JDK_VERSION}-jdk-alpine
3
4ARG PLANTUML_VERSION="1.2020.2"
5RUN apk add --update-cache graphviz wget && \
6 wget "http://downloads.sourceforge.net/project/plantuml/$PLANTUML_VERSION/plantuml.$PLANTUML_VERSION.jar"
Defining .dockerignore Lets You Restrict Which Files Go into the docker Image
Writing
excludes them from any folder. Note that the format differs somewhat from .gitignore
In this case, if you don’t exclude the .git folder, secrets could be restored with git reset, so be careful (though the real question is why a secret is in there at all).