Docker linux volumes location

Docker: where is docker volume located for this compose file

I was setting up some materials for a trainning, when I came around this sample compose file: https://github.com/dockersamples/example-voting-app/blob/master/docker-compose.yml and I couldn’t find out how this volume is mounted, on lines 48 and 49 of the file:

Can someone explain me where is this volume on the host? Couldn’t find it and I wouldn’t like to keep any postgresql data dangling around after the containers are gone. Similar thing happens to the networks:

networks: front-tier: back-tier: 

2 Answers 2

Finding the volumes

Volumes like this are internal to Docker and stored in the Docker store (which is usually all under /var/lib/docker ). You can get a list of volumes:

$ docker volume ls DRIVER VOLUME NAME local 1c59d5b7e90e9173ca30a7fcb6b9183c3f5a37bd2505ca78ad77cf4062bd0465 local 2f13b0cec834a0250845b9dcb2bce548f7c7f35ed9cdaa7d5990bf896e952d02 local a3d54ec4582c3c7ad5a6172e1d4eed38cfb3e7d97df6d524a3edd544dc455917 local e6c389d80768356cdefd6c04f6b384057e9fe2835d6e1d3792691b887d767724 

You can find out exactly where the volume is stored on your system if you want to:

$ docker inspect 1c59d5b7e90e9173ca30a7fcb6b9183c3f5a37bd2505ca78ad77cf4062bd0465 [ < "Driver": "local", "Labels": null, "Mountpoint": "/var/lib/docker/volumes/1c59d5b7e90e9173ca30a7fcb6b9183c3f5a37bd2505ca78ad77cf4062bd0465/_data", "Name": "1c59d5b7e90e9173ca30a7fcb6b9183c3f5a37bd2505ca78ad77cf4062bd0465", "Options": <>, "Scope": "local" > ] 

Cleaning up unused volumes

As far as just ensuring that things are not left dangling, you can use the prune commands, in this case docker volume prune . That will give you this output, and you choose whether to continue pruning or not.

$ docker volume prune WARNING! This will remove all volumes not used by at least one container. Are you sure you want to continue? [y/N] 

«Empty» definitions in docker-compose.yml

There is a tendency to accept these «empty» definitions for things like volumes and networks when you don’t need to do anything other than define that a volume or network should exist. That is, if you want to create it, but are okay with the default settings, then there is no particular reason to specify the parameters.

Источник

Where are Docker Images Stored? Docker Container Paths Explained

Sebastian Sigl

Docker has been widely adopted and is used to run and scale applications in production. Additionally, it can be used to start applications quickly by executing a single Docker command.

Companies also are investing more and more effort into improving development in local and remote Docker containers, which comes with a lot of advantages as well.

You can get the basic information about your Docker configuration by executing:

$ docker info . Storage Driver: overlay2 Docker Root Dir: /var/lib/docker . 

The output contains information about your storage driver and your docker root directory.

The storage location of Docker images and containers

A Docker container consists of network settings, volumes, and images. The location of Docker files depends on your operating system. Here is an overview for the most used operating systems:

• Ubuntu: /var/lib/docker/
• Fedora: /var/lib/docker/
• Debian: /var/lib/docker/
• Windows: C:\ProgramData\DockerDesktop
• MacOS: ~/Library/Containers/com.docker.docker/Data/vms/0/

In macOS and Windows, Docker runs Linux containers in a virtual environment. Therefore, there are some additional things to know.

Docker for Mac

Docker is not natively compatible with macOS, so Hyperkit is used to run a virtual image. Its virtual image data is located in:

Within the virtual image, the path is the default Docker path /var/lib/docker .

You can investigate your Docker root directory by creating a shell in the virtual environment:

$ screen ~/Library/Containers/com.docker.docker/Data/vms/0/tty 

You can kill this session by pressing Ctrl+a, followed by pressing k and y.

Docker for Windows

On Windows, Docker is a bit fractioned. There are native Windows containers that work similarly to Linux containers. Linux containers are run in a minimal Hyper-V based virtual environment.

The configuration and the virtual image to execute linux images are saved in the default Docker root folder.

If you inspect regular images then you will get linux paths like:

$ docker inspect nginx . "UpperDir": "/var/lib/docker/overlay2/585. 9eb/diff" . 

You can connect to the virtual image by:

docker run -it --privileged --pid=host debian nsenter -t 1 -m -u -i sh

There, you can go to the referenced location:

$ cd /var/lib/docker/overlay2/585. 9eb/ $ ls -lah drwx------ 4 root root 4.0K Feb 6 06:56 . drwx------ 13 root root 4.0K Feb 6 09:17 .. drwxr-xr-x 3 root root 4.0K Feb 6 06:56 diff -rw-r--r-- 1 root root 26 Feb 6 06:56 link -rw-r--r-- 1 root root 57 Feb 6 06:56 lower drwx------ 2 root root 4.0K Feb 6 06:56 work

The internal structure of the Docker root folder

Inside /var/lib/docker , different information is stored. For example, data for containers, volumes, builds, networks, and clusters.

$ ls -la /var/lib/docker total 152 drwx--x--x 15 root root 4096 Feb 1 13:09 . drwxr-xr-x 13 root root 4096 Aug 1 2019 .. drwx------ 2 root root 4096 May 20 2019 builder drwx------ 4 root root 4096 May 20 2019 buildkit drwx------ 3 root root 4096 May 20 2019 containerd drwx------ 2 root root 12288 Feb 3 19:35 containers drwx------ 3 root root 4096 May 20 2019 image drwxr-x--- 3 root root 4096 May 20 2019 network drwx------ 6 root root 77824 Feb 3 19:37 overlay2 drwx------ 4 root root 4096 May 20 2019 plugins drwx------ 2 root root 4096 Feb 1 13:09 runtimes drwx------ 2 root root 4096 May 20 2019 swarm drwx------ 2 root root 4096 Feb 3 19:37 tmp drwx------ 2 root root 4096 May 20 2019 trust drwx------ 15 root root 12288 Feb 3 19:35 volumes 

Docker images

The heaviest contents are usually images. If you use the default storage driver overlay2, then your Docker images are stored in /var/lib/docker/overlay2 . There, you can find different files that represent read-only layers of a Docker image and a layer on top of it that contains your changes.

Let’s explore the content by using an example:

$ docker image pull nginx $ docker image inspect nginx [ < "Id": "sha256:207. 6e1", "RepoTags": [ "nginx:latest" ], "RepoDigests": [ "nginx@sha256:ad5. c6f" ], "Parent": "", . "Architecture": "amd64", "Os": "linux", "Size": 126698063, "VirtualSize": 126698063, "GraphDriver": < "Data": < "LowerDir": "/var/lib/docker/overlay2/585. 9eb/diff: /var/lib/docker/overlay2/585. 9eb/diff", "MergedDir": "/var/lib/docker/overlay2/585. 9eb/merged", "UpperDir": "/var/lib/docker/overlay2/585. 9eb/diff", "WorkDir": "/var/lib/docker/overlay2/585. 9eb/work" >, . 

The LowerDir contains the read-only layers of an image. The read-write layer that represents changes are part of the UpperDir. In my case, the NGINX UpperDir folder contains the log files:

$ ls -la /var/lib/docker/overlay2/585. 9eb/diff total 8 drwxr-xr-x 2 root root 4096 Feb 2 08:06 . drwxr-xr-x 3 root root 4096 Feb 2 08:06 .. lrwxrwxrwx 1 root root 11 Feb 2 08:06 access.log -> /dev/stdout lrwxrwxrwx 1 root root 11 Feb 2 08:06 error.log -> /dev/stderr

The MergedDir represents the result of the UpperDir and LowerDir that is used by Docker to run the container. The WorkDir is an internal directory for overlay2 and should be empty.

Docker Volumes

It is possible to add a persistent store to containers to keep data longer than the container exists or to share the volume with the host or with other containers. A container can be started with a volume by using the -v option:

$ docker run --name nginx_container -v /var/log nginx

We can get information about the connected volume location by:

$ docker inspect nginx_container . "Mounts": [ < "Type": "volume", "Name": "1e4. d9c", "Source": "/var/lib/docker/volumes/1e4. d9c/_data", "Destination": "/var/log", "Driver": "local", "Mode": "", "RW": true, "Propagation": "" >], . 

The referenced directory contains files from the location /var/log of the NGINX container.

$ ls -lah /var/lib/docker/volumes/1e4. d9c/_data total 88 drwxr-xr-x 4 root root 4.0K Feb 3 21:02 . drwxr-xr-x 3 root root 4.0K Feb 3 21:02 .. drwxr-xr-x 2 root root 4.0K Feb 3 21:02 apt -rw-rw---- 1 root 43 0 Jan 30 00:00 btmp -rw-r--r-- 1 root root 34.7K Feb 2 08:06 dpkg.log -rw-r--r-- 1 root root 3.2K Feb 2 08:06 faillog -rw-rw-r-- 1 root 43 29.1K Feb 2 08:06 lastlog drwxr-xr-x 2 root root 4.0K Feb 3 21:02 nginx -rw-rw-r-- 1 root 43 0 Jan 30 00:00 w 

Clean up space used by Docker

It is recommended to use the Docker command to clean up unused containers. Container, networks, images, and the build cache can be cleaned up by executing:

Additionally, you can also remove unused volumes by executing:

Summary

Docker is an important part of many people’s environments and tooling. Sometimes, Docker feels a bit like magic by solving issues in a very smart way without telling the user how things are done behind the scenes. Still, Docker is a regular tool that stores its heavy parts in locations that can be opened and changed.

Sometimes, storage can fill up quickly. Therefore, it’s useful to inspect its root folder, but it is not recommended to delete or change any files manually. Instead, the prune commands can be used to free up disk space.

I hope you enjoyed the article. If you like it and feel the need for a round of applause, follow me on Twitter. I work at eBay Kleinanzeigen, one of the biggest classified companies globally. By the way, we are hiring!

References

Источник

Where docker volumes are located?

Need to know where docker volumes are located when using the docker machine on macOS. The installation is using boot2docker, so the VM works behind. Example:

docker volume create test-data 

5 Answers 5

It’s inside the virtual machine and isn’t directly accessible from the host.

Debug-level commands like docker volume inspect will give you a path, but they really are only for emergency debugging and not for routine use. If you have a way to get a shell in the VM you can see that path, but you really shouldn’t be directly accessing files there, and you shouldn’t be routinely docker inspect ing anything.

macOS use a virtual machine it’s different to linux where you can access to volumes from /var/lib/docker/volumes. For macOS you should connect to a VM to find your volumes.

If you use persistent data volumes in Docker, and you want to access them with command-line.

If your docker host is Linux, that’s not a problem; you can find Docker volumes by /var/lib/docker/volumes path.

However, that’s not the case when you use Docker for Mac. Try to cd /var/lib/docker/volumes from your MacOS terminal, you ‘ll get nothing.

You see, your Mac machine isn’t a real Docker host. Docker for Mac runs a virtual machine and hides it from you to make things simple.

So, to access persistent volumes created by Docker for Mac, you need to connect on that VM.

In order to accomplish this, we need to use a serial terminal on Mac. There’s a terminal application called “screen” that’s going to help us.

We need to “screen into” the Docker driver by executing a command:

1. screen ~/Library/Containers/com.docker.docker/Data/com.docker.driver.amd64-linux/tty
2. You should see a blank screen, just press Enter , and after a while, you should see a command line prompt
3. Now you’re inside Docker’s VM and you can cd into volumes dir by typing: cd /var/lib/docker/volumes

If you need to transfer files from your MacOS host into Docker host you can refer to File Sharing

Источник