Linux docker memory limit

Docker – Setting Memory And CPU Limits

Nowadays, container technologies are very popular. We can see that variety of applications are running as containers. One of the prime reasons for its wide adoption is that it increases computing utilization by sharing and/or isolating its resources. Resource sharing is noteworthy, but an unthoughtful configuration can lead to a noisy neighbor situation.

In computing, a noisy neighbor situation occurs when a process or group of processes over-consumes a host’s resources. This situation negatively impacts other processes running on the same host. It ultimately degrades the overall system’s performance as the host becomes less responsive due to low resources.

This tutorial will discuss how to set memory and CPU limits in Docker to mitigate the noisy neighbor situation.

1. Creating Container Without Limits

By default, Docker doesn’t set any resource constraints on a container. It allows the container to access all system resources. Let’s understand this with an example.

First, display the CPU and memory configuration of the host machine:

$ docker info | grep -iE "CPUs|Memory" CPUs: 4 Total Memory: 7.714GiB

The above output shows that the host machine has 4 CPUs and 7.714 GiB of memory.

Now, let’s create a new container and observe its resource limits using the docker stats command:

$ docker container run --rm -it -d --name web-server nginx:alpine $ docker stats web-server --no-stream --format ">" | python3 -m json.tool < "BlockIO": "0B / 12.3kB", "CPUPerc": "0.00%", "Container": "web-server", "ID": "2def8ff5e138", "MemPerc": "0.05%", "MemUsage": "3.734MiB / 7.714GiB", "Name": "web-server", "NetIO": "5.46kB / 0B", "PIDs": "5" >

The above output shows that the memory limit is 7.714 GiB which is the same as the host’s memory. The MemUsage field indicates it.

In this example, the output is redirected to the Python interpreter to print it in a pretty format.

2. Setting Resources Limits when Creating a Container

In Docker, we can create a container using the following ways:

Docker supports setting up resource limits using both of these options.

2.1. Using container run Child Command

The simplest way to set resource limits is using the container run child command. This command allows us to set memory and CPU limits.

2.1.1. Setting Memory Limits

To set a hard memory limit, use the —memory option with the container run child command. Docker doesn’t allow a container to use more than a given amount of user or system memory after setting this limit.

$ docker container run --rm -it -d --name mem-limit-demo --memory=256m nginx:alpine

Now, let’s check its memory limits:

$ docker stats mem-limit-demo --no-stream --format ">" | python3 -m json.tool < "BlockIO": "0B / 12.3kB", "CPUPerc": "0.00%", "Container": "mem-limit-demo", "ID": "b46befb6d196", "MemPerc": "1.45%", "MemUsage": "3.711MiB / 256MiB", "Name": "mem-limit-demo", "NetIO": "3.83kB / 0B", "PIDs": "5" >

In the above output, we can see that Docker has applied a memory limit of 256 MiB. The MemUsage field indicates this. The –memory option accepts a positive integer as an argument, followed by a suffix of b, k, m, g, to indicate bytes, kilobytes, megabytes, and gigabytes, respectively.

In addition to this, Docker allows us to set soft memory limits. This option allows a container to use as much memory as it requires unless the kernel detects memory contention. To set soft memory limits, use the –memory-reservation option with the container run child command:

$ docker container run --rm -it -d --name soft-mem-limit-demo --memory=1g --memory-reservation=512m nginx:alpine

In this example, we have set the hard memory limits to 1GiB and the soft memory limits to 512MiB.

We should note that the soft memory limits must be less than the hard ones. Because it is a soft limit and there isn’t a guarantee that the container will not exceed it.

Now, let’s verify the hard memory limits of the container:

$ docker stats soft-mem-limit-demo --no-stream --format ">" | python3 -m json.tool < "BlockIO": "135kB / 12.3kB", "CPUPerc": "0.00%", "Container": "soft-mem-limit-demo", "ID": "9b748ec04b2a", "MemPerc": "0.38%", "MemUsage": "3.863MiB / 1GiB", "Name": "soft-mem-limit-demo", "NetIO": "3.98kB / 0B", "PIDs": "5" >

Docker doesn’t provide any command to show the soft memory limits. However, we can find these details from the cgroups.

Let’s go inside the container and list the/sys/fs/cgroup/memory contents.low file:

$ docker exec -it soft-mem-limit-dem sh # cat /sys/fs/cgroup/memory.low 536870912 # exit

This file contains an integer value. It’s a Byte representation of 512 MiB, the same as the soft limits.

2.1.2. Setting CPU Limits

By default, containers can use the unlimited cycles of the host machine’s CPU. To restrict it, use the –cpus option with the container run child command.

$ docker container run --rm -it -d --name cpu-limit-demo --cpus=1 nginx:alpine

With this setting, Docker guarantees that the container can use at most 1 CPU at any given time.

Additionally, Docker allows us to set limits on a specific CPU. To achieve this, use the –cpuset-cpus option:

$ docker container run --rm -it -d --name cpu-sets-demo --cpus=1 --cpuset-cpus=2 nginx:alpine

In this example, we have set limits on the third CPU.

Now, to verify this, let’s go inside the container and list the contents of the /sys/fs/cgroup/cpuset.cpus file:

$ docker exec -it cpu-sets-demo sh # cat /sys/fs/cgroup/cpuset.cpus 2 # exit

This file contains an integer value. It represents a CPU number. It is important to note that the CPU numbering starts from 0.

2.2. Using docker-compose

Docker allows us to set resource limits using the docker-compose as well. Let’s understand this with an example.

First, create a docker-compose.yml configuration file:

$ cat docker-compose.yml version: "3.9" services: web-server: image: nginx:alpine container_name: res-limits-demo mem_limit: "1g" mem_reservation: "512m" cpus: "1" cpuset: "2"

• mem_limit – represents the hard memory limits. We have set it to 1GiB
• mem_reservation: represents the soft memory limits. We have set it to 512MiB
• cpus – represents the CPU limit. We have set it to 1
• cpuset – represents the limit on a specific CPU. We have set it on the third CPU

Let’s deploy the configuration to create a container with the specified limits:

Now, let’s verify the CPU and memory limits of the container:

$ docker stats res-limits-demo --no-stream --format ">" | python3 -m json.tool < "BlockIO": "0B / 12.3kB", "CPUPerc": "0.00%", "Container": "res-limits-demo", "ID": "0d7055f8eb31", "MemPerc": "0.18%", "MemUsage": "1.836MiB / 1GiB", "Name": "res-limits-demo", "NetIO": "11.2kB / 0B", "PIDs": "2" >$ docker exec -it res-limits-demo sh # cat /sys/fs/cgroup/memory.low 536870912 # cat /sys/fs/cgroup/cpuset.cpus 2 # exit

Here, we can see that the docker-compose has set CPU and memory limits correctly.

3. Setting Resources Limits on a Running Container

We used container run and docker-compose commands in the previous section to set resource limits. However, one of the major drawbacks of these commands is that they don’t allow updating limits dynamically. It means that we have to specify all the limits while creating a container. One more limitation of the container run command is that it only sets limits on a single container.

To overcome all these limitations, use the container update command. This command dynamically updates the configuration of the multiple containers. Hence we can use it with the running containers.

To understand this, first create a container without any resource limits:

$ docker container run --rm -it -d --name no-limits-demo nginx:alpine

Now, let’s set the CPU and memory limits using the container update command:

$ docker update --memory=1g --memory-reservation=512m --cpus=1 --cpuset-cpus=2 --memory-swap -1 no-limits-demo

In this example, we have used the –memory-swap -1 option. This option increases the swap memory limit up to the amount available on the host machine. In the absence of this option, the command generated the following error:

Error response from daemon: Cannot update container: Memory limit should be smaller than already set memoryswap limit, update the memoryswap at the same time

Let’s verify that the CPU and memory limits were appropriately updated for the container:

$ docker stats no-limits-demo --no-stream --format ">" | python3 -m json.tool < "BlockIO": "0B / 12.3kB", "CPUPerc": "0.00%", "Container": "no-limits-demo", "ID": "ef6bd714038a", "MemPerc": "0.36%", "MemUsage": "3.691MiB / 1GiB", "Name": "no-limits-demo", "NetIO": "5.29kB / 0B", "PIDs": "5" >$ docker exec -it no-limits-demo sh # cat /sys/fs/cgroup/memory.low 536870912 # cat /sys/fs/cgroup/cpuset.cpus 2 / # exit

It is important to note that, the container update child command is not supported for the containers that are running on the Windows host.

4. Conclusion

This tutorial discussed setting CPU and memory limits to keep the host machine healthy and responsive. First, we discussed using container run and docker-compose commands to set limits while creating containers. Then we saw how to dynamically use the container update child command to update the configuration.

Источник

Установка ограничений памяти и ЦП в Docker

Есть много случаев, когда нам нужно ограничить использование ресурсов на хост-компьютере докера.

В этом руководстве мы узнаем, как установить лимит памяти и ЦП для контейнеров Docker.

2. Установка лимита ресурсов с запуском докера ​

Мы можем установить лимиты ресурсов напрямую, используя команду запуска docker . Это простое решение. Однако ограничение будет применяться только к одному конкретному исполнению образа.

2.1. Память​

Например, давайте ограничим память, которую может использовать контейнер, до 512 мегабайт. Чтобы ограничить память, нам нужно использовать параметр m :

Мы также можем установить мягкое ограничение, называемое резервированием. Он активируется, когда докер обнаруживает нехватку памяти на хост-компьютере:

 $ docker run -m 512m --memory-reservation=256m nginx 

2.2. Процессор​

По умолчанию доступ к вычислительной мощности хост-машины неограничен. Мы можем установить лимит процессоров, используя параметр cpus . Например, давайте ограничим наш контейнер для использования не более двух процессоров:

Мы также можем указать приоритет выделения ЦП. По умолчанию 1024, чем больше число, тем выше приоритет:

 $ docker run --cpus=2 --cpu-shares=2000 nginx 

Как и в случае с резервированием памяти, доли ЦП играют основную роль, когда вычислительной мощности недостаточно и ее необходимо разделить между конкурирующими процессами.

3. Установка ограничения памяти с помощью файла docker-compose​

Мы можем добиться аналогичных результатов, используя файлы docker-compose . Имейте в виду, что формат и возможности будут различаться в разных версиях docker-compose .

3.1. Версии 3 и новее с docker swarm ​

Дадим сервису Nginx ограничение в половину ЦП и 512 мегабайт памяти и резервирование четверти ЦП и 128 мегабайт памяти. Нам нужно создать сегменты « deploy », а затем « resources » в нашей конфигурации сервиса :

 services:   service:   image: nginx  deploy:   resources:   limits:   cpus: 0.50   memory: 512M  reservations:   cpus: 0.25   memory: 128M 

Чтобы воспользоваться сегментом развертывания в файле docker-compose, нам нужно использовать команду docker stack . Чтобы развернуть стек в рой, мы запускаем команду deploy :

 $ docker stack deploy --compose-file docker-compose.yml bael_stack 

3.2. Версия 2 с docker-compose ​

В более старых версиях docker-compose мы можем установить ограничения ресурсов на том же уровне, что и основные свойства службы. Они также имеют немного другое название:

 service:  image: nginx  mem_limit: 512m  mem_reservation: 128M  cpus: 0.5   ports:  - "80:80" 

Чтобы создать настроенные контейнеры, нам нужно запустить команду docker-compose :

4. Проверка использования ресурсов​

После того, как мы установили ограничения, мы можем проверить их с помощью команды docker stats :

 $ docker stats CONTAINER ID NAME CPU % MEM USAGE / LIMIT MEM % NET I/O BLOCK I/O PIDS 8ad2f2c17078 bael_stack_service.1.jz2ks49finy61kiq1r12da73k 0.00% 2.578MiB / 512MiB 0.50% 936B / 0B 0B / 0B 2 

5. Резюме​

В этом руководстве мы рассмотрели способы ограничения доступа докера к ресурсам хоста. Мы рассмотрели использование команд docker run и docker-compose . Наконец, мы контролировали потребление ресурсов с помощью docker stats .

Источник