Containers isolation by namespace

This page details the additional steps required to isolate containers using Linux user namespaces.

The procedure differs depending on whether the Cyberwatch instance already has Docker volumes containing data or not:

Instance without existing volume (new installation): follow the section “Isolating an instance without existing volume”.
Already installed instance (existing volumes): follow the section “Isolating an instance with existing volumes”.

Isolating an instance without existing volume

Ensure that Docker and Cyberwatch are installed

Create the cyberwatch user:

sudo useradd --create-home --shell /bin/bash cyberwatch

Edit the /etc/subuid file:
```
cyberwatch:231072:65536
cyberwatch:1001:1
```
The first line defines the mapping of user/group ids in a user namespace. This line is usually added automatically by the system when creating the user, but some systems do not do this automatically (certain versions of CentOS, for example).
For example, cyberwatch:231072:65536 means that the cyberwatch user can use 65536 user ids starting from id 231072.
The line cyberwatch:1001:1 ensures that files created by root belong to the user with id 1001 (replace with the id of the cyberwatch user).

Edit the /etc/subgid file:

cyberwatch:231072:65536
cyberwatch:1001:1

Reboot the server for the changes to take effect:
```
sudo reboot
```

Configure docker to enable the userns-remap option:

cat >> /etc/docker/daemon.json <<EOL
{
  "userns-remap": "cyberwatch"
}
EOL

Restart docker:
```
systemctl restart docker
```

Isolating an instance with existing volumes

Enabling userns-remap changes the location of Docker data (/var/lib/docker/<uid>.<gid>): Docker then creates new empty volumes, not mapped to the existing volumes.

Without a database backup performed before isolation and restored afterwards, the instance data will not be recovered and the instance will start empty.

To enable isolation on an already installed Cyberwatch instance without losing its data:

Perform a backup of Cyberwatch before making any changes, following the Procedure to backup and restore Cyberwatch.
Set up isolation by following all the steps in the section “Isolating an instance without existing volume” above.
Reconfigure Cyberwatch:
```
sudo cyberwatch configure
```
Restore the backup made in step 1, following the Procedure to backup and restore Cyberwatch.

nginx certificate folder (Cyberwatch 5.31 and later)

Starting with Cyberwatch 5.31, nginx certificates are stored in the /etc/cyberwatch/ssl folder on the host instead of a Docker volume. On startup, Cyberwatch assigns this folder to the root user (UID and GID 0) and restricts access to its owner only.

With userns-remap, the host’s UID 0 does not correspond to root inside the containers. The nginx container can then no longer read its certificates, which prevents it from starting (container restarts in a loop, HTTPS interface unavailable).

To fix this behavior, you need to indicate to Cyberwatch the host UID and GID corresponding to root inside the containers.

Identify the host UID and GID corresponding to root inside the containers. When userns-remap is active, Docker displays them in the name of its active data directory:
```
docker info --format '{{.DockerRootDir}}'
```
The command returns a path like /var/lib/docker/<uid>.<gid>, for example /var/lib/docker/165536.165536: root inside the containers then corresponds to UID 165536 and GID 165536 on the host.
Enter these values in the /etc/cyberwatch/config.env file:
```
CBW_NGINX_UID=165536
CBW_NGINX_GID=165536
```
Restart Cyberwatch to apply the new permissions:
```
sudo cyberwatch restart
```

Troubleshooting

Issues that may arise from enabling userns-remap are generally related to permissions on volumes.

If, after enabling userns-remap on an already installed instance, the application starts without its data, this means that new volumes have been created: restore the backup made before isolation (see the section “Isolating an instance with existing volumes”).

It may be useful to check the logs of the database container to rule out permissions issues:

sudo cyberwatch logs db

Since Cyberwatch 5.31, if the nginx container does not start or if the HTTPS interface is unavailable after enabling userns-remap, check the configuration of CBW_NGINX_UID and CBW_NGINX_GID (see the nginx certificate folder section above). The nginx container logs help with diagnosis:

sudo cyberwatch logs nginx