Modifying pre-installed files in a container

When creating a backend URCap contribution it is sometimes needed to be able to modify files that is being added during the Docker build. An example could be installing an application that includes a predefined configuration file that must be edited during runtime. Such an application can be installed in the Dockerfile like this:

FROM docker.urrd.dk/python:3.11-alpine
...
  RUN apt update && apt install myApplication
...

Another example is the URCap itself that prefers to include a predefined configuration file that can be changed at runtime.

Problem

A backend URCap contribution container is read-only by design, to prevent loss of user data when the container is removed. Writable files, such as user data, must be stored using mounts defined in the manifest, as outlined in the Data Storage section. However, when a folder is mounted into the container, any existing folder with the same name inside the container will be hidden by the mounted one. As a result, the original contents of that folder become inaccessible from within the container.

For example myApplication installed as in the above Dockerfile includes the predefined settings.conf installed in the /var folder. Because the application wants to write to settings.conf, /var is mounted in the URCap’s manifest file. The problem is that all content of /var, including settings.confis being overshadowed by the mounted empty /var folder. A solution is needed that allows a mounted folder with pre-defined content.

Solution

To address the issue of files within a folder being hidden by a mounted volume, you can follow this three-step approach:

  • Define the Mount: In the mounts section of the URCap manifest, specify the folder that needs to contain writable files.

  • Backup During Build: In the Dockerfile, create a backup of the original folder by copying it to a different location that won’t be affected by the mount.

  • Restore at Runtime: In the Docker entrypoint script, restore the original files from the backup to the mounted folder. Since the entrypoint runs every time the container starts (e.g., on robot reboot or URCap update), it’s important to avoid restoring the files repeatedly. To handle this, a marker file can be used to ensure the restoration only occurs once during the container’s lifetime.

Example

The following example is based on the simple-docker URCap sample. In this example we want to add a custom file myconfiguration.txt to the /var system folder. The file should contain predefined data and also be writable at runtime. The 3-step solution is illustrated below and described in the following: Adding a writable custom file to the simple-docker URCap

Step 1: Add the containing folder as a persistent mount

The following is added to the simple-docker URCap manifest. This will overshadow the /var folder inside the container.

artifacts:
  containers:
    - ...
      ingress:
        - ...
      mounts:
        - ...
        - mount: persistent:/var
          access: rw

Step 2: Add the file and backup the parent folder

The following is added to the existing Dockerfile for simple-docker.

...
  # The commands.sh will be run in the ENTRYPOINT and thereby be executed at container start.
  COPY commands.sh ./
  RUN chmod a+x ./commands.sh

  # Copy configuration file with the initial values that must be ready at startup
  COPY myconfiguration.txt ./var

  # Backup everything in /var including my configurations file
  RUN mkdir /backup_var
  RUN cp -r /var/* /backup_var

  # The commands.sh file executes 2 commands
  # 1) copy /backup_var to /var
  # 2) Run the Python Flask server
  ENTRYPOINT ["./commands.sh"]

The entrypoint have been changed. Now we would like to run two commands, first restore the folder and then start the python flask server as before. This is easily done by putting both commands in an executable sh file.

First the commands.sh file is added and made executable, then the myconfiguration.txt file is added to the /var folder and then the entire folder is backed up to a new /backup_var folder. Finally, commands.sh is set to be executed at container startup. At container startup, any folder in the container that is not mounted via the URCap manifest will be read-only.

Step 3: Restore the files from the backup.

The shell script commands.sh is executed from the Docker entrypoint. The script will restore the content of /backup_var to the now mounted and writable /var folder. Finally the script will start the python flask server as it is done in the original simple-docker URCap. This is the commands.sh file

#!/bin/sh

# This file is run in the Docker entrypoint, meaning it will run at every container startup.
# We want to restore the overshadowed "/var" folder only once in the container lifetime.
# To keep track of the folder being restored we use a marker file
MARKER_FILE_NAME=".initialized"
if [ ! -f /var/"$MARKER_FILE_NAME" ]; then
    cp -r /backup_var/* /var
    touch /var/"$MARKER_FILE_NAME"
fi
# Run the REST server
flask run --host 0.0.0.0 --port 5000

The .ìnitialized marker file is used to make sure that the restore operation is only done the very first time the container is started. This prevents the folder being reset during URCap upgrades or robot reboots.