Docker Volume Management

The following provides an understanding of various ways to handle storage in URCapX container, namely Persistent Volumes, tmpfs mounts, and Removable Storage. We’ll also explore the potential risks, including data loss, associated with volume mounts. For the three types of mounts an example will follow of the changes needed for the manifest file which can also be found in the sample in the SDK.

Types of Volume Mounts

Different types of volume mounts in Docker serve distinct use cases. Each comes with its own advantages and disadvantages.

1. Persistent Volumes

Persistent Volumes retain data beyond the lifecycle of individual containers. Therefore, even when a container is stopped the data in the persistent volume remains intact. This preservation of data applies even during URCapX updates/upgrades. These volumes are particularly useful when you need to preserve data across container restarts, which is common with long-term data like database files and application logs. Typically, this type of storage resides on the host filesystem.

The following is an example from the SDK sample where a container named data-storage. This data-storage is then given a folder due to the persistent-keyword on the left side of the colon, that will persist after reboots of the robot. Inside the container it is then possible to put files/logs/configurations in /data/persistent which will stay there.

metadata:
    vendorID: universal-robots
    urcapID: persistent-storage-demo
    version: 1.0.0
    #...
artifacts:
    containers:
    - id: data-storage
      image: data-storage:latest
      mounts:
        # example: persistent:/etc/urcap/config
        # example: persistent:/var/log/urcap
        - mount: persistent:/data/persistent
          access: rw

2. tmpfs mounts

In contrast to Persistent Volumes, tmpfs mounts are temporary filesystems residing in memory or the host’s swap space. All content in tmpfs is cleared when the container stops, leading to high-speed but non-persistent storage. tmpfs mounts are ideal for storing sensitive information that should not be written to disk or temporary data that doesn’t need to persist. However, note that data in tmpfs mounts is lost during URCapX updates/upgrades.

In this example the container will receive a temporary folder which is located at /data/temporary.

metadata:
    vendorID: universal-robots
    urcapID: tmp-storage-demo
    version: 1.0.0
    #...
artifacts:
    containers:
    - id: data-storage
      image: data-storage:latest
      mounts:
        # example: tmpfs:/usr/local/urcap/tmp
        # example: tmpfs:/tmp
        - mount: tmpfs:/data/temporary
          access: rw

3. Removable Storage

Removable storage generally refers to physical storage devices like USB drives, external hard drives, or SD cards. Direct use of removable storage with Docker is less common because Docker performs best with consistently available storage. However, you can technically use removable storage with Docker if the storage is present and mounted on the host system. This mounted directory can then serve as a Docker volume.

In the following example if a removable storage is detected a folder inside the container at /data/removable is created.

metadata:
    vendorID: universal-robots
    urcapID: removable-storage-demo
    version: 1.0.0
    #...
artifacts:
    containers:
    - id: data-storage
      image: data-storage:latest
      mounts:
        # example: RemovableStorage:/etc/container
        # example: RemovableStorage:/mnt
        - mount: RemovableStorage:/data/removable
          access: rw

Risk of Data Loss with Volume Mounts

Regardless of the volume mount type (Persistent, tmpfs, or Removable Storage), data loss can occur when a volume is mounted onto an existing directory in a Docker container.

When a volume mount “covers” a directory with pre-existing data, the data becomes obscured and inaccessible for the duration of the mount. It’s crucial to note that this does not result in permanent data deletion or loss, but from the perspective of applications running inside the container, it seems as though the data has been lost.

To avoid obscuring crucial data, ensure that the target directory for volume mounts is specifically designed for this purpose and does not contain data required while the volume is mounted.