Skip to content

Allow containers to have different paths to the external ones #270

@drtconway

Description

@drtconway

Hi Bpipies,

At the moment, when using containered execution (e.g. singularity), it is possible to bind extra paths
to make them visible inside the container, using a filesystems declaration block. For example:

config {
    filesystems {
        reference {
            type='bind'
            bind='/reference'
        }
    }
}

Container systems (i.e. both Docker and Singularity) allow you to bind paths with different locations inside the container to the external host location. For example:

docker run -it -v /some/complicated/path/to/my/reference/data:/reference ...

The command line syntax for this is currently the same for both docker and singularity, namely:

src[:dest[:flags]]

So for now it might work to write my bpipe config:

config {
    filesystems {
        reference {
            type='bind'
            bind='/some/complicated/path/to/my/reference/data:/reference:ro'
        }
    }
}

but this seems undesirable for three reasons.

  1. If the syntax changes in either tool, my configuration will break.
  2. The bind string is effectively an opaque object to bpipe so it can't check that it makes sense.
  3. If editing the binding to localise the pipeline, it is easy to mess up, and change the mount in unintended ways.

I propose the following more nuanced specification; the current form could be retained but deprecated for compatability.

config {
    filesystems {
        reference {
            type='bind'
            source='/some/complicated/path/to/my/reference/data'
            destination='/reference'
            read_only=true
        }
    }
}

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions