Cannot mount an existing share via subDir parameter

[ppenzo]

What happened:
The subDir parameter is not honoured in PVC and, when used in a storage class causes all the volumes to be attached to the share/directory specified in the SC definition.

What you expected to happen:
AFAIK the subDir parameter should allow the PVC to request a volume bounded to an existing share as per issue [#398]

How to reproduce it:
To reproduce just define a SC like this

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: test-csi-smb
provisioner: smb.csi.k8s.io
reclaimPolicy: Delete
parameters:
  source: "//NAS_FILER/"
  subDir: "TESTSHARE"
  csi.storage.k8s.io/provisioner-secret-name: "smbcreds"
  csi.storage.k8s.io/provisioner-secret-namespace: "aatest-storage"
  csi.storage.k8s.io/node-stage-secret-name: "smbcreds"
  csi.storage.k8s.io/node-stage-secret-namespace: "aatest-storage"
volumeBindingMode: Immediate
mountOptions:
  - dir_mode=0777
  - file_mode=0777

and a PVC like this

kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: pvc-tsmb-12
spec:
  storageClassName: test-csi-smb
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 1Mi
  csi:
    volumeAttributes:
      subDir: "anotherSHARE"

As a result the volume will be bounded to //NAS_FILER/TESTSHARE and not to //NAS_FILER/anotherSHARE.
The same occurs when the subDir parameters is not specified into the PVC.

Anything else we need to know?:
Not defining the subDir parameter into the SC causes the PV to be bounded to a subdirectory of //NAS_FILER/TESTSHARE created "on the fly" no matter if the volumeAttributes.subDir parameters is defined into the PVC.

Also defining the volumeAttributes.source into the PVC has no effect.

Environment:

• CSI Driver version: 1.5.0

[andyzhangx]

I think this driver only honors subDir: "TESTSHARE" in storage class, it won't honor volumeAttributes in PersistentVolumeClaim. btw, is this pvc config valid?

kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: pvc-tsmb-12
spec:
  storageClassName: test-csi-smb
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 1Mi
  csi:
    volumeAttributes:
      subDir: "anotherSHARE"

# k apply -f /tmp/a.yaml
error: error validating "/tmp/a.patch2": error validating data: ValidationError(PersistentVolumeClaim.spec): unknown field "csi" in io.k8s.api.core.v1.PersistentVolumeClaimSpec; if you choose to ignore these errors, turn validation off with --validate=false

[andyzhangx]

if you want to create sub dir with pvc annotation, you could try like this:

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: test-csi-smb
provisioner: smb.csi.k8s.io
reclaimPolicy: Delete
parameters:
  source: "//NAS_FILER/"
  subDir: ${pvc.annotations['team.example.com/key']}
  csi.storage.k8s.io/provisioner-secret-name: "smbcreds"
  csi.storage.k8s.io/provisioner-secret-namespace: "aatest-storage"
  csi.storage.k8s.io/node-stage-secret-name: "smbcreds"
  csi.storage.k8s.io/node-stage-secret-namespace: "aatest-storage"
volumeBindingMode: Immediate
mountOptions:
  - dir_mode=0777
  - file_mode=0777

[ppenzo]

You are right! The PVC definition I've provided is not correct.
I'm sorry, I've applied it without enforcing the schema validation.

I'll try to use the annotation as you suggested. Please let me have some time to do some tests.

[ppenzo]

Unfortunately the configuration you suggested does not work since the PVC annotations are not expanded and so the subdir value is treated literally.
The same occurs using a slitly different syntax:
${.PVC.annotations.storagepath}

[ppenzo]

I went a big further on this issue discovering that it depends on the csi-external-provisioner which is used as sidecar container in the deployment.

Currently the csi-external-provisioner does not pass extra parameters nor annotations to the CSI driver and there are a couple of issues open on this: https://github.com/kubernetes-csi/external-provisioner/issues/714 and https://github.com/kubernetes-csi/external-provisioner/issues/86

So to have this working I guess that the csi-external-provisioner has to be modified or a different controller has to be used.

[MiddleMan5]

Are any context variables expanded? pod namespace? pvc namespace?
I keep seeing mention of this expansion behavior, but it's not clear where it's applicable

[ppenzo]

Actually this feature in unusable in autoprovision mode using the "default" csi-external-provider sidecar container, see issues 86 and 714 of that project.
As suggested in those issues the only possible(?) solution to this is to write your own sidecar container!

[andyzhangx]

with PR #503 "subDir":"subDirectory-${pvc.metadata.name}" would be converted to "subDir":"subDirectory-pvc-w7sbh"

• subDir parameter supports following pv/pvc metadata transform

if subDir value contains following string, it would transforms into corresponding pv/pvc name or namespace

• ${pvc.metadata.name}
• ${pvc.metadata.namespace}
• ${pv.metadata.name}

[servo1x]

with PR #503 "subDir":"subDirectory-${pvc.metadata.name}" would be converted to "subDir":"subDirectory-pvc-w7sbh"

* `subDir` parameter supports following pv/pvc metadata transform

if subDir value contains following string, it would transforms into corresponding pv/pvc name or namespace

* `${pvc.metadata.name}`

* `${pvc.metadata.namespace}`

* `${pv.metadata.name}`

Hi @andyzhangx - is there any plans to support annotations as described in #428 (comment)?

[andyzhangx]

with PR #503 "subDir":"subDirectory-${pvc.metadata.name}" would be converted to "subDir":"subDirectory-pvc-w7sbh"

* `subDir` parameter supports following pv/pvc metadata transform

if subDir value contains following string, it would transforms into corresponding pv/pvc name or namespace

* `${pvc.metadata.name}`

* `${pvc.metadata.namespace}`

* `${pv.metadata.name}`

Hi @andyzhangx - is there any plans to support annotations as described in #428 (comment)?

@servo1x there is no clear plan yet, and it depends on kubernetes-csi/external-provisioner#86

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue or PR as fresh with /remove-lifecycle stale
• Mark this issue or PR as rotten with /lifecycle rotten
• Close this issue or PR with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[k8s-triage-robot]

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue or PR as fresh with /remove-lifecycle rotten
• Close this issue or PR with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

[k8s-triage-robot]

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Reopen this issue with /reopen
• Mark this issue as fresh with /remove-lifecycle rotten
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close not-planned

[k8s-ci-robot]

@k8s-triage-robot: Closing this issue, marking it as "Not Planned".

In response to this:

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Reopen this issue with /reopen
• Mark this issue as fresh with /remove-lifecycle rotten
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close not-planned

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.