using smb csi driver and unable to list or touch netapp cifs share from within POD

[antak6]

Hi Team,

We have a windows user with full control and that user can access NetApp CIFS share via windows explorer. We are using smb csi driver v1.17.0. We have mounted the NetApp CIFS svm and can reach the mount point but get permission denied to ls or touch inside the netapp cifs mount. Can anyone confirm if SMB CSI driver works for NetApp CIFS shares? If we try to do exact same thing using same driver for a normal windows file share, it does work. We have checked all NetApp and user account details and nothing more to be done on those ends.

[andyzhangx]

@antak6 does ls or touch work if you mount smb share on the windows node directly?

$User = "Domain\USERNAME"
$PWord = ConvertTo-SecureString -String "PASSWORD" -AsPlainText -Force
$Credential = New-Object –TypeName System.Management.Automation.PSCredential –ArgumentList $User, $Pword
New-SmbGlobalMapping -LocalPath x: -RemotePath \\smb-server\fileshare -Credential $Credential
Get-SmbGlobalMapping
cd x:
dir

[antak6]

Yes, it does work with same credentials from windows UNC ... So I guess this is not a SMB csi issue .. probably it is something to do with how netapp is translating or not translating/mapping LINUX UIDs to windows user .. I will close this issue since it is not related to SMB CSI anymore.

@antak6 does ls or touch work if you mount smb share on the windows node directly?

$User = "Domain\USERNAME"
$PWord = ConvertTo-SecureString -String "PASSWORD" -AsPlainText -Force
$Credential = New-Object –TypeName System.Management.Automation.PSCredential –ArgumentList $User, $Pword
New-SmbGlobalMapping -LocalPath x: -RemotePath \\smb-server\fileshare -Credential $Credential
Get-SmbGlobalMapping
cd x:
dir

[antak6]

Re-opening since we did a lot of troubleshooting around this. We are able to manually mount the netapp cifs share inside the POD and also on the kubernetes node. However, when we define it in yaml files for csi driver to do the actual mounting, it doesn't work and gives permission denied.

[antak6]

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: smb-sc-test
provisioner: smb.csi.k8s.io
parameters:
  csi.storage.k8s.io/provisioner-secret-name: smb-credentials
  csi.storage.k8s.io/provisioner-secret-namespace: my-cifs-namespace
  csi.storage.k8s.io/node-stage-secret-name: smb-credentials
  csi.storage.k8s.io/node-stage-secret-namespace: my-cifs-namespace
  source: //my_netapp_share_path
volumeBindingMode: Immediate
mountOptions:
  - dir_mode=0777
  - file_mode=0777
  - uid=1000
  - gid=1000
  - nobrl
  - version=3.0
  - noperm
---
apiVersion: v1
kind: Secret
metadata:
  name: smb-credentials-test
  namespace: my-cifs-namespace
type: Opaque
stringData:
  password: My_PWD
  username: My_User
---
#apiVersion: v1
#kind: PersistentVolumeClaim
#metadata:
#  name: smb-pvc-test
#  namespace: my-cifs-namespace
#spec:
#  storageClassName: smb-sc-test
#  accessModes:
#    - ReadWriteOnce
#  resources:
#    requests:
#      storage: 5Gi
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: smb-pvc-test
  namespace: my-cifs-namespace
spec:
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 10Gi
  volumeName: pv-smb-test
  storageClassName: smb-sc-test
---
apiVersion: v1
kind: Pod
metadata:
  name: smb-test-pod-test
  namespace: my-cifs-namespace
spec:
  securityContext:
    runAsUser: 1000
    runAsGroup: 1000
    fsGroup: 1000
  containers:
  - name: test-container
    image: My_Image_repo_link
    command: ["/bin/sleep", "infinity"]
    resources:
      limits:
        cpu: "500m"
        memory: "512Mi"
      requests:
        cpu: "200m"
        memory: "256Mi"
    volumeMounts:
    - name: smb-volume
      mountPath: /tmp/test
      readOnly: false
  volumes:
  - name: smb-volume
    persistentVolumeClaim:
      claimName: smb-pvc-test
---
apiVersion: v1
kind: PersistentVolume
metadata:
  annotations:
    pv.kubernetes.io/provisioned-by: smb.csi.k8s.io
  name: pv-smb-test
spec:
  capacity:
    storage: 10Gi
  accessModes:
    - ReadWriteMany
  persistentVolumeReclaimPolicy: Retain
  storageClassName: smb-sc-test
  mountOptions:
    - dir_mode=0777
    - file_mode=0777
    - uid=1000
    - gid=1000
  csi:
    driver: smb.csi.k8s.io
    volumeHandle: my_netapp_share_path 
    readOnly: false
    volumeAttributes:
      source: "//my_netapp_share_path/"
    nodeStageSecretRef:
      name: smb-credentials-test
      namespace: my-cifs-namespace

[andyzhangx]

@antak6 could you create secret using following command?

kubectl create secret generic smb-credentials-test -n my-cifs-namespace --from-literal username=USERNAME --from-literal password="PASSWORD"

[antak6]

Yes, we have created the secret using the same command have also tried with yaml defination.

[andyzhangx]

Yes, we have created the secret using the same command have also tried with yaml defination.

@antak6 what is the error msg in csi driver logs if mount failed?

[antak6]

We cannot list or touch anything inside the mount. ls -la or touch shows permissions denied and we have tried with the same user or even root user and same output.

[andyzhangx]

@antak6 the driver would use domain Azure in username, .e.g. Azure\username , is this related? what is your username to mount on Windows?

csi-driver-smb/pkg/smb/nodeserver.go

Lines 215 to 223 in 887341d

	if domain == "" {
	domain = defaultDomainName
	}
	if requireUsernamePwdOption {
	if !strings.Contains(username, "\\") {
	username = fmt.Sprintf("%s\\%s", domain, username)
	}
	mountOptions = []string{username}
	sensitiveMountOptions = []string{password}

[antak6]

We have tried with both domian\username and just username , same error

[andyzhangx]

We have tried with both domian\username and just username , same error

this driver has windows e2e test running against smb file share and it works, so there would be some config diff between the example Windows smb file share and your netapp share. at least this example work on windows: https://github.com/kubernetes-csi/csi-driver-smb/tree/master/deploy/example/windows

[LeonardPoiana]

the problem is that if i use priviledged: true in the security context everythibng works fine, but if i remove it does not work anymore. Our parameters :

mountOptions:

• dir_mode=0777
• file_mode=0777
• sec=ntlmssp
• noserverino
• uid=1000
• gid=1000
• forceuid
• forcegid

Pod:
securityContext:
runAsUser: 1000
runAsGroup: 1000
fsGroup: 1000
fsGroupChangePolicy: Always

[Heap0017]

We are experiencing the exact same issue. the open call is denied:

/ $ strace ls /mnt/data
execve("/bin/ls", ["ls", "/mnt/data"], 0x7ffe7b1466d8 /* 19 vars */) = 0
arch_prctl(ARCH_SET_FS, 0x7fab7f33db28) = 0
set_tid_address(0x7fab7f33df90) = 58
brk(NULL) = 0x55a1d377e000
brk(0x55a1d3780000) = 0x55a1d3780000
mmap(0x55a1d377e000, 4096, PROT_NONE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x55a1d377e000
mprotect(0x7fab7f33a000, 4096, PROT_READ) = 0
mprotect(0x55a1d2ebf000, 16384, PROT_READ) = 0
getuid() = 1001
getgid() = 1001
rt_sigprocmask(SIG_BLOCK, ~[RTMIN RT_1 RT_2], [], 8) = 0
rt_sigprocmask(SIG_BLOCK, ~[], NULL, 8) = 0
setgid(1001) = 0
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
rt_sigprocmask(SIG_BLOCK, ~[RTMIN RT_1 RT_2], [], 8) = 0
rt_sigprocmask(SIG_BLOCK, ~[], NULL, 8) = 0
setuid(1001) = 0
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
ioctl(0, TIOCGWINSZ, {ws_row=76, ws_col=165, ws_xpixel=0, ws_ypixel=0}) = 0
ioctl(1, TIOCGWINSZ, {ws_row=76, ws_col=165, ws_xpixel=0, ws_ypixel=0}) = 0
ioctl(1, TIOCGWINSZ, {ws_row=76, ws_col=165, ws_xpixel=0, ws_ypixel=0}) = 0
stat("/mnt/data", {st_mode=S_IFDIR|0777, st_size=0, ...}) = 0
open("/mnt/data", O_RDONLY|O_LARGEFILE|O_CLOEXEC|O_DIRECTORY) = -1 EACCES (Permission denied)