Grafana 8.14.2 init-chown-data restart failure

[Kimi450]

Grafana 8.14.2 init-chown-data fails upon pod restart

After installing the chart, if grafana pod restarts it fails to come up. I suspect it is because the first start chowns the dir and then the subsequent restarts try to chown a dir that they do not own. root tries to change something owned by something else but since only "CHOWN" is provided to the container as the security context and all else is explicitly dropped, it fails to override.

How to reproduce

1. Install the chart

   helm upgrade -i my-release grafana/grafana --set initChownData.enabled=true --set persistence.enabled=true --wait
   

2. Once the pod is running, delete it
3. Observe that it cannot restart because the initcontainer fails with

   19:52:08 dell-e7440 [--0] ~  $ k logs -c init-chown-data my-release-grafana-846948d9f4-vgxm7
   chown: /var/lib/grafana/csv: Permission denied
   chown: /var/lib/grafana/png: Permission denied
   chown: /var/lib/grafana/pdf: Permission denied
   19:52:16 dell-e7440 [--0] ~  $
   

Workaround

chmod 777 the contents of the PV[C] (I tried this with a hostpath PV mount)

or

Delete the drop: {ALL} section from the init container's security context section.

Related issues/links

• Grafana chart 8.14.1 fails to deploy because of missing permissions #3690
• [grafana] Explicitly drop all unused capabilities for init-chown-data and set readonlyRootFilesystem #3684
• https://github.com/grafana/helm-charts/blob/main/charts/grafana/values.yaml#L486

[Kimi450]

@jcpunk I believe youll have the most context here.

[jcpunk]

I think #3698 resolves this. Can you test?

[Kimi450]

I think #3698 resolves this. Can you test?

Yes this fixes the problem observed in this issue at least. Thanks!

[jcpunk]

Can you note your test results in the PR?

[Kimi450]

Can you note your test results in the PR?

Done