Increase the ClickHouse Persistent Volume Size

[yo3ibw]

I followed the official tutorial and I encounter a problem when resizing the clickhouse persistent volume.

Other details,
Signoz runs on an AWS EKS cluster, version 1.24.

Error:

signoz$ helm -n logs upgrade my-release signoz/signoz --set clickhouse.persistence.size=55Gi coalesce.go:175: warning: skipped value for zookeeper.initContainers: Not a table. Error: UPGRADE FAILED: cannot patch "my-release-zookeeper" with kind StatefulSet: StatefulSet.apps "my-release-zookeeper" is invalid: spec: Forbidden: updates to statefulset spec for fields other than 'replicas', 'template', 'updateStrategy', 'persistentVolumeClaimRetentionPolicy' and 'minReadySeconds' are forbidden && cannot patch "my-release-signoz-alertmanager" with kind StatefulSet: StatefulSet.apps "my-release-signoz-alertmanager" is invalid: spec: Forbidden: updates to statefulset spec for fields other than 'replicas', 'template', 'updateStrategy', 'persistentVolumeClaimRetentionPolicy' and 'minReadySeconds' are forbidden && cannot patch "my-release-signoz-query-service" with kind StatefulSet: StatefulSet.apps "my-release-signoz-query-service" is invalid: spec: Forbidden: updates to statefulset spec for fields other than 'replicas', 'template', 'updateStrategy', 'persistentVolumeClaimRetentionPolicy' and 'minReadySeconds' are forbidden

PVC are gp2-resizable

signoz$ kubectl get pvc -n logs NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE data-my-release-zookeeper-0 Bound pvc-2fce7e91-a2a6-47c4-bad5-389793719c7b 8Gi RWO gp2-resizable 73m data-volumeclaim-template-chi-my-release-clickhouse-cluster-0-0-0 Bound pvc-022d2fea-fba5-4795-a89f-81f8558dd4ad 50Gi RWO gp2-resizable 73m signoz-db-my-release-signoz-query-service-0 Bound pvc-a3c3c2a8-d02a-4fc2-b319-bf7ab53b685e 1Gi RWO gp2-resizable 73m storage-my-release-signoz-alertmanager-0 Bound pvc-69441e98-ce38-4882-acd0-28779b750441 1Gi RWO gp2-resizable 73m

I appreciate any suggestion. Thanks!

[prashant-shahi]

helm -n logs upgrade my-release signoz/signoz --set clickhouse.persistence.size=55Gi

@yo3ibw can you include -f override-values.yaml as well in the upgrade command?

Also, for complete guide on increasing persistence of clickhouse can be found here: https://signoz.io/docs/operate/clickhouse/increase-clickhouse-pv/

For other PVCs, the instructions should be similar to that of clickhouse.

[yo3ibw]

Hey,

the content of override-values.yml is the one from the installation tutorial, to be more extacly.

`
global:
storageClass: gp2-resizable

clickhouse:
cloud: aws
installCustomStorageClass: true
`

Upgrade command is the one you mentioned and also the one that can be seen in the error section on my first post.

[yo3ibw]

Small update on this. Despite the error I receive aparently the resize is a success. It takes some time to be apply, like minutes, but in the end all goes well.