Lots of Helm release-related Secrets being created despite low release history setting in code

[bcbrockway]

I can see that the Helm release history is set to 2 here but we are still seeing an enormous number of Secrets being created in our Milvus namespaces which is triggering our quota alerts:

Any idea what might be going on here?

[haorenfsa]

Thank you for the feedback @bcbrockway. Could you provide your milvus CR & the logs of milvus-operator during that period?

[bcbrockway]

Hi @haorenfsa, as you can see from the logs, we had an immutable field issue with some StatefulSets at the time which we eventually resolved by orphan-deleting them and letting the milvus-operator reapply the releases, but I would still expect it to only keep 2 sh.helm.release.v1 secrets per StatefulSet.

Explore-logs-2024-07-03 08_54_03.txt

[haorenfsa]

@bcbrockway Looks like the helm pkg overwrite the MaxHistory with the value in Upgrade.MaxHistory struct when doing upgrade

I check the helm command's code, HelmSetting.MaxHistory is only useful when using helm command. When using it as a pkg, we should directly set Upgrade.HaxHistory. Otherwise it's considered as 0 (which means unlimited)