Upper limit on exporting yaml files?

[jwilkicki]

Hi!

I've just started writing Jsonnet and using Tanka. Everything was working pretty well until I hit a puzzling bit of behavior.

I'm building up a bunch of exported manifests from Helm charts for eventual application with Argo CD. They represent a handful of common services (e.g. service meshes, monitoring, etc).

I had a set of these addons configured just fine, but when I added one more, suddenly the output of one of the helm charts got truncated; I was suddenly missing three Service Accounts from the helm output for that chart while my new component (that isn't using Helm at all) generated just fine.

When I remove the problem component, the rendering of my chart returns to normal. When I remove a couple of manifests from my new component and try again, the number of deleted ServiceAccounts in the output of the other chart shrinks but I am still short three.

The number of manifests that I've generated seems to top out at 256, which seemed kind of suggestive. I didn't find anything in the issues or documentation that mentions similar behavior. Could I be hitting a memory limit?

UPDATE: To narrow the scope, I limited my addons to just the two that have problems and the issue is still happening. So somehow, the output from one of these addons is disrupting the other on export.

[zerok]

Hi 😄 Sorry for the late reply. Are the helm charts public by any chance so that I could to reproduce this somehow?

[jwilkicki]

That's okay, I appreciate the reply anyway.

TL;DR: this was something I did to myself somehow in my jsonnet/tanka learning journey. Adding in namespaces to keep the manifests separate seemed to be a component of the fix. I'm unblocked at this time but hopefully some troubleshooting details might be useful to others.

The charts aren't all publicly available, but ultimately I found a way through. I did narrow the problem and discover that it wasn't the quantity of charts (there were some big ones like kube-prometheus-stack) but two specific objects that were somehow conflicting with each other. I followed the design recommendation of creating separate local libraries for wrapping each helm chart and then I brought them together in a single object. So to troubleshoot, I started commenting out combinations of helm charts until I narrowed down the ones causing the problem.

I'm not entirely sure why, but I ended up having to add the namespace to the export path template for the manifests while using tk export. The reason why I'm puzzled about that is the two conflicting objects was one that was created via helm.template() and one that was created whole cloth from jsonnet functions and shouldn't have had any conflicting resources when rendered out. However, it did work when the namespaces separated them. Once I put the namespace in, everything started rendering fine again. No performance issues or missing files now.

[zerok]

Awesome! Thank you for all these details ❤️