fix: Use correct service for StatefulSet

[adwk67]

Description

The StatefulSet should use the headless service and not the group listener service. If the headless service is not used, the logs cannot be retrieved from within the airflow UI:

The upper part of the screenshot shows the listener service being used, and the lower part the (correct) headless service:

Definition of Done Checklist

• Not all of these items are applicable to all PRs, the author should update this template to only leave the boxes in that are relevant
• Please make sure all these things are done and tick the boxes

Author

• Changes are OpenShift compatible
• Helm chart can be installed and deployed operator works
• Integration tests passed (for non trivial changes)

Reviewer

• Code contains useful comments
• Code contains useful logging statements
• (Integration-)Test cases added
• Documentation added or updated. Follows the style guide.
• Changelog updated
• Cargo.toml only contains references to git tags (not specific commits or branches)

Acceptance

• Feature Tracker has been updated
• Proper release label has been added
• Roadmap has been updated

[NickLarsenNZ]

Just curious, is there a more authoritative source for the service name (eg: a function, in case it is needed in multiple places)?

[NickLarsenNZ]

LGTM

[sbernauer]

As already mentioned I don't really like abusing the metrics service for this. Instead we should create a fresh headless service meant for this.
You mentioned in a daily that the Airflow 3 PR will do this properly, did I recall that correctly? Should we wait for that instead?

[adwk67]

Closing this without merging as the bugfix was included in this PR (so that the logs are retrievable). We can consider how to tackle this in the medium-term in the context of https://github.com/stackabletech/decisions/issues/54.