Skip to content

Add SQL parsing from env var #89

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Jul 17, 2025

Conversation

piotrpdev
Copy link
Member

@piotrpdev piotrpdev commented Jul 16, 2025

Description

Resolves #84
Testing: streamshub/streams-e2e#151

Adds ability to parse SQL from environment variable (key is hard-coded) e.g. like this:

# recommendation-app.sql
CREATE TABLE ClickStreamTable
# ... more SQL ...
kubectl create configmap -n flink recommendation-app-sql \
  --from-file=SQL_STATEMENTS=recommendation-app.sql
# flink-deployment.yaml
# Only relevant parts included
apiVersion: flink.apache.org/v1beta1
kind: FlinkDeployment
spec:
  image: quay.io/streamshub/flink-sql-runner:0.2.0
  podTemplate:
    kind: Pod
    spec:
      containers:
        - name: flink-main-container
          envFrom:
            - configMapRef:
                name: recommendation-app-sql
  job:
    jarURI: local:///opt/streamshub/flink-sql-runner.jar
    parallelism: 1
    upgradeMode: stateless

Type of Change

  • New feature (non-breaking change which adds functionality)

Testing

Packit test job with smoke subset of tests is triggered by default.
If you want to run full flink test profile please type a following comment

/packit test --labels flink-all

@piotrpdev piotrpdev requested a review from tomncooper July 16, 2025 16:15
@piotrpdev
Copy link
Member Author

piotrpdev commented Jul 16, 2025

@tomncooper The code is throwing a generic exception like before, but SonarQube is complaining. Should I change it to an IllegalArgumentException or just silence the warning somehow?

@kornys
Copy link
Member

kornys commented Jul 17, 2025

IllegalArgumentException would be better

Copy link
Collaborator

@tomncooper tomncooper left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm glad this didn't turn out to be too complicated.

Can you also update the docs to describe this as an option and how users can use it.

We will also need to add coverage for this to the streams-e2e tests.

Copy link

@tomncooper tomncooper merged commit 000ace1 into streamshub:main Jul 17, 2025
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Allow external SQL scripts
3 participants