Skip to content

Add SQL parsing from env var#89

Merged
tomncooper merged 5 commits intostreamshub:mainfrom
piotrpdev:feat/sql-from-configmap
Jul 17, 2025
Merged

Add SQL parsing from env var#89
tomncooper merged 5 commits intostreamshub:mainfrom
piotrpdev:feat/sql-from-configmap

Conversation

@piotrpdev
Copy link
Member

@piotrpdev piotrpdev commented Jul 16, 2025

Description

Resolves #84
Testing: streamshub/streams-e2e#151

Adds ability to parse SQL from environment variable (key is hard-coded) e.g. like this:

# recommendation-app.sql
CREATE TABLE ClickStreamTable
# ... more SQL ...
kubectl create configmap -n flink recommendation-app-sql \
  --from-file=SQL_STATEMENTS=recommendation-app.sql
# flink-deployment.yaml
# Only relevant parts included
apiVersion: flink.apache.org/v1beta1
kind: FlinkDeployment
spec:
  image: quay.io/streamshub/flink-sql-runner:0.2.0
  podTemplate:
    kind: Pod
    spec:
      containers:
        - name: flink-main-container
          envFrom:
            - configMapRef:
                name: recommendation-app-sql
  job:
    jarURI: local:///opt/streamshub/flink-sql-runner.jar
    parallelism: 1
    upgradeMode: stateless

Type of Change

  • New feature (non-breaking change which adds functionality)

Testing

Packit test job with smoke subset of tests is triggered by default.
If you want to run full flink test profile please type a following comment

/packit test --labels flink-all

@piotrpdev piotrpdev requested a review from tomncooper July 16, 2025 16:15
@piotrpdev
Copy link
Member Author

piotrpdev commented Jul 16, 2025

@tomncooper The code is throwing a generic exception like before, but SonarQube is complaining. Should I change it to an IllegalArgumentException or just silence the warning somehow?

@kornys
Copy link
Member

kornys commented Jul 17, 2025

IllegalArgumentException would be better

Copy link
Collaborator

@tomncooper tomncooper left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm glad this didn't turn out to be too complicated.

Can you also update the docs to describe this as an option and how users can use it.

We will also need to add coverage for this to the streams-e2e tests.

@sonarqubecloud
Copy link

@tomncooper tomncooper merged commit 000ace1 into streamshub:main Jul 17, 2025
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Allow external SQL scripts

3 participants