Support Loading Routing Rules from Artifactory

[Peiyingy]

Background

Today, when the rulesType is set to FILE, Trino-Gateway loads routing rules from a local configuration file, and any update requires a service redeploy. While the gateway already supports dynamically refreshing routing rules in memory, the source of truth is still a local file, which limits operational flexibility and slows down configuration rollout.

At LinkedIn, we have been extending Trino-Gateway to support loading routing rules from Artifactory in addition local files. This allows us to:

• Update routing rules without redeploying the gateway
• Use Artifactory as a centralized, versioned, and auditable configuration store
• Reuse existing dynamic-rules refresh mechanisms already present in the gateway

We believe this approach is generally useful and could benefit the broader OSS community as well.

Proposed Approach

1. Configuration source for routing rules for file based routing rules

Introduce an abstraction that allows routing rules to be loaded from different sources, e.g.:

• Local file (current behavior)
• Artifactory (new)

2. Artifactory-backed implementation

Implement Artifactory configuration loader so that:

• The latest version of the routing rules can be fetched from Artifactory
• The existing periodic refresh logic in Trino-Gateway can reload rules automatically

At LinkedIn, this pattern is already in use in production for dynamic configuration, and we are in the process of applying it to Trino-Gateway routing rules as well. We’d like to explore upstreaming this capability so it can be broadly available and maintained in the open-source project.

Happy to contribute design, implementation, and follow-up PRs if the community agrees this direction makes sense.

[mosabua]

We have a plan to move the routing rules storage into the database. The initial loading could be done from a remote config system .. however making it specific to Artifactory makes not sense. We could just make it a generic load from a URL .. maybe with authentication and then that URL could point to Artifactory or whatever other storage really. And yes.. the remote file storage for the config could either be read only .. or potentially be a read and write system.

[Peiyingy]

We have a plan to move the routing rules storage into the database. The initial loading could be done from a remote config system .. however making it specific to Artifactory makes not sense. We could just make it a generic load from a URL .. maybe with authentication and then that URL could point to Artifactory or whatever other storage really. And yes.. the remote file storage for the config could either be read only .. or potentially be a read and write system.

Thanks for the reply. As for the "move the routing rules storage into the database" part, wondering if you're referring to this implementation? #772

[mosabua]

That is just an old PR that was never merged and is stored for reference. @EdenKik mentioned in a dev sync last year that she will work on a replacement PR.