-
Notifications
You must be signed in to change notification settings - Fork 106
Open
Description
Hi team, we are having some issue and may need your suggestion, and if there is no workaround I want to suggest some changes.
For context:
- We have about ~200 topics which needs to ingest in Snowflake, they are all domain events so we have some existing system to structure the
topic name by domain, egcom.abc.$domain.$subdomain.$version.$something
which guarantee there is no topic name collission. - For every new topic we use Terraform to create Snowflake table.
- Given the topic name has
.
in it, when we do not definetopic2table
config, Snowflake streaming connector will create table withhash
suffix to avoid naming collision, this generated table name apparently is not the same as what we created through TF. - Problem, to define the connector we cannot avoid defining
topics
andtopic2table
now since the constraint, and we have to define a lot.. also it also constraint us to usetopics.prefix
which automatically detect new topic.
I want to learn if there is existing use case like this and how people handle this. If possible, can we have a way to customize tablename
generation function through an interface since all we need is replace . with underscore and uppercase all characters
ryancrawcour
Metadata
Metadata
Assignees
Labels
No labels