-
Notifications
You must be signed in to change notification settings - Fork 166
Description
Disclaimer: I am still learning about Apache Hamilton and using it in local mode only for now
Is your feature request related to a problem? Please describe.
I'd like the following API
@rename("My Feature")
def my_feature(...) -> ...:
return this_featureSo that
- DAG node is named "My Feature" in the DAG,
- and I can further access it with source("My Feature") in the codebase if needed.
There is a discussion about this in the community Slack
Describe the solution you'd like
- is just display and could be solved by intervening at this level only,
- is partially solved by
injectandsourcefor caller code, but for declaration the actual way to achieve this is, as suggested by @skrawcz is to
do it via
extract_fieldsI think --> so return a dictionary and then pull that key out..."
Describe alternatives you've considered
@renameshould be easy
Problem is spaces in node name. I'd recommend keeping them as identifiers cause it makes things easier. Then add@tag(name="My Exciting Name!")and do the accounting yourself.
Additional context
There has been a few issues raised about edge cases and potential breaks depending how this feature is implemented, see Slack discussion
I don't have specific example in mind, but I believe having nodes be valid identifiers is a strong assumptions made in several places in the code.For example, when using @pipe_output we namespace things {entrypoint_node}.with_{step_node}. Similar things happen in @subdagIt creates a lot of edge cases if you can do
@rename("; DROP TABLE bar") # SQL injection def foo(...) -> str: ...@rename("<b><i>bingo<</b>") # valid or invalid HTML def baz(...) -> int: ...also:
break graphviz notation for visualization
Node names are used as keys in dictionaries, HamiltonGraph
node names are passed to SQL engines in the caching backends and the Hamilton UI (would need further sanitization)
could even break exceptions and logging message if it includes {