Skip to content

A rename feature #1413

@paulduf

Description

@paulduf

Disclaimer: I am still learning about Apache Hamilton and using it in local mode only for now

Is your feature request related to a problem? Please describe.

I'd like the following API

@rename("My Feature")
def my_feature(...) -> ...:
    return this_feature

So that

  1. DAG node is named "My Feature" in the DAG,
  2. and I can further access it with source("My Feature") in the codebase if needed.

There is a discussion about this in the community Slack

Describe the solution you'd like

  1. is just display and could be solved by intervening at this level only,
  2. is partially solved by inject and source for caller code, but for declaration the actual way to achieve this is, as suggested by @skrawcz is to

do it via extract_fields I think --> so return a dictionary and then pull that key out..."

Describe alternatives you've considered

@elijahbenizzy

@rename should be easy
Problem is spaces in node name. I'd recommend keeping them as identifiers cause it makes things easier. Then add @tag(name="My Exciting Name!") and do the accounting yourself.

Additional context

There has been a few issues raised about edge cases and potential breaks depending how this feature is implemented, see Slack discussion

I don't have specific example in mind, but I believe having nodes be valid identifiers is a strong assumptions made in several places in the code.For example, when using @pipe_output we namespace things {entrypoint_node}.with_{step_node}. Similar things happen in @subdagIt creates a lot of edge cases if you can do

@rename("; DROP TABLE bar")  # SQL injection
def foo(...) -> str: ...
@rename("<b><i>bingo<</b>")  # valid or invalid HTML
def baz(...) -> int: ...

also:

break graphviz notation for visualization
Node names are used as keys in dictionaries, HamiltonGraph
node names are passed to SQL engines in the caching backends and the Hamilton UI (would need further sanitization)
could even break exceptions and logging message if it includes {

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions