Skip to content

Conversation

Atharva1723
Copy link

@Atharva1723 Atharva1723 commented Oct 13, 2024

Tracking issue

Closes flyteorg/flyte#5534

Why are the changes needed?

In Flyte, when working with map tasks, outputs often risk being overwritten if they share the same static name. For example, if multiple nodes generate output files with the same name, only the last file would remain, as previous ones would be overwritten. By adding the array node index, it becomes possible to uniquely name these outputs, avoiding conflicts.

What changes were proposed in this pull request?

The key goal of this PR is to include the index of an array node in map_tasks within the current_context() returned value. This feature is especially useful when multiple nodes within a map task could potentially overwrite outputs

Summary by Bito

This pull request introduces a new feature that adds the index of an array node to the current context in Flyte, preventing output file overwrites in map tasks. Unit tests have been added to ensure the functionality of this enhancement.

Copy link

welcome bot commented Oct 13, 2024

Thank you for opening this pull request! 🙌

These tips will help get your PR across the finish line:

  • Most of the repos have a PR template; if not, fill it out to the best of your knowledge.
  • Sign off your commits (Reference: DCO Guide).

Copy link
Member

@pingsutw pingsutw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link

codecov bot commented Oct 14, 2024

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 50.59%. Comparing base (ff4c79c) to head (6f06ad8).

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #2812      +/-   ##
==========================================
+ Coverage   48.40%   50.59%   +2.18%     
==========================================
  Files         228      216      -12     
  Lines       23329    22677     -652     
  Branches     2970     2970              
==========================================
+ Hits        11292    11473     +181     
+ Misses      11459    10672     -787     
+ Partials      578      532      -46     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Copy link
Member

@pingsutw pingsutw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you run make fmt locally to fix the lint error?

@Atharva1723
Copy link
Author

Will write an unit test and will run formatter too

@Atharva1723 Atharva1723 requested a review from pingsutw October 19, 2024 19:15
@Atharva1723
Copy link
Author

i have run make fmt

@flyte-bot
Copy link
Contributor

Bito Automatic Review Skipped - Large PR

Bito didn't auto-review this change because the pull request exceeded the line limit. No action is needed if you didn't intend for the agent to review it. Otherwise, to manually trigger a review, type /review in a comment and save.

@Atharva1723 Atharva1723 force-pushed the add-array-node-index branch from d0f23d9 to 6f06ad8 Compare October 5, 2025 08:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add Array Node Index To current_context [Core feature]

3 participants