Skip to content
Merged
Show file tree
Hide file tree
Changes from 18 commits
Commits
Show all changes
85 commits
Select commit Hold shift + click to select a range
01182b4
[Flyte Decl] Streaming Decks
Future-Outlier Oct 1, 2024
9616fc3
print
Future-Outlier Oct 1, 2024
98ae7c7
sleep more
Future-Outlier Oct 1, 2024
4e92bb0
add dummy deck
Future-Outlier Oct 2, 2024
4df18b5
nit
Future-Outlier Oct 2, 2024
ebb4d4e
dummy deck
Future-Outlier Oct 2, 2024
99522d9
update
Future-Outlier Oct 2, 2024
c19d67d
nit
Future-Outlier Oct 2, 2024
67cd829
test
Future-Outlier Oct 2, 2024
06da3df
return html
Future-Outlier Oct 2, 2024
6d99d69
Change Deck
Future-Outlier Oct 2, 2024
b805cd7
fix
Future-Outlier Oct 2, 2024
4c97758
fix recursion error
Future-Outlier Oct 2, 2024
7b3574a
remove redundant code
Future-Outlier Oct 2, 2024
9b60564
add dummy deck to deck init
Future-Outlier Oct 2, 2024
18c994f
Better Dummy Deck Logic
Future-Outlier Oct 2, 2024
39f39d1
Merge branch 'master' into flytekit-streaming-deck
Future-Outlier Oct 7, 2024
aabcbbb
Deck Publish
Future-Outlier Oct 7, 2024
9ca43f3
Merge branch 'master' into flytekit-streaming-deck
Future-Outlier Nov 25, 2024
ed56352
litn
Future-Outlier Nov 27, 2024
b559fc9
remove dummy deck
Future-Outlier Nov 27, 2024
fc5578f
nit
Future-Outlier Nov 27, 2024
7139468
use auto refresh tab, 5 seconds as interval
Future-Outlier Dec 2, 2024
e0aee9e
revert
Future-Outlier Dec 2, 2024
3727588
test setDynamicTabs
Future-Outlier Dec 2, 2024
ce3ee15
change interval time
Future-Outlier Dec 2, 2024
d066231
test
Future-Outlier Dec 2, 2024
f9387ce
revert
Future-Outlier Dec 2, 2024
f14c3fa
test
Future-Outlier Dec 2, 2024
c33a909
nit
Future-Outlier Dec 2, 2024
8666c60
try dynamic containers
Future-Outlier Dec 2, 2024
93580d6
try dynamic containers v2
Future-Outlier Dec 2, 2024
bcaaabd
try dynamic containers v3
Future-Outlier Dec 2, 2024
a321700
debug
Future-Outlier Dec 2, 2024
6464fae
update
Future-Outlier Dec 2, 2024
884943c
nit
Future-Outlier Dec 3, 2024
d4b5b96
Refresh Botton
Future-Outlier Dec 3, 2024
406227c
fix
Future-Outlier Dec 3, 2024
1e77f54
lint
Future-Outlier Dec 3, 2024
d70a2d5
Merge branch 'master' into flytekit-streaming-deck
Future-Outlier Dec 3, 2024
7fc6393
test new refresh
Future-Outlier Dec 3, 2024
6980140
lint
Future-Outlier Dec 3, 2024
c87a342
Revert back html code, collaborating with Lyon
Future-Outlier Dec 5, 2024
f609760
lint
Future-Outlier Dec 5, 2024
473ae11
nit
Future-Outlier Dec 5, 2024
008fe52
Merge branch 'master' into flytekit-streaming-deck
Future-Outlier Dec 12, 2024
f0b9028
nit
Future-Outlier Dec 16, 2024
d48efa9
update
Future-Outlier Dec 17, 2024
6b55930
better code
Future-Outlier Dec 17, 2024
b5912fb
update
Future-Outlier Dec 17, 2024
a681ccd
some notes for giving user params builder deck enabled
Future-Outlier Dec 18, 2024
048fdff
update
Future-Outlier Jan 2, 2025
7bcf15e
raise error when disabled deck and called Deck.publish()
Future-Outlier Jan 2, 2025
b6c41c3
lint
Future-Outlier Jan 2, 2025
be02f9f
lint
Future-Outlier Jan 2, 2025
cf83e06
update
Future-Outlier Jan 8, 2025
e137328
static method by YEE
Future-Outlier Jan 9, 2025
a59a56e
make Deck.publish more like a wrapper by moving enable deck checking …
Future-Outlier Jan 9, 2025
b8383be
lint
Future-Outlier Jan 9, 2025
dc6d203
print monodocs err
Future-Outlier Jan 9, 2025
41d8760
Fix monodocs
Future-Outlier Jan 9, 2025
b71cc19
use builder
Future-Outlier Jan 9, 2025
b5976fe
add translator test for deck serialization settings
Future-Outlier Jan 9, 2025
0c1a5a3
update
Future-Outlier Jan 13, 2025
d082456
fix
Future-Outlier Jan 13, 2025
4a8c68f
test
Future-Outlier Jan 13, 2025
b58527b
update
Future-Outlier Jan 13, 2025
2764ed4
remove blank
Future-Outlier Jan 13, 2025
90372db
update kevin's advice
Future-Outlier Jan 23, 2025
d8c408c
master-branch-idl
Future-Outlier Jan 23, 2025
5cacf11
update
Future-Outlier Feb 3, 2025
c447793
Use ENABLE_DECK as constant
Future-Outlier Feb 4, 2025
c5cc967
Merge branch 'master' into flytekit-streaming-deck
Future-Outlier Feb 4, 2025
1d6417a
update pydoc lint
Future-Outlier Feb 4, 2025
b0cd1ae
unit tests for enable deck ctx user param
Future-Outlier Feb 4, 2025
974b882
update agent's advice
Future-Outlier Feb 4, 2025
ea8b6e0
use pytest.mark.parametrize in test
Future-Outlier Feb 4, 2025
a642c9d
update
Future-Outlier Feb 4, 2025
e9aef35
update
Future-Outlier Feb 4, 2025
04775d7
deck
Future-Outlier Feb 4, 2025
34a4146
update
Future-Outlier Feb 4, 2025
f30382b
Merge branch 'master' into flytekit-streaming-deck
Future-Outlier Feb 4, 2025
b649f8f
add executiuon parameter test
Future-Outlier Feb 4, 2025
0565282
Only wrap `generates_deck` in a BoolValue when moving in and out of p…
eapolinario Feb 4, 2025
c0fd23a
Streaming Deck
Future-Outlier Feb 5, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions flytekit/bin/entrypoint.py
Original file line number Diff line number Diff line change
Expand Up @@ -121,6 +121,11 @@ def _dispatch_execute(
input_proto = utils.load_proto_from_file(_literals_pb2.LiteralMap, local_inputs_file)
idl_input_literals = _literal_models.LiteralMap.from_flyte_idl(input_proto)

# # TEST DECK
# if task_def is not None and not getattr(task_def, "disable_deck", True):
# _output_deck(task_def.name.split(".")[-1], ctx.user_space_params)
# print("@@@ OUTPUT DECK @@@")

# Step2
# Invoke task - dispatch_execute
outputs = task_def.dispatch_execute(ctx, idl_input_literals)
Expand Down
2 changes: 1 addition & 1 deletion flytekit/deck/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -18,5 +18,5 @@
SourceCodeRenderer
"""

from .deck import Deck, DeckField
from .deck import Deck, DeckField, DummyDeck
from .renderer import MarkdownRenderer, SourceCodeRenderer, TopFrameRenderer
63 changes: 62 additions & 1 deletion flytekit/deck/deck.py
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,40 @@

OUTPUT_DIR_JUPYTER_PREFIX = "jupyter"
DECK_FILE_NAME = "deck.html"
DUMMY_DECK_HTML = """
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will make it automatically refresh the page.. probably need that on the empty deck but not sure how to do the realtime behavior once it has content (you don't want to refresh while the user is browsing around or clicking through various tabs)

Suggested change
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<meta http-equiv="refresh" content="5" >

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

COOL, will try it, thank you

<title>Flytekit Status</title>
<style>
body {
display: flex;
justify-content: center;
align-items: center;
height: 100vh;
margin: 0;
font-family: Arial, sans-serif;
background-color: #f0f0f0;
}
.message {
background-color: #ffffff;
padding: 20px;
border: 1px solid #cccccc;
border-radius: 5px;
box-shadow: 0 0 10px rgba(0, 0, 0, 0.1);
text-align: center;
}
</style>
</head>
<body>
<div class="message">
<p> Flyte decks have not been created yet. </p>
</div>
</body>
</html>
"""


class DeckField(str, enum.Enum):
Expand Down Expand Up @@ -86,6 +120,27 @@ def name(self) -> str:
def html(self) -> str:
return self._html

@classmethod
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

technically this can be static right?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes static is better I think

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

since static method is bound to a class rather than the objects for that class.

reference: https://www.digitalocean.com/community/tutorials/python-static-method

def publish(cls):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Deck.publish, pulls the ExecutionParams from the current flytekit context, and calls _output_deck, which again calls the current context to get at the file_access code, to do an upload. what do you think about moving this error check inside _output_deck? making this function just a wrapper around that?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes we can do this, this can make the code more efficient

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

image

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

now it changed to warning, and move the logic to _output_deck

task_name = FlyteContextManager.current_context().user_space_params.task_id.name
new_user_params = FlyteContextManager.current_context().user_space_params
_output_deck(task_name, new_user_params)


class DummyDeck(Deck):
"""
The DummyDeck class is designed
"""

def __init__(self):
name = "dummy_deck"
html = DUMMY_DECK_HTML
super().__init__(name, html)

@property
def html(self) -> str:
return self._html


class TimeLineDeck(Deck):
"""
Expand Down Expand Up @@ -154,7 +209,13 @@ def _get_deck(
Get flyte deck html string
If ignore_jupyter is set to True, then it will return a str even in a jupyter environment.
"""
deck_map = {deck.name: deck.html for deck in new_user_params.decks}

deck_map = {deck.name: deck.html for deck in new_user_params.decks if deck.name != "dummy_deck"}

# If deck_map is empty after filtering, add DummyDeck
if not deck_map:
deck_map = {"dummy_deck": DUMMY_DECK_HTML}

nav_htmls = []
body_htmls = []

Expand Down
Loading