[Core feature] Real-time Flyte Deck #5574
Description
Motivation: Why do you think this is important?
It's extremely useful to see the metrics, data, logs, and links on the Flyte Deck while the task is running. However, currently, the Flyte Deck HTML is uploaded to the remote bucket only after the task succeeds.
Goal: What should the final outcome look like, ideally?
FlytePropeller
- Propeller should send the deck URI (in NodeExecutionEvent) back to flyteadmin when the task starts running.
Flytekit
- Upload a dummy deck when the task starts running.
- Add
publish_decksAPI, so users can decide when to upload the decks. - Upload flyte deck if the task fails.
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Flytekit Status</title>
<style>
body {
display: flex;
justify-content: center;
align-items: center;
height: 100vh;
margin: 0;
font-family: Arial, sans-serif;
background-color: #f0f0f0;
}
.message {
background-color: #ffffff;
padding: 20px;
border: 1px solid #cccccc;
border-radius: 5px;
box-shadow: 0 0 10px rgba(0, 0, 0, 0.1);
text-align: center;
}
</style>
</head>
<body>
<div class="message">
<p> Flyte decks have not been created yet. </p>
</div>
</body>
</html>@task(enable_deck=True)
def foo():
ctx = flytekit.current_context()
ctx.publish_decks()
for epoch in range(100):
...
flytekit.Deck("metrics", ...)
ctx.publish_decks()FlyteConsole
- Show the Flyte deck button while the task is running
- Auto refresh flyte deck. Ifame should read the latest deck html in the remote bucket.
Describe alternatives you've considered
NA
Propose: Link/Inline OR Additional context
NA
Are you sure this issue hasn't been raised already?
- Yes
Have you read the Code of Conduct?
- Yes