Skip to content

SNOW-3138822: Add snow streamlit logs command for Streamlit apps live log streaming#2773

Open
sfc-gh-svishnu wants to merge 8 commits intomainfrom
svishnu/streamlit-logs-command
Open

SNOW-3138822: Add snow streamlit logs command for Streamlit apps live log streaming#2773
sfc-gh-svishnu wants to merge 8 commits intomainfrom
svishnu/streamlit-logs-command

Conversation

@sfc-gh-svishnu
Copy link
Contributor

@sfc-gh-svishnu sfc-gh-svishnu commented Feb 22, 2026

SNOW-3138822: Add snow streamlit logs command for Streamlit apps live log streaming

Pre-review checklist

  • I've confirmed that instructions included in README.md are still correct after my changes in the codebase.
  • I've added or updated automated unit tests to verify correctness of my new code.
  • I've added or updated integration tests to verify correctness of my new code.
  • I've confirmed that my changes are working by executing CLI's commands manually on MacOS.
  • I've confirmed that my changes are working by executing CLI's commands manually on Windows.
  • I've confirmed that my changes are up-to-date with the target branch.
  • I've described my changes in the release notes.
  • I've described my changes in the section below.
  • I've described my changes in the documentation.

Changes description

Context

Developers building Streamlit apps on Snowflake currently have no CLI way to stream logs from a deployed app. To inspect runtime behavior, view errors, or debug issues they must leave the terminal and navigate the Snowsight UI. This PR adds a snow streamlit logs command that connects to the app's container runtime and streams log entries directly to the terminal in real time experience to Streamlit in Snowflake applications running in container runtime. This also enables cortex code or any other coding agents to access app logs and perform actions based on them.

What this enables

  • Live log streaming — log entries from the deployed Streamlit container are printed to stdout as they are produced.
  • Historical tail — the --tail / -n option fetches up to N historical log lines before switching to live streaming (default 100, max 1000).
  • Two ways to identify the app:
    • From a project definition (snowflake.yml) — the standard Snowflake CLI workflow.
    • Via the --name flag — pass a fully qualified name directly without needing a project definition file.
  • JSON output — when the CLI output format is set to JSON, each log entry is emitted as single-line JSONL, suitable for piping to jq or downstream tools.
  • SPCSv2 runtime validation — the command validates that the target app runs on the SPCSv2 container runtime before attempting to connect. A clear error message is shown if the runtime is unsupported.

How it works (user-facing flow)

  1. The user runs snow streamlit logs (with snowflake.yml present) or snow streamlit logs --name db.schema.my_app.
  2. The CLI resolves the fully qualified name of the Streamlit app.
  3. A DESCRIBE STREAMLIT call verifies the app exists and runs on SPCSv2.
  4. The CLI calls SYSTEM$GET_STREAMLIT_DEVELOPER_API_TOKEN to obtain a short-lived authentication token and the app's resource URI.
  5. The resource URI is converted to a WebSocket URL and the CLI opens a persistent WebSocket connection, authenticating with the token.
  6. A StreamLogsRequest (protobuf) is sent to request historical + live logs.
  7. Log entries arrive as binary protobuf frames, are decoded, formatted, and printed to stdout in real time.
  8. The user presses Ctrl+C to stop streaming.

Flow diagram

flowchart TD
    A["snow streamlit logs [--name ...]"] --> B{--name provided?}

    B -- Yes --> C[Resolve FQN from --name flag]
    B -- No --> D{snowflake.yml exists?}

    D -- No --> E[Error: no app specified]
    D -- Yes --> F[Read entity from project definition]
    F --> G[Resolve FQN from entity model]

    C --> H["DESCRIBE STREAMLIT <fqn>"]
    G --> H

    H --> I{SPCSv2 runtime?}
    I -- No --> J[Error: unsupported runtime]
    I -- Yes --> K["SYSTEM$GET_STREAMLIT_DEVELOPER_API_TOKEN(fqn)"]

    K --> L[Receive token + resource URI]
    L --> M[Open WebSocket connection to resource URI]
    M --> N[Send StreamLogsRequest with tail_lines]

    N --> O[Receive loop]
    O --> P{Frame type?}

    P -- Binary --> Q[Decode protobuf LogEntry]
    Q --> R[Print formatted log line to stdout]
    R --> O

    P -- Close --> S[Connection closed by server]
    P -- Timeout --> O

    O -- Ctrl+C --> T[Close WebSocket gracefully]
    S --> T
    T --> U["Done — 'Log streaming stopped.'"]
Loading

Demo

Screen.Recording.2026-02-22.at.3.28.54.PM.mov

Files added/modified

src/snowflake/cli/_plugins/streamlit/
    ├── commands.py                          # modified — added `logs` subcommand
    ├── log_streaming.py                     # new — WebSocket streaming client
    ├── proto_codec.py                       # new — protobuf encode/decode + LogEntry dataclass
    ├── proto/
    │   ├── __init__.py                      # new — package marker
    │   ├── developer/
    │   │   └── v1/
    │   │       └── logs_service.proto       # new — source proto definition
    │   └── generated/
    │       ├── __init__.py                  # new — package marker
    │       └── developer/
    │           ├── __init__.py              # new — package marker
    │           └── v1/
    │               ├── __init__.py          # new — package marker
    │               └── logs_service_pb2.py  # new — generated protobuf bindings

    tests/streamlit/
    ├── __init__.py                          # new — package marker
    └── test_streamlit_logs.py               # new — 38 unit tests

    pyproject.toml                           # modified — added protobuf + websocket-client deps
    pylock.toml                              # modified — lockfile update
    snyk/requirements.txt                    # modified — dependency snapshot

Testing

38 tests covering token retrieval, WebSocket URL construction, runtime validation, protobuf encode/decode, the full recv-loop (binary frames, timeouts, close frames, pings, decode errors, JSON output), and both command-level paths (--name flag and project definition) including error cases.

Add a new `snow streamlit logs` subcommand that connects to a deployed
Streamlit app's developer log service via WebSocket and streams log
entries to the terminal in real time. The command reads the app name
from snowflake.yml (matching the `snow streamlit deploy` pattern) and
supports both human-readable (default) and JSONL (`--format json`)
output formats.

Key changes:
- Copy logs_service.proto and generate Python protobuf bindings
- Add log_streaming.py with WebSocket connection, token auth, and
  streaming loop (ping/pong disabled for compatibility)
- Add `logs` subcommand to streamlit commands with --tail and --format
- Add protobuf and websockets dependencies to pyproject.toml
- SQL injection guard on FQN interpolation
Replace the `websockets` (v16) library with `websocket-client` (v1.6+)
for WebSocket log streaming. websocket-client is a better fit: no
auto-ping (avoids the server compatibility issue), opcode-level frame
control via recv_data(), stable API, and lighter (no async internals).

Also add missing google.protobuf.timestamp_pb2 import in the generated
pb2 file — required for the serialized descriptor that depends on
google/protobuf/timestamp.proto.
- Add proto_codec.py with LogEntry dataclass, decode/encode helpers,
  and to_dict() for JSONL output. Decouples proto from app logic.
- Refactor log_streaming.py to use proto_codec instead of raw pb2
- Return DeveloperApiToken dataclass instead of Tuple[str, str]
- Add decode error handling (DecodeError, ValueError) with log.warning
- Use ws.close(status=STATUS_NORMAL) for explicit close status
- Use Typer min/max on --tail instead of manual validation
- Add protobuf runtime version compat (try/except) in pb2 file
- Add Apache 2.0 license headers to all new/empty files
- Add test_streamlit_logs.py with 26 unit tests covering URL building,
  token fetching, proto codec round-trips, and WebSocket streaming
Consolidate try/finally so ws.close() runs on connect failure, fix
MAX_TAIL_LINES to match proto spec (1000 not 10000), add streaming
flag to suppress misleading output on early errors, add missing
tests/streamlit/__init__.py, and add 3 new tests for KeyboardInterrupt,
connect failure, and send_binary verification (29 total).
Add --name option to `snow streamlit logs` so users can specify a
Streamlit app by fully qualified name without requiring snowflake.yml.
When --name is provided, the command validates SPCSv2 runtime via
server-side DESCRIBE STREAMLIT. When using project definition, it
validates from entity_model.runtime_name locally.
Narrow broad except-Exception in recv loop to WebSocketException/OSError.
Remove dead DEFAULT_TAIL_LINES/MAX_TAIL_LINES constants. Deduplicate
SPCSv2 validation — both --name and project-definition paths now call
validate_spcs_v2_runtime(). Error when both --name and entity_id are
provided. Add command-level tests for streamlit_logs(). Log ws.close()
errors at debug level instead of silencing. Extract WS mock boilerplate
to pytest fixtures. Type to_dict return as dict[str, str | int]. Add
safety comments for FQN validation and token header logging.
Move all local imports to module-level in log_streaming.py, commands.py,
and test_streamlit_logs.py. Update mock patch targets in command-level
tests to match. Update copyright year from 2024 to 2026 in all new
files added by this branch.
@@ -0,0 +1,62 @@
# -*- coding: utf-8 -*-
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

when this file needs to be generated? Do you have any process for that in mind?

tzlocal==5.3.1
urllib3==2.6.3
wcwidth==0.2.13
websocket-client==1.9.0
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will need to ask around if we need a formal approval to add a new package

@@ -1,4 +1,4 @@
# Copyright (c) 2024 Snowflake Inc.
# Copyright (c) 2026 Snowflake Inc.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it needed to update the year? I'd rather keep it as previous value

"Use --name to identify the app directly, or use an "
"entity ID to reference a snowflake.yml definition."
)
# --name flag provided: resolve FQN and validate via server-side DESCRIBE
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you find those comments useful then maybe it would be better to convert those into log.debug messages? That way we could get additional data when debugging client issues


if name is not None:
if entity_id is not None:
raise ClickException(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please use CliError and it's subclasses

Comment on lines +183 to +228
cli_console.step(f"Streaming logs (tail={tail_lines}). Press Ctrl+C to stop.")
sys.stdout.write("---\n")
sys.stdout.flush()
streaming = True

# 5. Read loop
while True:
try:
opcode, data = ws.recv_data()
except websocket.WebSocketTimeoutException:
# No message within the timeout window — loop back so we
# stay responsive to KeyboardInterrupt.
continue
except websocket.WebSocketConnectionClosedException:
log.debug("WebSocket connection closed by server")
break
except (websocket.WebSocketException, OSError) as e:
log.debug("WebSocket recv error: %s", e)
break

if opcode == websocket.ABNF.OPCODE_BINARY:
try:
entry = decode_log_entry(data)
except (DecodeError, ValueError) as e:
log.warning("Failed to decode log entry: %s", e)
continue
if json_output:
sys.stdout.write(json.dumps(entry.to_dict()) + "\n")
else:
sys.stdout.write(entry.format_line() + "\n")
sys.stdout.flush()
elif opcode == websocket.ABNF.OPCODE_CLOSE:
break
elif opcode == websocket.ABNF.OPCODE_PING:
ws.pong(data)

except KeyboardInterrupt:
pass
finally:
try:
ws.close(status=websocket.STATUS_NORMAL)
except Exception as e:
log.debug("Error closing WebSocket: %s", e)
if streaming:
sys.stdout.write("\n--- Log streaming stopped.\n")
sys.stdout.flush()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're not sanitizing the data coming server with sanitize_for_terminal(). Not sure if users can emit custom logs in their app code, but if yes, then a malicious app can cause issues

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants