SNOW-3138822: Add snow streamlit logs command for Streamlit apps live log streaming#2773
SNOW-3138822: Add snow streamlit logs command for Streamlit apps live log streaming#2773sfc-gh-svishnu wants to merge 8 commits intomainfrom
snow streamlit logs command for Streamlit apps live log streaming#2773Conversation
Add a new `snow streamlit logs` subcommand that connects to a deployed Streamlit app's developer log service via WebSocket and streams log entries to the terminal in real time. The command reads the app name from snowflake.yml (matching the `snow streamlit deploy` pattern) and supports both human-readable (default) and JSONL (`--format json`) output formats. Key changes: - Copy logs_service.proto and generate Python protobuf bindings - Add log_streaming.py with WebSocket connection, token auth, and streaming loop (ping/pong disabled for compatibility) - Add `logs` subcommand to streamlit commands with --tail and --format - Add protobuf and websockets dependencies to pyproject.toml - SQL injection guard on FQN interpolation
Replace the `websockets` (v16) library with `websocket-client` (v1.6+) for WebSocket log streaming. websocket-client is a better fit: no auto-ping (avoids the server compatibility issue), opcode-level frame control via recv_data(), stable API, and lighter (no async internals). Also add missing google.protobuf.timestamp_pb2 import in the generated pb2 file — required for the serialized descriptor that depends on google/protobuf/timestamp.proto.
- Add proto_codec.py with LogEntry dataclass, decode/encode helpers, and to_dict() for JSONL output. Decouples proto from app logic. - Refactor log_streaming.py to use proto_codec instead of raw pb2 - Return DeveloperApiToken dataclass instead of Tuple[str, str] - Add decode error handling (DecodeError, ValueError) with log.warning - Use ws.close(status=STATUS_NORMAL) for explicit close status - Use Typer min/max on --tail instead of manual validation - Add protobuf runtime version compat (try/except) in pb2 file - Add Apache 2.0 license headers to all new/empty files - Add test_streamlit_logs.py with 26 unit tests covering URL building, token fetching, proto codec round-trips, and WebSocket streaming
Consolidate try/finally so ws.close() runs on connect failure, fix MAX_TAIL_LINES to match proto spec (1000 not 10000), add streaming flag to suppress misleading output on early errors, add missing tests/streamlit/__init__.py, and add 3 new tests for KeyboardInterrupt, connect failure, and send_binary verification (29 total).
Add --name option to `snow streamlit logs` so users can specify a Streamlit app by fully qualified name without requiring snowflake.yml. When --name is provided, the command validates SPCSv2 runtime via server-side DESCRIBE STREAMLIT. When using project definition, it validates from entity_model.runtime_name locally.
Narrow broad except-Exception in recv loop to WebSocketException/OSError. Remove dead DEFAULT_TAIL_LINES/MAX_TAIL_LINES constants. Deduplicate SPCSv2 validation — both --name and project-definition paths now call validate_spcs_v2_runtime(). Error when both --name and entity_id are provided. Add command-level tests for streamlit_logs(). Log ws.close() errors at debug level instead of silencing. Extract WS mock boilerplate to pytest fixtures. Type to_dict return as dict[str, str | int]. Add safety comments for FQN validation and token header logging.
Move all local imports to module-level in log_streaming.py, commands.py, and test_streamlit_logs.py. Update mock patch targets in command-level tests to match. Update copyright year from 2024 to 2026 in all new files added by this branch.
| @@ -0,0 +1,62 @@ | |||
| # -*- coding: utf-8 -*- | |||
There was a problem hiding this comment.
when this file needs to be generated? Do you have any process for that in mind?
| tzlocal==5.3.1 | ||
| urllib3==2.6.3 | ||
| wcwidth==0.2.13 | ||
| websocket-client==1.9.0 |
There was a problem hiding this comment.
I will need to ask around if we need a formal approval to add a new package
| @@ -1,4 +1,4 @@ | |||
| # Copyright (c) 2024 Snowflake Inc. | |||
| # Copyright (c) 2026 Snowflake Inc. | |||
There was a problem hiding this comment.
Is it needed to update the year? I'd rather keep it as previous value
| "Use --name to identify the app directly, or use an " | ||
| "entity ID to reference a snowflake.yml definition." | ||
| ) | ||
| # --name flag provided: resolve FQN and validate via server-side DESCRIBE |
There was a problem hiding this comment.
If you find those comments useful then maybe it would be better to convert those into log.debug messages? That way we could get additional data when debugging client issues
|
|
||
| if name is not None: | ||
| if entity_id is not None: | ||
| raise ClickException( |
There was a problem hiding this comment.
Please use CliError and it's subclasses
| cli_console.step(f"Streaming logs (tail={tail_lines}). Press Ctrl+C to stop.") | ||
| sys.stdout.write("---\n") | ||
| sys.stdout.flush() | ||
| streaming = True | ||
|
|
||
| # 5. Read loop | ||
| while True: | ||
| try: | ||
| opcode, data = ws.recv_data() | ||
| except websocket.WebSocketTimeoutException: | ||
| # No message within the timeout window — loop back so we | ||
| # stay responsive to KeyboardInterrupt. | ||
| continue | ||
| except websocket.WebSocketConnectionClosedException: | ||
| log.debug("WebSocket connection closed by server") | ||
| break | ||
| except (websocket.WebSocketException, OSError) as e: | ||
| log.debug("WebSocket recv error: %s", e) | ||
| break | ||
|
|
||
| if opcode == websocket.ABNF.OPCODE_BINARY: | ||
| try: | ||
| entry = decode_log_entry(data) | ||
| except (DecodeError, ValueError) as e: | ||
| log.warning("Failed to decode log entry: %s", e) | ||
| continue | ||
| if json_output: | ||
| sys.stdout.write(json.dumps(entry.to_dict()) + "\n") | ||
| else: | ||
| sys.stdout.write(entry.format_line() + "\n") | ||
| sys.stdout.flush() | ||
| elif opcode == websocket.ABNF.OPCODE_CLOSE: | ||
| break | ||
| elif opcode == websocket.ABNF.OPCODE_PING: | ||
| ws.pong(data) | ||
|
|
||
| except KeyboardInterrupt: | ||
| pass | ||
| finally: | ||
| try: | ||
| ws.close(status=websocket.STATUS_NORMAL) | ||
| except Exception as e: | ||
| log.debug("Error closing WebSocket: %s", e) | ||
| if streaming: | ||
| sys.stdout.write("\n--- Log streaming stopped.\n") | ||
| sys.stdout.flush() |
There was a problem hiding this comment.
You're not sanitizing the data coming server with sanitize_for_terminal(). Not sure if users can emit custom logs in their app code, but if yes, then a malicious app can cause issues
SNOW-3138822: Add
snow streamlit logscommand for Streamlit apps live log streamingPre-review checklist
Changes description
Context
Developers building Streamlit apps on Snowflake currently have no CLI way to stream logs from a deployed app. To inspect runtime behavior, view errors, or debug issues they must leave the terminal and navigate the Snowsight UI. This PR adds a
snow streamlit logscommand that connects to the app's container runtime and streams log entries directly to the terminal in real time experience to Streamlit in Snowflake applications running in container runtime. This also enables cortex code or any other coding agents to access app logs and perform actions based on them.What this enables
--tail/-noption fetches up to N historical log lines before switching to live streaming (default 100, max 1000).snowflake.yml) — the standard Snowflake CLI workflow.--nameflag — pass a fully qualified name directly without needing a project definition file.jqor downstream tools.How it works (user-facing flow)
snow streamlit logs(withsnowflake.ymlpresent) orsnow streamlit logs --name db.schema.my_app.DESCRIBE STREAMLITcall verifies the app exists and runs on SPCSv2.SYSTEM$GET_STREAMLIT_DEVELOPER_API_TOKENto obtain a short-lived authentication token and the app's resource URI.StreamLogsRequest(protobuf) is sent to request historical + live logs.Flow diagram
flowchart TD A["snow streamlit logs [--name ...]"] --> B{--name provided?} B -- Yes --> C[Resolve FQN from --name flag] B -- No --> D{snowflake.yml exists?} D -- No --> E[Error: no app specified] D -- Yes --> F[Read entity from project definition] F --> G[Resolve FQN from entity model] C --> H["DESCRIBE STREAMLIT <fqn>"] G --> H H --> I{SPCSv2 runtime?} I -- No --> J[Error: unsupported runtime] I -- Yes --> K["SYSTEM$GET_STREAMLIT_DEVELOPER_API_TOKEN(fqn)"] K --> L[Receive token + resource URI] L --> M[Open WebSocket connection to resource URI] M --> N[Send StreamLogsRequest with tail_lines] N --> O[Receive loop] O --> P{Frame type?} P -- Binary --> Q[Decode protobuf LogEntry] Q --> R[Print formatted log line to stdout] R --> O P -- Close --> S[Connection closed by server] P -- Timeout --> O O -- Ctrl+C --> T[Close WebSocket gracefully] S --> T T --> U["Done — 'Log streaming stopped.'"]Demo
Screen.Recording.2026-02-22.at.3.28.54.PM.mov
Files added/modified
Testing
38 tests covering token retrieval, WebSocket URL construction, runtime validation, protobuf encode/decode, the full recv-loop (binary frames, timeouts, close frames, pings, decode errors, JSON output), and both command-level paths (
--nameflag and project definition) including error cases.