Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion broker/cloud_run/lsst/ps_to_storage/deploy.sh
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,6 @@ cr_module_name=$(define_GCP_resources "${survey}-${MODULE_NAME}") # lower case
gcs_avro_bucket=$(define_GCP_resources "${PROJECT_ID}-${survey}_alerts")
ps_deadletter_topic=$(define_GCP_resources "${survey}-deadletter")
ps_input_subscrip=$(define_GCP_resources "${survey}-alerts_raw") # pub/sub subscription used to trigger cloud run module
ps_subscription_avro=$(define_GCP_resources "${survey}-alert_avros-counter")
ps_topic_avro=$(define_GCP_resources "projects/${PROJECT_ID}/topics/${survey}-alert_avros")
ps_trigger_topic=$(define_GCP_resources "${survey}-alerts_raw")
runinvoker_svcact="cloud-run-invoker@${PROJECT_ID}.iam.gserviceaccount.com"
Expand Down
45 changes: 45 additions & 0 deletions docs/source/working-notes/troyraen/stream-looper/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -76,6 +76,51 @@
--machine-type "${vmtype}"
```

### LSST Stream

This section documents commands used to start a stream looper for LSST alerts.

```sh
# ssh into existing stream-looper VM.
# Option 1: Go to the VM's page on GCP Console and click "SSH" button
# Option 2: From terminal, do:
gcloud compute ssh stream-looper

# Start a new screen.
screen -S lsst-stream-looper
# Note:
# To detach from the screen: "Ctrl+a" then "d".
# To reattach: `screen -r lsst-stream-looper`.
# Only the user that started the screen can reattach to it.

cd /home/consumer_sim

# Create the topic. We assume the subscription already exists.
project_id="ardent-cycling-243415"
topic="lsst-loop"
gcloud pubsub topics create "${topic}"
# Make the topic public.
user="allUsers"
roleid="projects/${project_id}/roles/userPublic"
gcloud pubsub topics add-iam-policy-binding "${topic}" --member="${user}" --role="${roleid}"
# Note: Got PERMISSION_DENIED (accessed VM from Console SSH). Workaround:
# Went to GCP Console, navigate to the topic, click SHOW INFO PANEL, click ADD PRINCIPAL,
# set principal = allUsers and role = pubsub topics attachSubscription.

python3
```

```python
from streaming_stream_looper import StreamLooper

topic = "lsst-loop" # Must be same as above.
subscrip = "lsst-alerts-reservoir"

# Note: The script currently on the VM has Project ID hardcoded to "ardent-cycling-243415".
# Not changing that now because ztf looper is also using the script.
StreamLooper(topic, subscrip).run_looper()
```

### ZTF Stream

Set a startup script to execute the python file in a background thread:
Expand Down
4 changes: 4 additions & 0 deletions docs/source/working-notes/troyraen/v0.5.0/stream-looper.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,9 @@
# docs/source/working-notes/troyraen/v0.5.0/stream-looper.md

> [!WARNING]
> This page describes an old implementation of the stream-looper and will not be updated.
> For the current implementation, see [../stream-looper/README.md](../stream-looper/README.md).

## Setup stream-looper VM and topic

### Create the Pub/Sub topic and allow public subscriptions
Expand Down