Skip to content

Commit a652429

Browse files
committed
provide description on dead letter topics
1 parent 0cc8fc6 commit a652429

File tree

1 file changed

+11
-4
lines changed

1 file changed

+11
-4
lines changed

broker/setup_broker/lvk/README.md

Lines changed: 11 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# Connect Pitt-Google to the LIGO/Virgo/KAGRA (LVK) Gravitational Wave Alert Stream
22

3-
April 2024 - Author: Christopher Hernandez
3+
Updated January 2025 - Author: Christopher Hernández
44

55
- [Overview](#overview)
66
- [Setup](#setup)
@@ -37,8 +37,10 @@ You may want to
3737
or
3838
[set up a GCP project from scratch](https://pitt-broker.readthedocs.io/en/latest/broker/run-a-broker-instance/initial-setup.html#setup-local-environment).
3939

40-
[Create secrets](https://cloud.google.com/secret-manager/docs/creating-and-accessing-secrets#create) for your client ID
41-
and client secret
40+
41+
### Use GCP's Secret Manager
42+
[Secret Manager](https://cloud.google.com/secret-manager/docs/overview) is a service that allows users to manage and store sensitive data. Use the following code snippet to [create secrets](https://cloud.google.com/secret-manager/docs/creating-and-accessing-secrets#create) for your client ID
43+
and client secret. This information will be used to deploy a broker instance with your client credentials.
4244

4345
```bash
4446
# define parameters
@@ -50,7 +52,6 @@ client_id="${survey}-${PROJECT_ID}-client-id"
5052
client_secret="${survey}-${PROJECT_ID}-client-secret"
5153

5254
# create secret(s)
53-
5455
gcloud secrets create "${client_id}" \
5556
--replication-policy="automatic"
5657
gcloud secrets create "${client_secret}" \
@@ -84,6 +85,10 @@ gcloud secrets add-iam-policy-binding "${client_id}" --member="serviceAccount:${
8485
gcloud secrets add-iam-policy-binding "${client_secret}" --member="serviceAccount:${user}" --role="${roleid}"
8586
```
8687

88+
### BigQuery subscriptions
89+
Our broker uses [BigQuery subscriptions](https://cloud.google.com/pubsub/docs/bigquery) to write alert data directly to a BigQuery table.
90+
Messages receieved by BigQuery subscriptions that fail to write data to BigQuery are negatively acknowledged and re-sent. If the messages fail enough times (default number of attempts = 5), then the message is subsequently moved to a [dead letter topic](https://cloud.google.com/pubsub/docs/handling-failures#dead_letter_topic). A subscription to this dead letter topic is automatically made, allowing the user to identify which messages failed to write data to BigQuery and why the write operation failed.
91+
8792
## Deploy broker instance
8893

8994
Clone the repo and cd into the directory:
@@ -110,6 +115,8 @@ complete, the VM will shut down automatically. You can check the status of the V
110115
[Google Cloud Console](https://console.cloud.google.com/compute).
111116
This entire process should take less than 10 minutes.
112117

118+
119+
113120
## Start the Consumer VM to ingest the LVK alert stream
114121

115122
```bash

0 commit comments

Comments
 (0)