You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This is a Lambda-lith hit through the Lambda Function URLs (FURL) by reverse proxing through CloudFront. It is written
168
+
This is a Lambda-lith hit through the Lambda Function URLs (FURL) by reverse proxying through CloudFront. It is written
163
169
in TypeScript and uses [tRPC](https://trpc.io/) to handle API requests.
164
170
165
171
The Queries to Athena are synchronous, the connection timeout between CloudFront and the FURL has been increased
@@ -171,20 +177,22 @@ There are three available authentication configurations:
171
177
-**AWS Cogntio**, recommended for production
172
178
173
179
⚠️ Partitions are not automatically created in Athena, they have to be created manually by the user by clicking the
174
-
"Create/Refresh Partitions" button in the frontend. This has to be done when ever a new site is added or a new month
180
+
"Create/Refresh Partitions" button in the frontend. This has to be done whenever a new site is added or a new month
175
181
starts.
176
182
177
183
### Ingestion API
178
184
179
-
Similarly to the backend, it is also a TS Lambda-lith that is hit through the FURL by reverse proxing through CloudFront.
185
+
Similarly to the backend, it is also a TS Lambda-lith that is hit through the FURL by reverse proxying through CloudFront.
180
186
It also uses [tRPC](https://trpc.io/) but uses the [trpc-openapi](https://github.com/jlalmes/trpc-openapi) package to
181
187
generate an OpenAPI spec. This is used to generate the API types used in the [client JS package](https://www.npmjs.com/package/serverless-website-analytics-client).
182
188
and can also be used to generate other language client libraries.
183
189
184
190
The lambda function then saves the data to S3 through a Kinesis Firehose. The Firehose is configured to save the data
185
-
in a partitioned manner, by site, year and month. The data is saved in parquet format. Location data is obtained by
186
-
looking the IP address up in the [MaxMind GeoLite2](https://dev.maxmind.com/geoip/geoip2/geolite2/) database. We don't
187
-
store any Personally Identifiable Information (PII) in the logs, the IP address is never stored.
191
+
in a partitioned manner, by site, year and month. The data is saved in parquet format, buffered for 1 minute, which means
192
+
the date will be stored after about 1min ± 1min.
193
+
194
+
Location data is obtained by looking the IP address up in the [MaxMind GeoLite2](https://dev.maxmind.com/geoip/geoip2/geolite2/) database.
195
+
We don't store any Personally Identifiable Information (PII) in the logs or S3, the IP address is never stored.
188
196
189
197
## Sponsors
190
198
@@ -193,7 +201,6 @@ Proudly sponsored by:
193
201
all your AWS credentials and more.
194
202
- Interested in [sponsoring](https://rehanvdm.com/contact-me)?
195
203
196
-
197
204
## Contributing
198
205
199
206
See [CONTRIBUTING.md](docs/CONTRIBUTING.md) for more info on how to contribute + design decisions.
0 commit comments