Skip to content

Conversation

@segiddins
Copy link
Contributor

@segiddins segiddins commented Mar 20, 2025

The goal here is to output files for click house to ingest, with a row for each download.

  • Fix platform parsing
  • How should this be deployed?
  • Store output in new s3 bucket
  • Serialize the output in a different format? Could try figuring out serde + parquet
  • Strip out non-gem-download requests?

@segiddins segiddins force-pushed the segiddins/clickhouse branch 5 times, most recently from 3638bf8 to bce0a43 Compare March 27, 2025 22:25
Signed-off-by: Samuel Giddins <[email protected]>
Signed-off-by: Samuel Giddins <[email protected]>
@segiddins segiddins force-pushed the segiddins/clickhouse branch from bce0a43 to e66098e Compare March 28, 2025 20:01
Signed-off-by: Samuel Giddins <[email protected]>
@segiddins segiddins marked this pull request as ready for review March 28, 2025 20:08
S: serde::Serializer,
{
lazy_static! {
static ref USER_AGENT_PARSER: ParseCtx = ParseCtx::new();
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@iximeow any recommendations for avoiding these two lazy_static!s inside serde serialize methods?

@segiddins segiddins merged commit e85c97b into main Mar 28, 2025
2 checks passed
@segiddins segiddins deleted the segiddins/clickhouse branch March 28, 2025 21:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants