Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Continuous Aggregates DSL #73

Merged
merged 17 commits into from
Sep 18, 2024
Merged

Continuous Aggregates DSL #73

merged 17 commits into from
Sep 18, 2024

Conversation

jonatas
Copy link
Collaborator

@jonatas jonatas commented Sep 13, 2024

It's related to #64.

Full example:

# lib/timescaledb/continuous_aggregates.rb
require 'bundler/inline'

gemfile do
  source 'https://rubygems.org'
  gem 'timescaledb', path: '.'
  gem 'pry'
end

class Download < ActiveRecord::Base
  extend Timescaledb::ActsAsHypertable
  include Timescaledb::ContinuousAggregatesHelper

  acts_as_hypertable time_column: 'ts'

  scope :total_downloads, -> { select("count(*) as total") }
  scope :downloads_by_gem, -> { select("gem_name, count(*) as total").group(:gem_name) }
  scope :downloads_by_version, -> { select("gem_name, gem_version, count(*) as total").group(:gem_name, :gem_version) }

  continuous_aggregates(
    timeframes: [:minute, :hour, :day, :month],
    scopes: [:total_downloads, :downloads_by_gem, :downloads_by_version],
    refresh_policy: {
      minute: { start_offset: "10 minutes", end_offset: "1 minute", schedule_interval: "1 minute" },
      hour:   { start_offset: "4 hour",     end_offset: "1 hour",   schedule_interval: "1 hour" },
      day:    { start_offset: "3 day",      end_offset: "1 day",    schedule_interval: "1 day" },
      month:  { start_offset: "3 month",    end_offset: "1 day",  schedule_interval: "1 say" }
  })
end

ActiveRecord::Base.establish_connection(ENV['DATABASE_URL'])
ActiveRecord::Base.connection.instance_exec do
  ActiveRecord::Base.logger = Logger.new(STDOUT)
  Download.drop_continuous_aggregates

  drop_table(:downloads, force: :cascade) if Download.table_exists?

  hypertable_options = {
    time_column: 'ts',
    chunk_time_interval: '1 day',
    compress_segmentby: 'gem_name, gem_version',
    compress_orderby: 'ts DESC',
    compression_interval: '7 days'
  }

  create_table(:downloads, id: false, hypertable: hypertable_options) do |t|
    t.timestamptz :ts, null: false
    t.text :gem_name, :gem_version, null: false
    t.jsonb :payload
  end

  Download.create_continuous_aggregates
end

# To refresh aggregates:
Download.refresh_aggregates([:hour, :day])  # Refreshes hourly and daily aggregates
# or
Download.refresh_aggregates  # Refreshes all defined timeframes
binding.pry

@@ -35,7 +35,7 @@ def define_association_scopes
CompressionSettings.where(hypertable_name: table_name)
end

scope :continuous_aggregates, -> do
scope :caggs, -> do
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I got some mad issue with this name, still not sure how to handle the naming clash. Just renaming this one as seems less prioritized.

@jonatas jonatas changed the title Continous Aggregates DSL Continuous Aggregates DSL Sep 14, 2024
@jonatas jonatas merged commit 2faa168 into master Sep 18, 2024
6 checks passed
@jonatas jonatas deleted the caggs-dsl branch September 18, 2024 17:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant