Merge branch 'master' into chore-cli-and-cloud-pkg

cube-js · Jan 13, 2025 · 4b2ec1c · 4b2ec1c
2 parents 87bcc16 + e55beb1
commit 4b2ec1c
Show file tree

Hide file tree

Showing 194 changed files with 13,963 additions and 665 deletions.
diff --git a/.github/actions/integration/vertica.sh b/.github/actions/integration/vertica.sh
@@ -0,0 +1,12 @@
+#!/bin/bash
+set -eo pipefail
+
+# Debug log for test containers
+export DEBUG=testcontainers
+
+export TEST_VERTICA_VERSION=12.0.4-0
+
+echo "::group::Vertica ${TEST_VERTICA_VERSION}"
+docker pull vertica/vertica-ce:${TEST_VERTICA_VERSION}
+yarn lerna run --concurrency 1 --stream --no-prefix integration:vertica
+echo "::endgroup::"
diff --git a/.github/actions/smoke.sh b/.github/actions/smoke.sh
@@ -57,6 +57,10 @@ echo "::group::MongoBI"
 yarn lerna run --concurrency 1 --stream --no-prefix smoke:mongobi
 echo "::endgroup::"
 
+echo "::group::Vertica"
+yarn lerna run --concurrency 1 --stream --no-prefix smoke:vertica
+echo "::endgroup::"
+
 echo "::group::RBAC"
 yarn lerna run --concurrency 1 --stream --no-prefix smoke:rbac
 echo "::endgroup::"
diff --git a/.github/pull_request_template.md b/.github/pull_request_template.md
@@ -1,5 +1,5 @@
 **Check List**
-- [ ] Tests has been run in packages where changes made if available
+- [ ] Tests have been run in packages where changes made if available
 - [ ] Linter has been run for changed code
 - [ ] Tests for the changes have been added if not covered yet
 - [ ] Docs have been added / updated if required

diff --git a/.github/workflows/drivers-tests.yml b/.github/workflows/drivers-tests.yml
@@ -24,6 +24,7 @@ on:
       - 'packages/cubejs-mysql-driver/**'
       - 'packages/cubejs-postgres-driver/**'
       - 'packages/cubejs-snowflake-driver/**'
+      - 'packages/cubejs-vertica-driver/**'
 
       # To test SQL API Push down
       - 'packages/cubejs-backend-native/**'
@@ -49,6 +50,7 @@ on:
       - 'packages/cubejs-mysql-driver/**'
       - 'packages/cubejs-postgres-driver/**'
       - 'packages/cubejs-snowflake-driver/**'
+      - 'packages/cubejs-vertica-driver/**'
 
       # To test SQL API Push down
       - 'packages/cubejs-backend-native/**'

diff --git a/.github/workflows/push.yml b/.github/workflows/push.yml
@@ -336,7 +336,7 @@ jobs:
         node-version: [20.x]
         db: [
             'clickhouse', 'druid', 'elasticsearch', 'mssql', 'mysql', 'postgres', 'prestodb',
-            'mysql-aurora-serverless', 'crate', 'mongobi', 'firebolt', 'dremio'
+            'mysql-aurora-serverless', 'crate', 'mongobi', 'firebolt', 'dremio', 'vertica'
         ]
       fail-fast: false
 

diff --git a/.yarnrc.yml b/.yarnrc.yml
@@ -0,0 +1 @@
+nodeLinker: node-modules
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -3,6 +3,68 @@
 All notable changes to this project will be documented in this file.
 See [Conventional Commits](https://conventionalcommits.org) for commit guidelines.
 
+## [1.1.15](https://github.com/cube-js/cube/compare/v1.1.14...v1.1.15) (2025-01-13)
+
+
+### Bug Fixes
+
+* **cubesql:** Add folders to Cube Metadata ([#9089](https://github.com/cube-js/cube/issues/9089)) ([8d714d6](https://github.com/cube-js/cube/commit/8d714d6637862b36024aee5c3857267c0c167dbf))
+
+
+
+
+
+## [1.1.14](https://github.com/cube-js/cube/compare/v1.1.13...v1.1.14) (2025-01-09)
+
+
+### Bug Fixes
+
+* **cubesql:** add title field to Dimension Metadata ([#9084](https://github.com/cube-js/cube/issues/9084)) ([9653a23](https://github.com/cube-js/cube/commit/9653a23d9f477b391627755a533b2d6e8eae5656))
+
+
+
+
+
+## [1.1.13](https://github.com/cube-js/cube/compare/v1.1.12...v1.1.13) (2025-01-09)
+
+
+### Bug Fixes
+
+* **clickhouse-driver:** Support overriding Username & Password from Driver Config ([#9085](https://github.com/cube-js/cube/issues/9085)) ([c5aa6cc](https://github.com/cube-js/cube/commit/c5aa6cc4aca0c22dacd24253fe116f8619f96601))
+* **cubejs-cli:** Fix content type and body after switching from request to node-fetch ([#9088](https://github.com/cube-js/cube/issues/9088)) ([bf332ef](https://github.com/cube-js/cube/commit/bf332ef6761e8354115cc485d7d762abc9268181))
+* **schema-compiler:** Handle member expressions in keyDimensions ([#9083](https://github.com/cube-js/cube/issues/9083)) ([056a1d8](https://github.com/cube-js/cube/commit/056a1d88b4fd8055575dbb07967bc64b7065ce86))
+
+
+
+
+
+## [1.1.12](https://github.com/cube-js/cube/compare/v1.1.11...v1.1.12) (2025-01-09)
+
+
+### Bug Fixes
+
+* **api-gateway:** fix DAP RLS issue with denied queries and python conf ([#9054](https://github.com/cube-js/cube/issues/9054)) ([e661d2a](https://github.com/cube-js/cube/commit/e661d2af432f57f1109a3a24f91ea0ef0fe2af6c))
+* **backend-native:** Fix request sequence span ids ([#9077](https://github.com/cube-js/cube/issues/9077)) ([d48ef99](https://github.com/cube-js/cube/commit/d48ef99548d49ae0e9aa25ae052b3c432bd752f6))
+* **backend-native:** Pass req.securityContext to Python config ([#9049](https://github.com/cube-js/cube/issues/9049)) ([95021f2](https://github.com/cube-js/cube/commit/95021f277967ad88e783f766b2e546e3e6ebebf2))
+* **cubejs-client-core:** hierarchy typings ([#9042](https://github.com/cube-js/cube/issues/9042)) ([7c38845](https://github.com/cube-js/cube/commit/7c38845388fde0939423f0c35cf65cf1d67c1c36))
+* **dremio-driver:** Fix stale version of testing-shared dependency ([#9079](https://github.com/cube-js/cube/issues/9079)) ([ee826e1](https://github.com/cube-js/cube/commit/ee826e10c4e1a078083d32580a5d63bcdeb16628))
+* **schema-compiler:** fix time dimension granularity origin formatting in local timezone ([#9071](https://github.com/cube-js/cube/issues/9071)) ([c97526f](https://github.com/cube-js/cube/commit/c97526ff9145195786d4ec9617bcf298567bce7e))
+* **tesseract:** sqlAlias support ([f254f64](https://github.com/cube-js/cube/commit/f254f641f6f084f4f6dfc1a5d0d90e0c6f8be9cf))
+
+
+### Features
+
+* **cubejs-client-core:** Fill missing dates with custom measure value ([#8843](https://github.com/cube-js/cube/issues/8843)) ([ed3f6c9](https://github.com/cube-js/cube/commit/ed3f6c9b00f48889843080e800e4321d939ce5ec))
+* **cubesql:** Penalize zero members in wrapper ([#8927](https://github.com/cube-js/cube/issues/8927)) ([171ea35](https://github.com/cube-js/cube/commit/171ea351e739f705ddbf0d803a34b944cb8c9da5))
+* **duckdb-driver:** Add support for installing and loading DuckDB Extensions. ([#8744](https://github.com/cube-js/cube/issues/8744)) ([0e6ecd9](https://github.com/cube-js/cube/commit/0e6ecd925fc342bd4ac692e3ee364f29ee6b20df))
+* **server-core:** Support for scheduledRefreshTimeZones as function, passing securityContext ([#9002](https://github.com/cube-js/cube/issues/9002)) ([10e47fc](https://github.com/cube-js/cube/commit/10e47fc5472a3532a8f40f6f980f9802536a39de))
+* **tesseract:** Make measure an entry point for multi-fact join for now and support template for params ([#9053](https://github.com/cube-js/cube/issues/9053)) ([cb507c1](https://github.com/cube-js/cube/commit/cb507c1ab345828435d192d1f783998bbc5130b3))
+* **vertica-driver:** Introduce VerticaDriver ([#9081](https://github.com/cube-js/cube/issues/9081)) ([c43340d](https://github.com/cube-js/cube/commit/c43340d9cc7d06d7caea4803b4d6e00a3e82acdf)), closes [#2](https://github.com/cube-js/cube/issues/2) [#5](https://github.com/cube-js/cube/issues/5) [#6](https://github.com/cube-js/cube/issues/6)
+
+
+
+
+
 ## [1.1.11](https://github.com/cube-js/cube/compare/v1.1.10...v1.1.11) (2024-12-16)
 
 

diff --git a/dev-env.sh b/dev-env.sh
@@ -5,6 +5,8 @@ set -e
 CURRENT_DIR=$(pwd)
 SCRIPT_DIR=$(dirname "$(realpath "$0")")
 
+IGNORED_PACKAGES=("cubejs-testing*" "cubejs-linter" "cubejs-docker")
+
 # Function to install dependencies in root
 install_root_dependencies() {
     echo "Running 'yarn install' in the root directory..."
@@ -35,6 +37,21 @@ link_packages() {
     cd "$SCRIPT_DIR"
     for package in packages/*; do
         if [ -d "$package" ]; then
+            package_name=$(basename "$package")
+
+            skip_package="false"
+            for pattern in "${IGNORED_PACKAGES[@]}"; do
+                # shellcheck disable=SC2053
+                if [[ "$package_name" == $pattern ]]; then
+                    echo "Skipping $package_name..."
+                    skip_package="true"
+                fi
+            done
+
+            if [ "$skip_package" = "true" ]; then
+                continue
+            fi
+
             echo "Linking $package..."
             cd "$package"
             yarn link
@@ -106,6 +123,21 @@ link_project_packages() {
     cd "$CURRENT_DIR/$app_name"
     for package in "$SCRIPT_DIR"/packages/*; do
         if [ -d "$package" ]; then
+            package_name=$(basename "$package")
+
+            skip_package="false"
+            for pattern in "${IGNORED_PACKAGES[@]}"; do
+                # shellcheck disable=SC2053
+                if [[ "$package_name" == $pattern ]]; then
+                    echo "Skipping $package_name..."
+                    skip_package="true"
+                fi
+            done
+
+            if [ "$skip_package" = "true" ]; then
+                continue
+            fi
+
             package_name=$(node -p "require('$package/package.json').name")
             echo "Linking $package_name..."
             yarn link "$package_name"
@@ -146,6 +178,7 @@ show_help() {
     echo ""
     echo "  link           Link all packages and link them to a project"
     echo "                  Usage: ./dev-env.sh link [app_name]"
+    echo "                  If argument is omitted, cube packages will be marked as linked"
     echo ""
     echo "  setup          Run all steps (install, build, link, create project)"
     echo "                  Usage: ./dev-env.sh setup [app_name] [db_type]"
@@ -177,7 +210,9 @@ case "$command" in
         ;;
     "link")
         link_packages
-        link_project_packages "$2"
+        if [ -n "$2" ]; then
+            link_project_packages "$2"
+        fi
         ;;
     "drivers")
         get_db_types

diff --git a/docs/package.json b/docs/package.json
@@ -21,7 +21,7 @@
     "classnames": "^2.3.2",
     "cubedev-tracking": "^1.1.0",
     "date-fns": "^2.30.0",
-    "next": "^14.2.10",
+    "next": "^14.2.21",
     "next-sitemap": "^4.1.3",
     "nextra": "^2.13.2",
     "nextra-theme-docs": "^2.4.0",

diff --git a/docs/pages/guides/dbt.mdx b/docs/pages/guides/dbt.mdx
@@ -299,7 +299,7 @@ cubes:
         type: count
 
       - name: statuses
-        sql: "STRING_AGG({status})"
+        sql: "STRING_AGG({status}, ',')"
         type: string
 
     # Model-specific joins

diff --git a/docs/pages/guides/recipes/analytics/funnels.mdx b/docs/pages/guides/recipes/analytics/funnels.mdx
@@ -306,21 +306,6 @@ Funnel-based cubes have the following structure:
 - **time** - time dimension for the funnel. Use it to filter your analysis for
   specific dates or to analyze how conversion changes over time.
 
-In the following example, we use the `conversions` measure along with the
-`steps` dimension to display a classic bar chart showing the funnel's steps.
-
-<iframe
-  src="https://codesandbox.io/embed/nw87w1nnjm?fontsize=14"
-  style={{
-    width: "100%",
-    height: 500,
-    border: 0,
-    borderRadius: 4,
-    overflow: "hidden",
-  }}
-  sandbox="allow-modals allow-forms allow-popups allow-scripts allow-same-origin"
-></iframe>
-
 ## Performance considerations
 
 Funnel joins are extremely heavy for most modern databases and complexity grows

diff --git a/docs/pages/product/apis-integrations/rest-api/query-format.mdx b/docs/pages/product/apis-integrations/rest-api/query-format.mdx
@@ -520,8 +520,11 @@ only one of the following properties:
 }
 ```
 
-> **Note:** You can not put dimensions and measures filters in the same logical
-> operator.
+**You can not put dimensions and measures filters in the same logical operator.**
+When Cube generates a SQL query to the data source, dimension and measure filters
+are translated to expressions in `WHERE` and `HAVING` clauses, respectively.  
+In other words, dimension filters apply to raw (unaggregated) data and measure filters
+apply to aggregated data, so it's not possible to express such filters in SQL semantics.
 
 ## Time Dimensions Format
 

diff --git a/docs/pages/product/configuration/data-sources.mdx b/docs/pages/product/configuration/data-sources.mdx
@@ -59,6 +59,11 @@ sources][ref-config-multi-data-src] out of the box.
     imageUrl="https://static.cube.dev/icons/firebolt.svg"
     title="Firebolt"
   />
+   <GridItem
+    url="data-sources/vertica"
+    imageUrl="https://static.cube.dev/icons/vertica.svg"
+    title="Vertica"
+  />
 </Grid>
 
 ## Query engines

diff --git a/docs/pages/product/configuration/data-sources/snowflake.mdx b/docs/pages/product/configuration/data-sources/snowflake.mdx
@@ -150,6 +150,26 @@ CUBEJS_DB_EXPORT_GCS_CREDENTIALS=<BASE64_ENCODED_SERVICE_CREDENTIALS_JSON>
 CUBEJS_DB_EXPORT_INTEGRATION=gcs_int
 ```
 
+#### Azure
+
+To use Azure Blob Storage as an export bucket, follow [the guide on 
+using a Snowflake storage integration (Option 1)][snowflake-docs-azure]. 
+Take note of the integration name (`azure_int` from the example link)
+as you'll need it to configure Cube.
+
+[Retrieve the storage account access key][azure-bs-docs-get-key] from your Azure
+account.
+
+Once the Snowflake integration is set up, configure Cube using the following:
+
+```dotenv
+CUBEJS_DB_EXPORT_BUCKET_TYPE=azure
+CUBEJS_DB_EXPORT_BUCKET=wasbs://[email protected]
+CUBEJS_DB_EXPORT_BUCKET_AZURE_KEY=<AZURE_STORAGE_ACCOUNT_ACCESS_KEY>
+CUBEJS_DB_EXPORT_INTEGRATION=azure_int
+```
+
+
 ## SSL
 
 Cube does not require any additional configuration to enable SSL as Snowflake
@@ -176,3 +196,5 @@ connections are made over HTTPS.
 [snowflake-docs-warehouse]:
   https://docs.snowflake.com/en/user-guide/warehouses.html
 [snowflake-format-2]: https://docs.snowflake.com/en/user-guide/admin-account-identifier#format-2-account-locator-in-a-region
+[snowflake-docs-azure]: https://docs.snowflake.com/en/user-guide/data-load-azure-config#option-1-configuring-a-snowflake-storage-integration
+[azure-bs-docs-get-key]: https://docs.microsoft.com/en-us/azure/storage/common/storage-account-keys-manage?toc=%2Fazure%2Fstorage%2Fblobs%2Ftoc.json&tabs=azure-portal#view-account-access-keys
diff --git a/docs/pages/product/configuration/data-sources/vertica.mdx b/docs/pages/product/configuration/data-sources/vertica.mdx
@@ -0,0 +1,46 @@
+# Vertica
+
+## Prerequisites
+
+- The hostname for the [Vertica][vertica] database server
+- The username/password for the [Vertica][vertica] database server
+- The name of the database to use within the [Vertica][vertica] database server
+
+## Setup
+
+### Manual
+
+Add the following to a `.env` file in your Cube project:
+
+```dotenv
+CUBEJS_DB_TYPE=vertica
+CUBEJS_DB_HOST=my.vertica.host
+CUBEJS_DB_USER=vertica_user
+CUBEJS_DB_PASS=**********
+CUBEJS_DB_SCHEMA=my_vertica_schema
+```
+
+## Environment Variables
+
+| Environment Variable       | Description                                                                         | Possible Values                               | Required |
+| -------------------------- | ----------------------------------------------------------------------------------- | --------------------------------------------- | :------: |
+| `CUBEJS_DB_HOST`           | The host URL for a database                                                         | A valid database host URL                     |    ✅    |
+| `CUBEJS_DB_PORT`           | The port for the database connection                                                | A valid port number                           |    ❌    |
+| `CUBEJS_DB_USER`           | The username used to connect to the database                                        | A valid database username                     |    ✅    |
+| `CUBEJS_DB_PASS`           | The password used to connect to the database                                        | A valid database password                     |    ✅    |
+| `CUBEJS_DB_SCHEMA`         | The schema within the database to connect to                                        | A valid schema name within a Presto database  |    ✅    |
+| `CUBEJS_DB_SSL`            | If `true`, enables SSL encryption for database connections from Cube                | `true`, `false`                               |    ❌    |
+| `CUBEJS_CONCURRENCY`       | The number of concurrent connections each queue has to the database. Default is `2` | A valid number                                |    ❌    |
+| `CUBEJS_DB_MAX_POOL`       | The maximum number of concurrent database connections to pool. Default is `8`       | A valid number                                |    ❌    |
+
+
+## SSL
+
+To enable SSL-encrypted connections between Cube and Verica, set the
+`CUBEJS_DB_SSL` environment variable to `true`. For more information on how to
+configure custom certificates, please check out [Enable SSL Connections to the
+Database][ref-recipe-enable-ssl].
+
+[vertica]: https://www.vertica.com/documentation/vertica/all/
+[ref-recipe-enable-ssl]:/guides/recipes/data-sources/using-ssl-connections-to-data-source
+
diff --git a/docs/pages/product/data-modeling/concepts/polymorphic-cubes.mdx b/docs/pages/product/data-modeling/concepts/polymorphic-cubes.mdx
@@ -46,7 +46,7 @@ dimensions:
 ```yaml
 cubes:
   - name: users
-    sql_table: users
+    sql: SELECT * FROM users
 
     measures:
       - name: count
@@ -64,7 +64,7 @@ cubes:
 
 ```javascript
 cube(`users`, {
-  sql_table: `users`,
+  sql: `SELECT * FROM users`,
 
   measures: {
     count: {

diff --git a/docs/pages/product/distribution.mdx b/docs/pages/product/distribution.mdx
@@ -54,6 +54,13 @@ that breaking changes can happen both in _major_ and _minor_ versions. Before
 upgrading, it is recommended that you check the [release notes][link-releases]
 on GitHub and read [changelog entries][link-changelog] in the blog.
 
+<WarningBox>
+
+You can find a list of currently [deprecated features][link-github-deprecation] on GitHub.
+Usually, there are also deprecation notices in the documentation that look similar to this one.
+
+</WarningBox>
+
 Additionally, Cube Cloud provides [update channels][ref-update-channels] where
 versions are marked as either <Btn>Stable</Btn> or <Btn>Latest</Btn>.
 
@@ -92,6 +99,7 @@ List of LTS releases:
 [link-dockerhub-cubestore]: https://hub.docker.com/r/cubejs/cubestore/tags
 [link-apple-silicon]: https://support.apple.com/en-us/116943
 [link-semver]: https://semver.org
+[link-github-deprecation]: https://github.com/cube-js/cube/blob/master/DEPRECATION.md
 [ref-infrastructure]: /product/deployment/cloud
 [ref-workspace]: /product/workspace
 [ref-deployment]: /product/deployment

diff --git a/docs/pages/product/workspace/playground.mdx b/docs/pages/product/workspace/playground.mdx
@@ -156,6 +156,13 @@ on the <Btn>Generated SQL</Btn> tab. Click <Btn>Copy</Btn> to put it into your c
   src="https://ucarecdn.com/d2b6e514-9219-49ea-bae6-bcd0f1936542/"
 />
 
+<InfoBox>
+
+Note that a generated query can contain placeholders, e.g., `?` or `$1`, for parameters
+that Cube passes to an underlying data source driver later when executing the query.
+
+</InfoBox>
+
 ## Copying API queries
 
 View the queries for respective data APIs on the <Btn>SQL API</Btn>, <Btn>REST API</Btn>,