Skip to content

Update docs #1550

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
alyssajoyner wants to merge 1 commit into
base: main
Choose a base branch
from
Draft

Update docs #1550

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
35 changes: 35 additions & 0 deletions docs/sources/_index.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
---
description: This document introduces the ClickHouse data source
labels:
products:
- Grafana Cloud
keywords:
- data source
menuTitle: ClickHouse data source
title: ClickHouse data source
weight: 10
version: 0.1
---

# Official ClickHouse data source for Grafana

The ClickHouse data source plugin allows you to query and visualize ClickHouse data in Grafana.

- [Configure the ClickHouse data source](/docs/plugins/grafana-clickhouse-datasource/<CLICKHOUSE_PLUGIN_VERSION>/configure/)
- [ClickHouse query editor](/docs/plugins/grafana-clickhouse-datasource/<CLICKHOUSE_PLUGIN_VERSION>/editor/)
- [ClickHouse templates and variables](/docs/plugins/grafana-clickhouse-datasource/<CLICKHOUSE_PLUGIN_VERSION>/templates-and-variables/)

## Version compatibility

Users on Grafana `v9.x` and higher of Grafana can use `v4`.
Users on Grafana `v8.x` are encouraged to continue using `v2.2.0` of the plugin.

\* _As of 2.0 this plugin will only support ad hoc filters when using ClickHouse 22.7+_

## Get the most out of the ClickHouse data source

After installing and configuring ClickHouse you can:

- Add [Annotations](https://grafana.com/docs/grafana/latest/dashboards/annotations/).
- Add [Transformations](https://grafana.com/docs/grafana/latest/panels/transformations/)
- Set up [Alerting](https://grafana.com/docs/grafana/latest/alerting/)
80 changes: 80 additions & 0 deletions docs/sources/ad-hoc-filters.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,80 @@
---
description: This document outlines ad hoc filter options for the ClickHouse data source
labels:
products:
- Grafana Cloud
keywords:
- data source
menuTitle: Ad hoc filter for the ClickHouse data source
title: ClickHouse ad hoc filters
weight: 20
version: 0.1
---

### Ad Hoc Filters

Ad hoc filters are only supported with version 22.7+ of ClickHouse.

Ad hoc filters allow you to add key/value filters that are automatically added
to all metric queries that use the specified data source, without being
explicitly used in queries.

By default, Ad Hoc filters will be populated with all Tables and Columns. If
you have a default database defined in the Datasource settings, all Tables from
that database will be used to populate the filters. As this could be
slow/expensive, you can introduce a second variable to allow limiting the
Ad Hoc filters. It should be a `constant` type named `clickhouse_adhoc_query`
and can contain: a comma delimited list of databases, just one database, or a
database.table combination to show only columns for a single table.

Ad Hoc filters also work with the Map and JSON types for OTel data.
Map is the default, and will automatically convert the merged labels output into a usable filter.
To have the filter logic use JSON syntax, add a dashboard variable with a `constant` type called `clickhouse_adhoc_use_json` (the variable's `value` is ignored, it just has to be present).

For more information on Ad Hoc filters, check the [Grafana
docs](https://grafana.com/docs/grafana/latest/variables/variable-types/add-ad-hoc-filters/)

#### Using a query for Ad Hoc filters

The second `clickhouse_adhoc_query` also allows any valid ClickHouse query. The
query results will be used to populate your ad-hoc filter's selectable filters.
You may choose to hide this variable from view as it serves no further purpose.

For example, if `clickhouse_adhoc_query` is set to `SELECT DISTINCT
machine_name FROM mgbench.logs1` you would be able to select which machine
names are filtered for in the dashboard.

#### Manual Ad Hoc Filter Placement with `$__adHocFilters`

By default, ad-hoc filters are automatically applied to queries by detecting the
target table using SQL parsing. However, for queries that use CTEs or ClickHouse-specific
syntax like `INTERVAL` or aggregate functions with parameters, the automatic
detection may fail. In these cases, you can manually specify where to apply
ad-hoc filters using the `$__adHocFilters('table_name')` macro.

This macro expands to the ClickHouse `additional_table_filters` setting with the
currently active ad-hoc filters. It should be placed in the `SETTINGS` clause of
your query.

Example:

```sql
SELECT *
FROM (
SELECT * FROM my_complex_table
WHERE complicated_condition
) AS result
SETTINGS $__adHocFilters('my_complex_table')
```

When ad-hoc filters are active (e.g., `status = 'active'` and `region = 'us-west'`),
this expands to:

```sql
SELECT *
FROM (
SELECT * FROM my_complex_table
WHERE complicated_condition
) AS result
SETTINGS additional_table_filters={'my_complex_table': 'status = \'active\' AND region = \'us-west\''}
```
102 changes: 102 additions & 0 deletions docs/sources/configure.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,102 @@
---
description: This document outlines configuration options for the ClickHouse data source
labels:
products:
- Grafana Cloud
keywords:
- data source
menuTitle: Configure the ClickHouse data source
title: ClickHouse data source
weight: 20
version: 0.1
---

## Configure the ClickHouse data source

### ClickHouse user for the data source

Set up an ClickHouse user account with [readonly](https://clickhouse.com/docs/en/operations/settings/permissions-for-queries#settings_readonly) permission and access to
databases and tables you want to query.
Please note that Grafana does not validate that queries are safe. Queries can contain any SQL statement.
For example, statements like `ALTER TABLE system.users DELETE WHERE name='sadUser'`
and `DROP TABLE sadTable;` would be executed.

To configure a readonly user, follow these steps:

1. Create a `readonly` user profile following the [Creating Users and Roles in ClickHouse](https://clickhouse.com/docs/en/operations/access-rights) guide.
2. Ensure the `readonly` user has enough permission to modify the `max_execution_time` setting required by the underlying [clickhouse-go client](https://github.com/ClickHouse/clickhouse-go/).
3. If you're using a public ClickHouse instance, it's not recommended to set `readonly=2` in the `readonly` profile. Instead, leave `readonly=1` and set the constraint type of `max_execution_time` to [changeable_in_readonly](https://clickhouse.com/docs/en/operations/settings/constraints-on-settings) to allow modification of this setting.

### ClickHouse protocol support

The plugin supports both `Native` (default) and `HTTP` transport protocols.
This can be enabled in the configuration via the `protocol` configuration parameter.
Both protocols exchange data with ClickHouse using optimized native format.

Note that the default ports for `HTTP/S` and `Native` differ:

- HTTP - 8123
- HTTPS - 8443
- Native - 9000
- Native with TLS - 9440

### Manual configuration via UI

Once the plugin is installed on your Grafana instance, follow
[these instructions](https://grafana.com/docs/grafana/latest/datasources/add-a-data-source/)
to add a new ClickHouse data source, and enter configuration options.

### With a configuration file

It is possible to configure data sources using configuration files with Grafana’s provisioning system.
To read about how it works, refer to
[Provisioning Grafana data sources](https://grafana.com/docs/grafana/latest/administration/provisioning/#data-sources).

Here are some provisioning examples for this data source using basic authentication:

```yaml
apiVersion: 1
datasources:
- name: ClickHouse
type: grafana-clickhouse-datasource
jsonData:
defaultDatabase: database
port: 9000
host: localhost
username: username
tlsSkipVerify: false
# tlsAuth: <bool>
# tlsAuthWithCACert: <bool>
# secure: <bool>
# dialTimeout: <seconds>
# queryTimeout: <seconds>
# protocol: <native|http>
# defaultTable: <string>
# httpHeaders:
# - name: X-Example-Header
# secure: false
# value: <string>
# - name: Authorization
# secure: true
# logs:
# defaultDatabase: <string>
# defaultTable: <string>
# otelEnabled: <bool>
# otelVersion: <string>
# timeColumn: <string>
# ...Column: <string>
# traces:
# defaultDatabase: <string>
# defaultTable: <string>
# otelEnabled: <bool>
# otelVersion: <string>
# durationUnit: <seconds|milliseconds|microseconds|nanoseconds>
# traceIdColumn: <string>
# ...Column: <string>
secureJsonData:
password: password
# tlsCACert: <string>
# tlsClientCert: <string>
# tlsClientKey: <string>
# secureHttpHeaders.Authorization: <string>
```
117 changes: 117 additions & 0 deletions docs/sources/editor.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,117 @@
---
description: This document describes the ClickHouse query editor
labels:
products:
- Grafana Cloud
keywords:
- data source
menuTitle: ClickHouse query editor
title: ClickHouse query editor
weight: 30
version: 0.1
---

## Building queries

Queries can be built using the raw SQL editor or the query builder.
Queries can contain macros which simplify syntax and allow for
dynamic SQL generation.

### Time series

Time series visualization options are selectable after adding a `datetime`
field type to your query. This field will be used as the timestamp. You can
select time series visualizations using the visualization options. Grafana
interprets timestamp rows without explicit time zone as UTC. Any column except
`time` is treated as a value column.

#### Multi-line time series

To create multi-line time series, the query must return at least 3 fields in
the following order:

- field 1: `datetime` field with an alias of `time`
- field 2: value to group by
- field 3+: the metric values

For example:

```sql
SELECT log_time AS time, machine_group, avg(disk_free) AS avg_disk_free
FROM mgbench.logs1
GROUP BY machine_group, log_time
ORDER BY log_time
```

### Tables

Table visualizations will always be available for any valid ClickHouse query.

### Visualizing logs with the Logs Panel

To use the Logs panel your query must return a timestamp and string values. To default to the logs visualization in Explore mode, set the timestamp alias to _log_time_.

For example:

```sql
SELECT log_time AS log_time, machine_group, toString(avg(disk_free)) AS avg_disk_free
FROM logs1
GROUP BY machine_group, log_time
ORDER BY log_time
```

To force rendering as logs, in absence of a `log_time` column, set the Format to `Logs` (available from 2.2.0).

### Visualizing traces with the Traces Panel

Ensure your data meets the [requirements of the traces panel](https://grafana.com/docs/grafana/latest/explore/trace-integration/#data-api). This applies if using the visualization or Explore view.

Set the Format to `Trace` when constructing the query (available from 2.2.0).

If using the [Open Telemetry Collector and ClickHouse exporter](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/exporter/clickhouseexporter), the following query produces the required column names (these are case sensitive):

```sql
SELECT
TraceId AS traceID,
SpanId AS spanID,
SpanName AS operationName,
ParentSpanId AS parentSpanID,
ServiceName AS serviceName,
Duration / 1000000 AS duration,
Timestamp AS startTime,
arrayMap(key -> map('key', key, 'value', SpanAttributes[key]), mapKeys(SpanAttributes)) AS tags,
arrayMap(key -> map('key', key, 'value', ResourceAttributes[key]), mapKeys(ResourceAttributes)) AS serviceTags,
if(StatusCode IN ('Error', 'STATUS_CODE_ERROR'), 2, 0) AS statusCode
FROM otel.otel_traces
WHERE TraceId = '61d489320c01243966700e172ab37081'
ORDER BY startTime ASC
```

### Macros

To simplify syntax and to allow for dynamic parts, like date range filters, the query can contain macros.

Here is an example of a query with a macro that will use Grafana's time filter:

```sql
SELECT date_time, data_stuff
FROM test_data
WHERE $__timeFilter(date_time)
```

| Macro | Description | Output example |
| ---------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------- |
| _$\_\_dateFilter(columnName)_ | Replaced by a conditional that filters the data (using the provided column) based on the date range of the panel | `date >= toDate('2022-10-21') AND date <= toDate('2022-10-23')` |
| _$\_\_timeFilter(columnName)_ | Replaced by a conditional that filters the data (using the provided column) based on the time range of the panel in seconds | `time >= toDateTime(1415792726) AND time <= toDateTime(1447328726)` |
| _$\_\_timeFilter_ms(columnName)_ | Replaced by a conditional that filters the data (using the provided column) based on the time range of the panel in milliseconds | `time >= fromUnixTimestamp64Milli(1415792726123) AND time <= fromUnixTimestamp64Milli(1447328726456)` |
| _$\_\_dateTimeFilter(dateColumn, timeColumn)_ | Shorthand that combines $**dateFilter() AND $**timeFilter() using separate Date and DateTime columns. | `$__dateFilter(dateColumn) AND $__timeFilter(timeColumn)` |
| _$\_\_fromTime_ | Replaced by the starting time of the range of the panel casted to `DateTime` | `toDateTime(1415792726)` |
| _$\_\_toTime_ | Replaced by the ending time of the range of the panel casted to `DateTime` | `toDateTime(1447328726)` |
| _$\_\_fromTime_ms_ | Replaced by the starting time of the range of the panel casted to `DateTime64(3)` | `fromUnixTimestamp64Milli(1415792726123)` |
| _$\_\_toTime_ms_ | Replaced by the ending time of the range of the panel casted to `DateTime64(3)` | `fromUnixTimestamp64Milli(1447328726456)` |
| _$\_\_interval_s_ | Replaced by the interval in seconds | `20` |
| _$\_\_timeInterval(columnName)_ | Replaced by a function calculating the interval based on window size in seconds, useful when grouping | `toStartOfInterval(toDateTime(column), INTERVAL 20 second)` |
| _$\_\_timeInterval_ms(columnName)_ | Replaced by a function calculating the interval based on window size in milliseconds, useful when grouping | `toStartOfInterval(toDateTime64(column, 3), INTERVAL 20 millisecond)` |
| _$\_\_conditionalAll(condition, $templateVar)_ | Replaced by the first parameter when the template variable in the second parameter does not select every value. Replaced by the 1=1 when the template variable selects every value. | `condition` or `1=1` |

The plugin also supports notation using braces {}. Use this notation when queries are needed inside parameters.
22 changes: 22 additions & 0 deletions docs/sources/template-and-variables.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
---
description: This document describes using templates and variables with the ClickHouse data source
labels:
products:
- Grafana Cloud
keywords:
- data source
menuTitle: ClickHouse templates and variables
title: ClickHouse templates and variables
weight: 20
version: 0.1
---

### Templates and variables

To add a new ClickHouse query variable, refer to [Add a query
variable](https://grafana.com/docs/grafana/latest/variables/variable-types/add-query-variable/).

After creating a variable, you can use it in your ClickHouse queries by using
[Variable syntax](https://grafana.com/docs/grafana/latest/variables/syntax/).
For more information about variables, refer to [Templates and
variables](https://grafana.com/docs/grafana/latest/variables/).
Loading