Skip to content

Commit d8c8992

Browse files
alyssajoyneralyssajoyner
committed
Update docs
1 parent 0fc5c3d commit d8c8992

File tree

5 files changed

+356
-0
lines changed

5 files changed

+356
-0
lines changed

‎docs/sources/_index.md‎

Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,35 @@
1+
---
2+
description: This document introduces the ClickHouse data source
3+
labels:
4+
products:
5+
- Grafana Cloud
6+
keywords:
7+
- data source
8+
menuTitle: ClickHouse data source
9+
title: ClickHouse data source
10+
weight: 10
11+
version: 0.1
12+
---
13+
14+
# Official ClickHouse data source for Grafana
15+
16+
The ClickHouse data source plugin allows you to query and visualize ClickHouse data in Grafana.
17+
18+
- [Configure the ClickHouse data source](/docs/plugins/grafana-clickhouse-datasource/<CLICKHOUSE_PLUGIN_VERSION>/configure/)
19+
- [ClickHouse query editor](/docs/plugins/grafana-clickhouse-datasource/<CLICKHOUSE_PLUGIN_VERSION>/editor/)
20+
- [ClickHouse templates and variables](/docs/plugins/grafana-clickhouse-datasource/<CLICKHOUSE_PLUGIN_VERSION>/templates-and-variables/)
21+
22+
## Version compatibility
23+
24+
Users on Grafana `v9.x` and higher of Grafana can use `v4`.
25+
Users on Grafana `v8.x` are encouraged to continue using `v2.2.0` of the plugin.
26+
27+
\* _As of 2.0 this plugin will only support ad hoc filters when using ClickHouse 22.7+_
28+
29+
## Get the most out of the ClickHouse data source
30+
31+
After installing and configuring ClickHouse you can:
32+
33+
- Add [Annotations](https://grafana.com/docs/grafana/latest/dashboards/annotations/).
34+
- Add [Transformations](https://grafana.com/docs/grafana/latest/panels/transformations/)
35+
- Set up [Alerting](https://grafana.com/docs/grafana/latest/alerting/)

‎docs/sources/ad-hoc-filters.md‎

Lines changed: 80 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,80 @@
1+
---
2+
description: This document outlines ad hoc filter options for the ClickHouse data source
3+
labels:
4+
products:
5+
- Grafana Cloud
6+
keywords:
7+
- data source
8+
menuTitle: Ad hoc filter for the ClickHouse data source
9+
title: ClickHouse ad hoc filters
10+
weight: 20
11+
version: 0.1
12+
---
13+
14+
### Ad Hoc Filters
15+
16+
Ad hoc filters are only supported with version 22.7+ of ClickHouse.
17+
18+
Ad hoc filters allow you to add key/value filters that are automatically added
19+
to all metric queries that use the specified data source, without being
20+
explicitly used in queries.
21+
22+
By default, Ad Hoc filters will be populated with all Tables and Columns. If
23+
you have a default database defined in the Datasource settings, all Tables from
24+
that database will be used to populate the filters. As this could be
25+
slow/expensive, you can introduce a second variable to allow limiting the
26+
Ad Hoc filters. It should be a `constant` type named `clickhouse_adhoc_query`
27+
and can contain: a comma delimited list of databases, just one database, or a
28+
database.table combination to show only columns for a single table.
29+
30+
Ad Hoc filters also work with the Map and JSON types for OTel data.
31+
Map is the default, and will automatically convert the merged labels output into a usable filter.
32+
To have the filter logic use JSON syntax, add a dashboard variable with a `constant` type called `clickhouse_adhoc_use_json` (the variable's `value` is ignored, it just has to be present).
33+
34+
For more information on Ad Hoc filters, check the [Grafana
35+
docs](https://grafana.com/docs/grafana/latest/variables/variable-types/add-ad-hoc-filters/)
36+
37+
#### Using a query for Ad Hoc filters
38+
39+
The second `clickhouse_adhoc_query` also allows any valid ClickHouse query. The
40+
query results will be used to populate your ad-hoc filter's selectable filters.
41+
You may choose to hide this variable from view as it serves no further purpose.
42+
43+
For example, if `clickhouse_adhoc_query` is set to `SELECT DISTINCT
44+
machine_name FROM mgbench.logs1` you would be able to select which machine
45+
names are filtered for in the dashboard.
46+
47+
#### Manual Ad Hoc Filter Placement with `$__adHocFilters`
48+
49+
By default, ad-hoc filters are automatically applied to queries by detecting the
50+
target table using SQL parsing. However, for queries that use CTEs or ClickHouse-specific
51+
syntax like `INTERVAL` or aggregate functions with parameters, the automatic
52+
detection may fail. In these cases, you can manually specify where to apply
53+
ad-hoc filters using the `$__adHocFilters('table_name')` macro.
54+
55+
This macro expands to the ClickHouse `additional_table_filters` setting with the
56+
currently active ad-hoc filters. It should be placed in the `SETTINGS` clause of
57+
your query.
58+
59+
Example:
60+
61+
```sql
62+
SELECT *
63+
FROM (
64+
SELECT * FROM my_complex_table
65+
WHERE complicated_condition
66+
) AS result
67+
SETTINGS $__adHocFilters('my_complex_table')
68+
```
69+
70+
When ad-hoc filters are active (e.g., `status = 'active'` and `region = 'us-west'`),
71+
this expands to:
72+
73+
```sql
74+
SELECT *
75+
FROM (
76+
SELECT * FROM my_complex_table
77+
WHERE complicated_condition
78+
) AS result
79+
SETTINGS additional_table_filters={'my_complex_table': 'status = \'active\' AND region = \'us-west\''}
80+
```

‎docs/sources/configure.md‎

Lines changed: 102 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,102 @@
1+
---
2+
description: This document outlines configuration options for the ClickHouse data source
3+
labels:
4+
products:
5+
- Grafana Cloud
6+
keywords:
7+
- data source
8+
menuTitle: Configure the ClickHouse data source
9+
title: ClickHouse data source
10+
weight: 20
11+
version: 0.1
12+
---
13+
14+
## Configure the ClickHouse data source
15+
16+
### ClickHouse user for the data source
17+
18+
Set up an ClickHouse user account with [readonly](https://clickhouse.com/docs/en/operations/settings/permissions-for-queries#settings_readonly) permission and access to
19+
databases and tables you want to query.
20+
Please note that Grafana does not validate that queries are safe. Queries can contain any SQL statement.
21+
For example, statements like `ALTER TABLE system.users DELETE WHERE name='sadUser'`
22+
and `DROP TABLE sadTable;` would be executed.
23+
24+
To configure a readonly user, follow these steps:
25+
26+
1. Create a `readonly` user profile following the [Creating Users and Roles in ClickHouse](https://clickhouse.com/docs/en/operations/access-rights) guide.
27+
2. Ensure the `readonly` user has enough permission to modify the `max_execution_time` setting required by the underlying [clickhouse-go client](https://github.com/ClickHouse/clickhouse-go/).
28+
3. If you're using a public ClickHouse instance, it's not recommended to set `readonly=2` in the `readonly` profile. Instead, leave `readonly=1` and set the constraint type of `max_execution_time` to [changeable_in_readonly](https://clickhouse.com/docs/en/operations/settings/constraints-on-settings) to allow modification of this setting.
29+
30+
### ClickHouse protocol support
31+
32+
The plugin supports both `Native` (default) and `HTTP` transport protocols.
33+
This can be enabled in the configuration via the `protocol` configuration parameter.
34+
Both protocols exchange data with ClickHouse using optimized native format.
35+
36+
Note that the default ports for `HTTP/S` and `Native` differ:
37+
38+
- HTTP - 8123
39+
- HTTPS - 8443
40+
- Native - 9000
41+
- Native with TLS - 9440
42+
43+
### Manual configuration via UI
44+
45+
Once the plugin is installed on your Grafana instance, follow
46+
[these instructions](https://grafana.com/docs/grafana/latest/datasources/add-a-data-source/)
47+
to add a new ClickHouse data source, and enter configuration options.
48+
49+
### With a configuration file
50+
51+
It is possible to configure data sources using configuration files with Grafana’s provisioning system.
52+
To read about how it works, refer to
53+
[Provisioning Grafana data sources](https://grafana.com/docs/grafana/latest/administration/provisioning/#data-sources).
54+
55+
Here are some provisioning examples for this data source using basic authentication:
56+
57+
```yaml
58+
apiVersion: 1
59+
datasources:
60+
- name: ClickHouse
61+
type: grafana-clickhouse-datasource
62+
jsonData:
63+
defaultDatabase: database
64+
port: 9000
65+
host: localhost
66+
username: username
67+
tlsSkipVerify: false
68+
# tlsAuth: <bool>
69+
# tlsAuthWithCACert: <bool>
70+
# secure: <bool>
71+
# dialTimeout: <seconds>
72+
# queryTimeout: <seconds>
73+
# protocol: <native|http>
74+
# defaultTable: <string>
75+
# httpHeaders:
76+
# - name: X-Example-Header
77+
# secure: false
78+
# value: <string>
79+
# - name: Authorization
80+
# secure: true
81+
# logs:
82+
# defaultDatabase: <string>
83+
# defaultTable: <string>
84+
# otelEnabled: <bool>
85+
# otelVersion: <string>
86+
# timeColumn: <string>
87+
# ...Column: <string>
88+
# traces:
89+
# defaultDatabase: <string>
90+
# defaultTable: <string>
91+
# otelEnabled: <bool>
92+
# otelVersion: <string>
93+
# durationUnit: <seconds|milliseconds|microseconds|nanoseconds>
94+
# traceIdColumn: <string>
95+
# ...Column: <string>
96+
secureJsonData:
97+
password: password
98+
# tlsCACert: <string>
99+
# tlsClientCert: <string>
100+
# tlsClientKey: <string>
101+
# secureHttpHeaders.Authorization: <string>
102+
```

‎docs/sources/editor.md‎

Lines changed: 117 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,117 @@
1+
---
2+
description: This document describes the ClickHouse query editor
3+
labels:
4+
products:
5+
- Grafana Cloud
6+
keywords:
7+
- data source
8+
menuTitle: ClickHouse query editor
9+
title: ClickHouse query editor
10+
weight: 30
11+
version: 0.1
12+
---
13+
14+
## Building queries
15+
16+
Queries can be built using the raw SQL editor or the query builder.
17+
Queries can contain macros which simplify syntax and allow for
18+
dynamic SQL generation.
19+
20+
### Time series
21+
22+
Time series visualization options are selectable after adding a `datetime`
23+
field type to your query. This field will be used as the timestamp. You can
24+
select time series visualizations using the visualization options. Grafana
25+
interprets timestamp rows without explicit time zone as UTC. Any column except
26+
`time` is treated as a value column.
27+
28+
#### Multi-line time series
29+
30+
To create multi-line time series, the query must return at least 3 fields in
31+
the following order:
32+
33+
- field 1: `datetime` field with an alias of `time`
34+
- field 2: value to group by
35+
- field 3+: the metric values
36+
37+
For example:
38+
39+
```sql
40+
SELECT log_time AS time, machine_group, avg(disk_free) AS avg_disk_free
41+
FROM mgbench.logs1
42+
GROUP BY machine_group, log_time
43+
ORDER BY log_time
44+
```
45+
46+
### Tables
47+
48+
Table visualizations will always be available for any valid ClickHouse query.
49+
50+
### Visualizing logs with the Logs Panel
51+
52+
To use the Logs panel your query must return a timestamp and string values. To default to the logs visualization in Explore mode, set the timestamp alias to _log_time_.
53+
54+
For example:
55+
56+
```sql
57+
SELECT log_time AS log_time, machine_group, toString(avg(disk_free)) AS avg_disk_free
58+
FROM logs1
59+
GROUP BY machine_group, log_time
60+
ORDER BY log_time
61+
```
62+
63+
To force rendering as logs, in absence of a `log_time` column, set the Format to `Logs` (available from 2.2.0).
64+
65+
### Visualizing traces with the Traces Panel
66+
67+
Ensure your data meets the [requirements of the traces panel](https://grafana.com/docs/grafana/latest/explore/trace-integration/#data-api). This applies if using the visualization or Explore view.
68+
69+
Set the Format to `Trace` when constructing the query (available from 2.2.0).
70+
71+
If using the [Open Telemetry Collector and ClickHouse exporter](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/exporter/clickhouseexporter), the following query produces the required column names (these are case sensitive):
72+
73+
```sql
74+
SELECT
75+
TraceId AS traceID,
76+
SpanId AS spanID,
77+
SpanName AS operationName,
78+
ParentSpanId AS parentSpanID,
79+
ServiceName AS serviceName,
80+
Duration / 1000000 AS duration,
81+
Timestamp AS startTime,
82+
arrayMap(key -> map('key', key, 'value', SpanAttributes[key]), mapKeys(SpanAttributes)) AS tags,
83+
arrayMap(key -> map('key', key, 'value', ResourceAttributes[key]), mapKeys(ResourceAttributes)) AS serviceTags,
84+
if(StatusCode IN ('Error', 'STATUS_CODE_ERROR'), 2, 0) AS statusCode
85+
FROM otel.otel_traces
86+
WHERE TraceId = '61d489320c01243966700e172ab37081'
87+
ORDER BY startTime ASC
88+
```
89+
90+
### Macros
91+
92+
To simplify syntax and to allow for dynamic parts, like date range filters, the query can contain macros.
93+
94+
Here is an example of a query with a macro that will use Grafana's time filter:
95+
96+
```sql
97+
SELECT date_time, data_stuff
98+
FROM test_data
99+
WHERE $__timeFilter(date_time)
100+
```
101+
102+
| Macro | Description | Output example |
103+
| ---------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------- |
104+
| _$\_\_dateFilter(columnName)_ | Replaced by a conditional that filters the data (using the provided column) based on the date range of the panel | `date >= toDate('2022-10-21') AND date <= toDate('2022-10-23')` |
105+
| _$\_\_timeFilter(columnName)_ | Replaced by a conditional that filters the data (using the provided column) based on the time range of the panel in seconds | `time >= toDateTime(1415792726) AND time <= toDateTime(1447328726)` |
106+
| _$\_\_timeFilter_ms(columnName)_ | Replaced by a conditional that filters the data (using the provided column) based on the time range of the panel in milliseconds | `time >= fromUnixTimestamp64Milli(1415792726123) AND time <= fromUnixTimestamp64Milli(1447328726456)` |
107+
| _$\_\_dateTimeFilter(dateColumn, timeColumn)_ | Shorthand that combines $**dateFilter() AND $**timeFilter() using separate Date and DateTime columns. | `$__dateFilter(dateColumn) AND $__timeFilter(timeColumn)` |
108+
| _$\_\_fromTime_ | Replaced by the starting time of the range of the panel casted to `DateTime` | `toDateTime(1415792726)` |
109+
| _$\_\_toTime_ | Replaced by the ending time of the range of the panel casted to `DateTime` | `toDateTime(1447328726)` |
110+
| _$\_\_fromTime_ms_ | Replaced by the starting time of the range of the panel casted to `DateTime64(3)` | `fromUnixTimestamp64Milli(1415792726123)` |
111+
| _$\_\_toTime_ms_ | Replaced by the ending time of the range of the panel casted to `DateTime64(3)` | `fromUnixTimestamp64Milli(1447328726456)` |
112+
| _$\_\_interval_s_ | Replaced by the interval in seconds | `20` |
113+
| _$\_\_timeInterval(columnName)_ | Replaced by a function calculating the interval based on window size in seconds, useful when grouping | `toStartOfInterval(toDateTime(column), INTERVAL 20 second)` |
114+
| _$\_\_timeInterval_ms(columnName)_ | Replaced by a function calculating the interval based on window size in milliseconds, useful when grouping | `toStartOfInterval(toDateTime64(column, 3), INTERVAL 20 millisecond)` |
115+
| _$\_\_conditionalAll(condition, $templateVar)_ | Replaced by the first parameter when the template variable in the second parameter does not select every value. Replaced by the 1=1 when the template variable selects every value. | `condition` or `1=1` |
116+
117+
The plugin also supports notation using braces {}. Use this notation when queries are needed inside parameters.

‎docs/sources/template-and-variables.md‎

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,22 @@
1+
---
2+
description: This document describes using templates and variables with the ClickHouse data source
3+
labels:
4+
products:
5+
- Grafana Cloud
6+
keywords:
7+
- data source
8+
menuTitle: ClickHouse templates and variables
9+
title: ClickHouse templates and variables
10+
weight: 20
11+
version: 0.1
12+
---
13+
14+
### Templates and variables
15+
16+
To add a new ClickHouse query variable, refer to [Add a query
17+
variable](https://grafana.com/docs/grafana/latest/variables/variable-types/add-query-variable/).
18+
19+
After creating a variable, you can use it in your ClickHouse queries by using
20+
[Variable syntax](https://grafana.com/docs/grafana/latest/variables/syntax/).
21+
For more information about variables, refer to [Templates and
22+
variables](https://grafana.com/docs/grafana/latest/variables/).

0 commit comments

Comments
 (0)