Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[#5511] Update playground doc to latest #5586

Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
110 changes: 100 additions & 10 deletions docs/how-to-use-the-playground.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,8 @@ Depending on your network and computer, startup time may take 3-5 minutes. Once

## Prerequisites

Install Git, Docker, Docker Compose.
Install Git (optional), Docker, Docker Compose.
Docker Desktop (or Orbstack) with Kubenetes enabled, and helm CLI are required if you use helm-chart to deploy services.

## System Resource Requirements

Expand All @@ -24,34 +25,110 @@ Install Git, Docker, Docker Compose.
The playground runs several services. The TCP ports used may clash with existing services you run, such as MySQL or Postgres.

| Docker container | Ports used |
|-----------------------|------------------------|
| --------------------- | ---------------------- |
| playground-gravitino | 8090 9001 |
| playground-hive | 3307 19000 19083 60070 |
| playground-mysql | 13306 |
| playground-postgresql | 15342 |
| playground-trino | 18080 |
| playground-jupyter | 18888 |
| playground-prometheus | 19090 |
| playground-grafana | 13000 |

## Playground usage

### Launch playground
### One curl command launch playground

```shell
/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/apache/gravitino-playground/HEAD/install.sh)"
```

### Use git to download and launch playground

```shell
git clone [email protected]:apache/gravitino-playground.git
cd gravitino-playground
./playground.sh start
```

### Check status
#### Docker

##### Start

```
./playground.sh docker start
```

##### Check status

```shell
./playground.sh docker status
```

##### Stop playground

```shell
./playground.sh docker stop
```

#### Kubernetes

Enable Kubernetes in Docker Desktop or Orbstack.

In Project root directory, execute this command:

```
helm upgrade --install gravitino-playground ./helm-chart/ --create-namespace --namespace gravitino-playground --set projectRoot=$(pwd)
```

##### Start

```
./playground.sh k8s start
```

##### Check status

```shell
./playground.sh status
./playground.sh k8s status
```
### Stop playground

##### Port Forwarding

To access pods or services at `localhost`, you needs to do these steps:

1. Log in to the Gravitino playground Trino pod using the following command:

```
TRINO_POD=$(kubectl get pods --namespace gravitino-playground -l app=trino -o jsonpath="{.items[0].metadata.name}")
kubectl exec $TRINO_POD -n gravitino-playground -it -- /bin/bash
```

2. Log in to the Gravitino playground Spark pod using the following command:

```
SPARK_POD=$(kubectl get pods --namespace gravitino-playground -l app=spark -o jsonpath="{.items[0].metadata.name}")
kubectl exec $SPARK_POD -n gravitino-playground -it -- /bin/bash
```

3. Port-forwarding Gravitino Service, so that you can access it at `localhost:8090`.

```
kubectl port-forward svc/gravitino -n gravitino-playground 8090:8090
```

4. Port-forwarding Jupyter Notebook Service, so that you can access it at `localhost:8888`.

```
kubectl port-forward svc/jupyternotebook -n gravitino-playground 8888:8888
```

##### Stop playground

```shell
./playground.sh stop
./playground.sh k8s stop
```

## Using Apache Gravitino with Trino SQL
## Experiencing Apache Gravitino with Trino SQL

### Using Trino CLI in Docker Container

Expand Down Expand Up @@ -86,9 +163,17 @@ docker exec -it playground-spark bash
2. Open the Spark SQL client in the container.

```shell
spark@container_id:/$ cd /opt/spark && /bin/bash bin/spark-sql
spark@container_id:/$ cd /opt/spark && /bin/bash bin/spark-sql
```

## Monitoring Gravitino

1. Open the Grafana in the browser at [http://localhost:13000](http://localhost:13000).

2. In the navigation menu, click **Dashboards** -> **Gravitino Playground**.

3. Experiment with the default template.

## Example

### Simple Trino queries
Expand Down Expand Up @@ -193,6 +278,8 @@ INSERT OVERWRITE TABLE employees PARTITION(department='Marketing') VALUES (3, 'M
SELECT * FROM catalog_hive.product.employees WHERE department = 'Engineering';
```

The demo is located in the `jupyter` folder, you can open the `gravitino-spark-trino-example.ipynb`
demo via Jupyter Notebook by [http://localhost:18888](http://localhost:18888).

### Using Apache Iceberg REST service

Expand Down Expand Up @@ -249,6 +336,9 @@ union
select * from catalog_iceberg.sales.customers;
```

The demo is located in the `jupyter` folder, you can open the `gravitino-spark-trino-example.ipynb`
demo via Jupyter Notebook by [http://localhost:18888](http://localhost:18888).

### Using Gravitino with LlamaIndex

The Gravitino playground also provides a simple RAG demo with LlamaIndex. This demo will show you the
Expand Down
Loading