-
Notifications
You must be signed in to change notification settings - Fork 336
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Browse files
Browse the repository at this point in the history
### What changes were proposed in this pull request? Sync the playground doc ### Why are the changes needed? Fix: #5511 ### Does this PR introduce _any_ user-facing change? N/A ### How was this patch tested? N/A
- Loading branch information
1 parent
6be5fc2
commit 8a51a6d
Showing
1 changed file
with
100 additions
and
10 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -13,7 +13,8 @@ Depending on your network and computer, startup time may take 3-5 minutes. Once | |
|
||
## Prerequisites | ||
|
||
Install Git, Docker, Docker Compose. | ||
Install Git (optional), Docker, Docker Compose. | ||
Docker Desktop (or Orbstack) with Kubenetes enabled, and helm CLI are required if you use helm-chart to deploy services. | ||
|
||
## System Resource Requirements | ||
|
||
|
@@ -24,34 +25,110 @@ Install Git, Docker, Docker Compose. | |
The playground runs several services. The TCP ports used may clash with existing services you run, such as MySQL or Postgres. | ||
|
||
| Docker container | Ports used | | ||
|-----------------------|------------------------| | ||
| --------------------- | ---------------------- | | ||
| playground-gravitino | 8090 9001 | | ||
| playground-hive | 3307 19000 19083 60070 | | ||
| playground-mysql | 13306 | | ||
| playground-postgresql | 15342 | | ||
| playground-trino | 18080 | | ||
| playground-jupyter | 18888 | | ||
| playground-prometheus | 19090 | | ||
| playground-grafana | 13000 | | ||
|
||
## Playground usage | ||
|
||
### Launch playground | ||
### One curl command launch playground | ||
|
||
```shell | ||
/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/apache/gravitino-playground/HEAD/install.sh)" | ||
``` | ||
|
||
### Use git to download and launch playground | ||
|
||
```shell | ||
git clone [email protected]:apache/gravitino-playground.git | ||
cd gravitino-playground | ||
./playground.sh start | ||
``` | ||
|
||
### Check status | ||
#### Docker | ||
|
||
##### Start | ||
|
||
``` | ||
./playground.sh docker start | ||
``` | ||
|
||
##### Check status | ||
|
||
```shell | ||
./playground.sh docker status | ||
``` | ||
|
||
##### Stop playground | ||
|
||
```shell | ||
./playground.sh docker stop | ||
``` | ||
|
||
#### Kubernetes | ||
|
||
Enable Kubernetes in Docker Desktop or Orbstack. | ||
|
||
In Project root directory, execute this command: | ||
|
||
``` | ||
helm upgrade --install gravitino-playground ./helm-chart/ --create-namespace --namespace gravitino-playground --set projectRoot=$(pwd) | ||
``` | ||
|
||
##### Start | ||
|
||
``` | ||
./playground.sh k8s start | ||
``` | ||
|
||
##### Check status | ||
|
||
```shell | ||
./playground.sh status | ||
./playground.sh k8s status | ||
``` | ||
### Stop playground | ||
|
||
##### Port Forwarding | ||
|
||
To access pods or services at `localhost`, you needs to do these steps: | ||
|
||
1. Log in to the Gravitino playground Trino pod using the following command: | ||
|
||
``` | ||
TRINO_POD=$(kubectl get pods --namespace gravitino-playground -l app=trino -o jsonpath="{.items[0].metadata.name}") | ||
kubectl exec $TRINO_POD -n gravitino-playground -it -- /bin/bash | ||
``` | ||
|
||
2. Log in to the Gravitino playground Spark pod using the following command: | ||
|
||
``` | ||
SPARK_POD=$(kubectl get pods --namespace gravitino-playground -l app=spark -o jsonpath="{.items[0].metadata.name}") | ||
kubectl exec $SPARK_POD -n gravitino-playground -it -- /bin/bash | ||
``` | ||
|
||
3. Port-forwarding Gravitino Service, so that you can access it at `localhost:8090`. | ||
|
||
``` | ||
kubectl port-forward svc/gravitino -n gravitino-playground 8090:8090 | ||
``` | ||
|
||
4. Port-forwarding Jupyter Notebook Service, so that you can access it at `localhost:8888`. | ||
|
||
``` | ||
kubectl port-forward svc/jupyternotebook -n gravitino-playground 8888:8888 | ||
``` | ||
|
||
##### Stop playground | ||
|
||
```shell | ||
./playground.sh stop | ||
./playground.sh k8s stop | ||
``` | ||
|
||
## Using Apache Gravitino with Trino SQL | ||
## Experiencing Apache Gravitino with Trino SQL | ||
|
||
### Using Trino CLI in Docker Container | ||
|
||
|
@@ -86,9 +163,17 @@ docker exec -it playground-spark bash | |
2. Open the Spark SQL client in the container. | ||
```shell | ||
spark@container_id:/$ cd /opt/spark && /bin/bash bin/spark-sql | ||
spark@container_id:/$ cd /opt/spark && /bin/bash bin/spark-sql | ||
``` | ||
|
||
## Monitoring Gravitino | ||
|
||
1. Open the Grafana in the browser at [http://localhost:13000](http://localhost:13000). | ||
|
||
2. In the navigation menu, click **Dashboards** -> **Gravitino Playground**. | ||
|
||
3. Experiment with the default template. | ||
|
||
## Example | ||
|
||
### Simple Trino queries | ||
|
@@ -193,6 +278,8 @@ INSERT OVERWRITE TABLE employees PARTITION(department='Marketing') VALUES (3, 'M | |
SELECT * FROM catalog_hive.product.employees WHERE department = 'Engineering'; | ||
``` | ||
|
||
The demo is located in the `jupyter` folder, you can open the `gravitino-spark-trino-example.ipynb` | ||
demo via Jupyter Notebook by [http://localhost:18888](http://localhost:18888). | ||
|
||
### Using Apache Iceberg REST service | ||
|
||
|
@@ -249,6 +336,9 @@ union | |
select * from catalog_iceberg.sales.customers; | ||
``` | ||
|
||
The demo is located in the `jupyter` folder, you can open the `gravitino-spark-trino-example.ipynb` | ||
demo via Jupyter Notebook by [http://localhost:18888](http://localhost:18888). | ||
|
||
### Using Gravitino with LlamaIndex | ||
|
||
The Gravitino playground also provides a simple RAG demo with LlamaIndex. This demo will show you the | ||
|