Skip to content

Commit d1b07df

Browse files
machichimamachichima
authored
[DOCS] KubeRay APIServer V2 document (#3594)
1 parent e80e84b commit d1b07df

File tree

6 files changed

+481
-0
lines changed

6 files changed

+481
-0
lines changed

‎apiserversdk/Installation.md

Lines changed: 86 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,86 @@
1+
# KubeRay APIServer Installation
2+
3+
## Step 1: Create a Kubernetes cluster
4+
5+
This step creates a local Kubernetes cluster using [Kind](https://kind.sigs.k8s.io/). If you already have a Kubernetes
6+
cluster, you can skip this step.
7+
8+
```sh
9+
kind create cluster --image=kindest/node:v1.26.0
10+
```
11+
12+
## Step 2: Deploy a KubeRay operator
13+
14+
Follow [this
15+
document](https://docs.ray.io/en/latest/cluster/kubernetes/getting-started/kuberay-operator-installation.html#kuberay-operator-deploy)
16+
to install the latest stable KubeRay operator from the Helm repository.
17+
18+
## Step 3: Install APIServer with Helm
19+
20+
### Without security proxy
21+
22+
- Install a stable version via Helm repository
23+
24+
```sh
25+
helm repo add kuberay https://ray-project.github.io/kuberay-helm/
26+
helm repo update
27+
# Install KubeRay APIServer without security proxy
28+
helm install kuberay-apiserver kuberay/kuberay-apiserver --version 1.4.0 --set security=null
29+
```
30+
31+
- Install the nightly version
32+
33+
```sh
34+
# Step1: Clone KubeRay repository
35+
36+
# Step2: Navigate to `helm-chart/kuberay-apiserver`
37+
cd helm-chart/kuberay-apiserver
38+
39+
# Step3: Install the KubeRay apiserver
40+
helm install kuberay-apiserver . --set security=null
41+
```
42+
43+
### With security proxy
44+
45+
- Install a stable version via Helm repository
46+
47+
```sh
48+
helm repo add kuberay https://ray-project.github.io/kuberay-helm/
49+
helm repo update
50+
# Install KubeRay APIServer
51+
helm install kuberay-apiserver kuberay/kuberay-apiserver --version 1.4.0
52+
```
53+
54+
- Install the nightly version
55+
56+
```sh
57+
# Step1: Clone KubeRay repository
58+
59+
# Step2: Navigate to `helm-chart/kuberay-apiserver`
60+
cd helm-chart/kuberay-apiserver
61+
62+
# Step3: Install the KubeRay apiserver
63+
helm install kuberay-apiserver .
64+
```
65+
66+
> [!IMPORTANT]
67+
> You may receive an "Unauthorized" error when making a request if you install the
68+
> APIServer with security proxy. Please try the APIServer without a security proxy.
69+
70+
## Step 4: Port-forwarding the APIServer service
71+
72+
Use the following command for port-forwarding to access the APIServer through port 31888:
73+
74+
```sh
75+
kubectl port-forward service/kuberay-apiserver-service 31888:8888
76+
```
77+
78+
## Step 5: Validate installation
79+
80+
Check that the KubeRay APIServer is running in the "default" namespace.
81+
82+
```sh
83+
kubectl get pods
84+
# NAME READY STATUS RESTARTS AGE
85+
# kuberay-apiserver-xxxxxx 1/1 Running 0 17s
86+
```

‎apiserversdk/README.md

Lines changed: 77 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,77 @@
1+
# KubeRay APIServer
2+
3+
The KubeRay APIServer provides an HTTP proxy to the Kubernetes APIServer with the same
4+
interface. Users can directly use the Kubernetes OpenAPI Spec and KubeRay CRD to create, query,
5+
update, and delete Ray resources. It contains the following highlighted features:
6+
7+
1. Compatibility with existing Kubernetes clients and API interfaces, allowing users to use
8+
existing Kubernetes clients to interact with the proxy provided by the APIServer.
9+
2. Provides the APIServer as a Go library for users to build their proxies with custom HTTP middleware functions.
10+
11+
## When to use APIServer
12+
13+
Consider using the APIServer if:
14+
15+
- You want to manage Ray clusters in Kubernetes via HTTP/REST (e.g., from a UI, SDK, or CLI).
16+
- You want to create templates or default values to simplify configuration setup.
17+
18+
## Installation
19+
20+
Please follow the [Installation](./Installation.md) guide to install the APIServer, and
21+
port-forward the HTTP endpoint to local port 31888.
22+
23+
## Quick Start
24+
25+
- [RayCluster Quickstart](./docs/raycluster-quickstart.md)
26+
- [RayJob Quickstart](./docs/rayjob-quickstart.md)
27+
- [RayService Quickstart](./docs/rayservice-quickstart.md)
28+
29+
## Usage
30+
31+
The KubeRay APIServer exposes a RESTful API that mirrors the Kubernetes APIServer. You
32+
can interact with it using Kubernetes-style endpoints and request patterns to create,
33+
retrieve, update, and delete custom resources such as `RayCluster` and `RayJob`.
34+
35+
### API Structure
36+
37+
The KubeRay API follows standard Kubernetes conventions. The structure of the endpoint
38+
path depends on whether you are interacting with custom resources (e.g., `RayCluster`) or
39+
core Kubernetes resources.
40+
41+
#### Custom Resources
42+
43+
For custom resources defined by CRDs (e.g., `RayCluster`, `RayJob`, etc.), the endpoint format is:
44+
45+
```sh
46+
<baseURL>/apis/ray.io/v1/namespaces/<namespace>/<resourceType>/<resourceName>
47+
```
48+
49+
- `namespace`: Your target Kubernetes namespace (e.g., `default`)
50+
- `resourceType`: Custom resource type (e.g., `rayclusters`, `rayjobs`, `rayservices`)
51+
- `resourceName`: Name of the resource.
52+
53+
### Label and Field Selectors
54+
55+
When listing resources (using `GET` on a collection endpoint), you can filter results using selectors:
56+
57+
- Label selector: filters resources by their labels.
58+
59+
```sh
60+
# Get all RayClusters with the label key=value
61+
GET /apis/ray.io/v1/namespaces/default/rayclusters?labelSelector=key%3Dvalue
62+
```
63+
64+
- Field selector: Filters resources based on the value of their resource
65+
fields (e.g., `metadata.name`, `status.phase`).
66+
67+
```sh
68+
# Retrieve the RayCluster where the name is raycluster-prod
69+
GET /apis/ray.io/v1/namespaces/default/rayclusters?fieldSelector=metadata.name%3Draycluster-prod
70+
```
71+
72+
- Combined selectors: You can combine label and field selectors using `&`
73+
74+
```sh
75+
# Get the RayCluster named raycluster-prod that also has the label env=prod.
76+
GET /apis/ray.io/v1/namespaces/default/rayclusters?labelSelector=env%3Dprod&fieldSelector=metadata.name%3Draycluster-prod
77+
```

‎apiserversdk/Troubleshooting.md

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
# Troubleshooting
2+
3+
This page contains common issues and their solutions for using the APIServer.
4+
5+
## Getting "Unauthorized" when sending a request
6+
7+
If you receive an "Unauthorized" error when making a request, and you installed APIServer
8+
using the Helm chart, please try the following solution
9+
10+
- Install the APIServer without a security proxy. Refer to the [Installation](./Installation.md) guide.

‎apiserversdk/docs/raycluster-quickstart.md

Lines changed: 97 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,97 @@
1+
# RayCluster QuickStart
2+
3+
This document focuses on explaining how to manage and interact with RayCluster using the
4+
KubeRay APIServer. For a detailed introduction and more advanced usage with Kubernetes,
5+
please refer to [this
6+
guide](https://docs.ray.io/en/latest/cluster/kubernetes/getting-started/raycluster-quick-start.html).
7+
8+
> [!IMPORTANT]
9+
> If you encounter any problems while following this guide, please refer to the [Troubleshooting](../Troubleshooting.md)
10+
> page.
11+
12+
## Preparation
13+
14+
- Install [Helm](https://helm.sh/docs/intro/install/) (>= v3.4),
15+
[Kind](https://kind.sigs.k8s.io/docs/user/quick-start/#installation), and
16+
[Docker](https://docs.docker.com/engine/install/).
17+
- KubeRay v1.4.0 or higher and Ray 2.41.0.
18+
- Make sure your Kubernetes cluster has at least 4 CPU and 4 GB RAM.
19+
20+
## Step 1: Create a Kubernetes cluster
21+
22+
This step creates a local Kubernetes cluster using [Kind](https://kind.sigs.k8s.io/). If you already have a Kubernetes
23+
cluster, you can skip this step.
24+
25+
```sh
26+
kind create cluster --image=kindest/node:v1.26.0
27+
```
28+
29+
## Step 2: Install KubeRay operator and APIServer
30+
31+
Follow the [Installation Guide](../Installation.md) to install the latest stable KubeRay
32+
operator and APIServer (without the security proxy) from the Helm repository, and
33+
port-forward the HTTP endpoint to local port 31888.
34+
35+
## Step 3: Deploy a RayCluster custom resource
36+
37+
Once the KubeRay operator is running, you are ready to deploy a RayCluster. While we are using APIServer, we can do this
38+
with curl. The following command will create a RayCluster CR named `raycluster-kuberay` in your current cluster:
39+
40+
```sh
41+
curl -s https://raw.githubusercontent.com/ray-project/kuberay/master/ray-operator/config/samples/ray-cluster.sample.yaml | \
42+
curl -X POST http://localhost:31888/apis/ray.io/v1/namespaces/default/rayclusters \
43+
-H "Content-Type: application/yaml" \
44+
--data-binary @-
45+
```
46+
47+
Once the RayCluster CR has been created, you can view its details by executing the following command:
48+
49+
```sh
50+
curl http://localhost:31888/apis/ray.io/v1/namespaces/default/rayclusters/raycluster-kuberay
51+
```
52+
53+
## Step 4: Modify Created RayCluster
54+
55+
To modify the created RayCluster, we can use the `PATCH` method of the KubeRay APIServer.
56+
The following command adds an `annotation` to the raycluster-kuberay resource:
57+
58+
```sh
59+
curl -X PATCH 'http://localhost:31888/apis/ray.io/v1/namespaces/default/rayclusters/raycluster-kuberay' \
60+
-H 'Content-Type: application/merge-patch+json' \
61+
--data '{
62+
"metadata": {
63+
"annotations": {
64+
"example.com/purpose": "model-training"
65+
}
66+
}
67+
}'
68+
```
69+
70+
You can verify if the `annotation` is added with the following command. You should see the
71+
annotations you added in the output:
72+
73+
```sh
74+
curl -s http://localhost:31888/apis/ray.io/v1/namespaces/default/rayclusters/raycluster-kuberay \
75+
| jq '.metadata.annotations'
76+
```
77+
78+
## Step 5: Delete the RayCluster
79+
80+
To delete the RayCluster with KubeRay APIServer, execute the following command. The `raycluster-kuberay` is the name of
81+
the RayCluster that we created earlier:
82+
83+
```sh
84+
curl -X DELETE 'localhost:31888/apis/ray.io/v1/namespaces/default/rayclusters/raycluster-kuberay'
85+
```
86+
87+
You can then verify if the RayCluster is removed. The following command should return 404:
88+
89+
```sh
90+
curl -s http://localhost:31888/apis/ray.io/v1/namespaces/default/rayclusters/raycluster-kuberay
91+
```
92+
93+
## Clean up
94+
95+
```sh
96+
kind delete cluster
97+
```

‎apiserversdk/docs/rayjob-quickstart.md

Lines changed: 74 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,74 @@
1+
# RayJob QuickStart
2+
3+
RayJob automatically creates the RayCluster, submits the job when ready, and can
4+
optionally delete the RayCluster after the job finishes. This document focus on explaining
5+
how to manage and interact with RayJob using the KubeRay APIServer. For detailed
6+
introduction and more advanced usage with Kubernetes, please refer to [this
7+
guide](https://docs.ray.io/en/latest/cluster/kubernetes/getting-started/rayjob-quick-start.html).
8+
9+
> [!IMPORTANT]
10+
> If you encounter any problems while following this guide, please refer to the [Troubleshooting](../Troubleshooting.md)
11+
> page.
12+
13+
## Preparation
14+
15+
- Install [Helm](https://helm.sh/docs/intro/install/) (>= v3.4),
16+
[Kind](https://kind.sigs.k8s.io/docs/user/quick-start/#installation), and
17+
[Docker](https://docs.docker.com/engine/install/).
18+
- KubeRay v1.4.0 or higher and Ray 2.41.0.
19+
20+
## Step 1: Create a Kubernetes cluster
21+
22+
This step creates a local Kubernetes cluster using [Kind](https://kind.sigs.k8s.io/). If you already have a Kubernetes
23+
cluster, you can skip this step.
24+
25+
```sh
26+
kind create cluster --image=kindest/node:v1.26.0
27+
```
28+
29+
## Step 2: Install KubeRay operator and APIServer
30+
31+
Follow the [Installation Guide](../Installation.md) to install the latest stable KubeRay
32+
operator and APIServer (without the security proxy) from the Helm repository, and
33+
port-forward the HTTP endpoint to local port 31888.
34+
35+
## Step 3: Install a RayJob
36+
37+
Once the KubeRay operator is running, we can install a RayJob through APIServer with
38+
following command. This will create a RayJob named `rayjob-interactive-mode`:
39+
40+
```sh
41+
curl -s https://raw.githubusercontent.com/ray-project/kuberay/master/ray-operator/config/samples/ray-job.interactive-mode.yaml \
42+
| curl -X POST http://localhost:31888/apis/ray.io/v1/namespaces/default/rayjobs \
43+
-H "Content-Type: application/yaml" \
44+
--data-binary @-
45+
```
46+
47+
## Step 4: Check RayJob Status
48+
49+
You can check the detail of the submitted RayJob by executing following command:
50+
51+
```sh
52+
curl -s http://localhost:31888/apis/ray.io/v1/namespaces/default/rayjobs/rayjob-interactive-mode
53+
```
54+
55+
## Step 5: Delete the RayJob
56+
57+
To delete the RayJob with KubeRay APIServer, execute the following command. The `rayjob-interactive-mode` is the name of
58+
the RayJob we created.
59+
60+
```sh
61+
curl -X DELETE 'localhost:31888/apis/ray.io/v1/namespaces/default/rayjobs/rayjob-interactive-mode'
62+
```
63+
64+
You can then verify if the RayJob is removed. The following command should return 404:
65+
66+
```sh
67+
curl -s http://localhost:31888/apis/ray.io/v1/namespaces/default/rayjobs/rayjob-interactive-mode
68+
```
69+
70+
## Clean up
71+
72+
```sh
73+
kind delete cluster
74+
```

0 commit comments

Comments
 (0)