Skip to content

[apiserver] Use ClusterIP instead of NodePort for KubeRay API server service #3708

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 13 commits into from
Jun 1, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion apiserver/Autoscaling.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,9 @@ run an example.

## Setup

Refer to the [README](README.md) for setting up the KubeRay operator and APIServer.
Refer to the [Install with Helm](README.md#install-with-helm) section in the README for
setting up the KubeRay operator and APIServer, and port-forward the HTTP endpoint to local
port 31888.

## Example

Expand Down
4 changes: 3 additions & 1 deletion apiserver/CreatingServe.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,9 @@ creating the cluster:

## Setup

Refer to the [README](README.md) for setting up the KubeRay operator and APIServer.
Refer to the [Install with Helm](README.md#install-with-helm) section in the README for
setting up the KubeRay operator and APIServer, and port-forward the HTTP endpoint to local
port 31888.

## Example

Expand Down
4 changes: 3 additions & 1 deletion apiserver/HACluster.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,9 @@ We will provide a detailed example on how to create this highly available APISer

### Setup Ray Operator and APIServer

Refer to the [README](README.md) for setting up the KubeRay operator and APIServer.
Refer to the [Install with Helm](README.md#install-with-helm) section in the README for
setting up the KubeRay operator and APIServer, and port-forward the HTTP endpoint to local
port 31888.

## Example

Expand Down
4 changes: 3 additions & 1 deletion apiserver/JobSubmission.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,9 @@ kubectl delete raycluster --all

### Deploy KubeRay operator and APIServer

Refer to [README](README.md) for setting up the KubeRay operator and APIServer.
Refer to the [Install with Helm](README.md#install-with-helm) section in the README for
setting up the KubeRay operator and APIServer, and port-forward the HTTP endpoint to local
port 31888.

### Install ConfigMap

Expand Down
6 changes: 4 additions & 2 deletions apiserver/Monitoring.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,8 +14,10 @@ If this flag is enabled, an `http` port at the `/metrics` endpoint provides APIS

### Deploy KubeRay operator and APIServer

Refer to [README](README.md) for setting up the KubeRay operator and APIServer. This will
set the flag `collectMetricsFlag` to `true`, which enables metrics collection.
Refer to the [Install with Helm](README.md#install-with-helm) section in the README for
setting up the KubeRay operator and APIServer, and port-forward the HTTP endpoint to local
port 31888. This will set the flag `collectMetricsFlag` to `true`, which enables metrics
collection.

> [!IMPORTANT]
> All the following guidance requires you to switch your working directory to the KubeRay project root.
Expand Down
116 changes: 41 additions & 75 deletions apiserver/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,41 +9,21 @@ The KubeRay APIServer is an optional component that provides a layer of simplifi

## Installation

### Start a local apiserver

You can build and start the APIServer from scratch in your local environment with a single command. This will deploy all the necessary components to a local kind cluster.

```sh
make start-local-apiserver
```
### Install with Helm

The APIServer supports HTTP requests, so you can easily verify its successful startup by issuing two simple curl commands.
Ensure that the version of Helm is v3+. Currently, [existing CI tests](https://github.com/ray-project/kuberay/blob/master/.github/workflows/helm-lint.yaml) are based on Helm v3.4.1 and v3.9.4.

```sh
# Create complete template.
curl --silent -X 'POST' \
'http://localhost:31888/apis/v1/namespaces/ray-system/compute_templates' \
-H 'accept: application/json' \
-H 'Content-Type: application/json' \
-d '{
"name": "default-template",
"namespace": "ray-system",
"cpu": 2,
"memory": 4
}'

# Check whether compute template is created successfully.
curl --silent -X 'GET' \
'http://localhost:31888/apis/v1/namespaces/ray-system/compute_templates' \
-H 'accept: application/json'
helm version
```

### Install with Helm
#### Create a Kubernetes cluster

Ensure that the version of Helm is v3+. Currently, [existing CI tests](https://github.com/ray-project/kuberay/blob/master/.github/workflows/helm-lint.yaml) are based on Helm v3.4.1 and v3.9.4.
Create a local Kubernetes cluster using [Kind](https://kind.sigs.k8s.io/). If you already
have a Kubernetes cluster, you can skip this step.

```sh
helm version
kind create cluster --image=kindest/node:v1.26.0
```

#### Install KubeRay Operator
Expand All @@ -52,66 +32,52 @@ Refer to [this document](https://docs.ray.io/en/master/cluster/kubernetes/gettin

#### Install KubeRay APIServer

- Install a stable version via the Helm repository (supports KubeRay v0.4.0+ only)
Refer to [this document](../helm-chart/kuberay-apiserver/README.md#without-security-proxy) to install the latest
stable KubeRay operator and APIServer (without the security proxy) from the Helm
repository.

```sh
# Install the KubeRay helm repo
helm repo add kuberay https://ray-project.github.io/kuberay-helm/
> [!IMPORTANT]
> If you install APIServer with security proxy, you may receive an "Unauthorized" error
> when making a request. Please add an authorization header to the request: `-H 'Authorization: 12345'`
> or install the APIServer without a security proxy.

# Install KubeRay APIServer.
helm install kuberay-apiserver kuberay/kuberay-apiserver
#### Port-forwarding the APIServer service

# Check the KubeRay APIServer Pod in `default` namespace
kubectl get pods
# NAME READY STATUS RESTARTS AGE
# kuberay-apiserver-857869f665-b94px 1/1 Running 0 86m
# kuberay-operator-7456c6b69b-t6pt7 1/1 Running 0 172m
```
Use the following command for port-forwarding to access the APIServer through port 31888:

- Install the nightly version

```sh
# Step1: Clone KubeRay repository

# Step2: Move to `helm-chart/kuberay-apiserver`
cd helm-chart/kuberay-apiserver

# Step3: Install KubeRay APIServer
helm install kuberay-apiserver .
```

- Install the current working branch version

```sh
# Step1: Clone KubeRay repository

# Step2: Change directory to the api server
cd apiserver

# Step3: Build docker image, create a local kind cluster and deploy api server (using helm)
make docker-image cluster load-image deploy
```
```sh
kubectl port-forward service/kuberay-apiserver-service 31888:8888
```

#### List the chart
### For Development: Start a Local APIServer

To list the deployed charts:
You can build and start the APIServer from scratch in your local environment with a single command. This will deploy all the necessary components to a local kind cluster.

```sh
helm ls
# NAME NAMESPACE REVISION UPDATED STATUS CHART APP VERSION
# kuberay-apiserver default 1 2023-09-25 10:42:34.267328 +0300 EEST deployed kuberay-apiserver-1.0.0
# kuberay-operator default 1 2023-09-25 10:41:48.355831 +0300 EEST deployed kuberay-operator-1.0.0
make start-local-apiserver
```

#### Uninstall the Chart
### Verify the installation

The APIServer supports HTTP requests, so you can easily verify its successful startup by issuing two simple curl commands.

```sh
# Uninstall the `kuberay-apiserver` release
helm uninstall kuberay-apiserver
# Create complete template.
curl --silent -X 'POST' \
'http://localhost:31888/apis/v1/namespaces/ray-system/compute_templates' \
-H 'accept: application/json' \
-H 'Content-Type: application/json' \
-d '{
"name": "default-template",
"namespace": "ray-system",
"cpu": 2,
"memory": 4
}'

# The KubeRay APIServer Pod should be removed.
kubectl get pods
# No resources found in default namespace.
# Check whether compute template is created successfully.
curl --silent -X 'GET' \
'http://localhost:31888/apis/v1/namespaces/ray-system/compute_templates' \
-H 'accept: application/json'
```

## Usage
Expand All @@ -122,7 +88,7 @@ After deployment, you can use the `{{baseUrl}}` to access the service. Refer to

- for ingress access, you will need to create your own ingress

Details of the request parameters can be found in [KubeRay Swagger](https://github.com/ray-project/kuberay/tree/master/proto/swagger). This document only provides basic examples.
Details of the request parameters can be found in [KubeRay Swagger](https://github.com/ray-project/kuberay/tree/master/proto/swagger). This README provides only basic examples.

### Setup a smoke test

Expand Down
60 changes: 43 additions & 17 deletions helm-chart/kuberay-apiserver/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,30 +13,56 @@ helm version

## Install KubeRay API Server

* Install a stable version via Helm repository (only supports KubeRay v0.4.0+)
### Without security proxy

```sh
helm repo add kuberay https://ray-project.github.io/kuberay-helm/
- Install a stable version via Helm repository

# Install the KubeRay API Server at Version v1.1.0.
helm install kuberay-apiserver kuberay/kuberay-apiserver --version 1.1.0
```sh
helm repo add kuberay https://ray-project.github.io/kuberay-helm/
helm repo update
# Install KubeRay APIServer without security proxy
helm install kuberay-apiserver kuberay/kuberay-apiserver --version 1.4.0 --set security=null
```

- Install the nightly version

```sh
# Step1: Clone KubeRay repository

# Step2: Navigate to `helm-chart/kuberay-apiserver`
cd helm-chart/kuberay-apiserver

# Check that the KubeRay API Server is running in the "default" namespaces.
kubectl get pods
# NAME READY STATUS RESTARTS AGE
# kuberay-apiserver-xxxxxx 1/1 Running 0 17s
```
# Step3: Install the KubeRay apiserver
helm install kuberay-apiserver . --set security=null
```

* Install the nightly version
### With security proxy

```sh
# Step1: Clone KubeRay repository
- Install a stable version via Helm repository

# Step2: Move to `helm-chart/kuberay-apiserver`
```sh
helm repo add kuberay https://ray-project.github.io/kuberay-helm/
helm repo update
# Install KubeRay APIServer
helm install kuberay-apiserver kuberay/kuberay-apiserver --version 1.4.0
```

- Install the nightly version

```sh
# Step1: Clone KubeRay repository

# Step2: Navigate to `helm-chart/kuberay-apiserver`
cd helm-chart/kuberay-apiserver

# Step3: Install the KubeRay apiserver
helm install kuberay-apiserver .
```

# Step3: Install the KubeRay apiserver
helm install kuberay-apiserver .
```
> [!IMPORTANT]
> If you receive an "Unauthorized" error when making a request, please add an
> authorization header to the request: `-H 'Authorization: 12345'` or install the
> APIServer without a security proxy.

## List the chart

Expand Down
19 changes: 4 additions & 15 deletions helm-chart/kuberay-apiserver/values.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -66,28 +66,17 @@ tolerations: []

# Only one service type needs to be picked
service:
# Nodeport service
type: NodePort
# ClusterIP service
type: ClusterIP
ports:
- name: http
protocol: TCP
port: 8888
targetPort: 8888
nodePort: 31888
- name: rpc
protocol: TCP
port: 8887
targetPort: 8887
nodePort: 31887
# ClusterIP service
# type: ClusterIP
# ports:
# - name: http
# protocol: TCP
# port: 8888
# targetPort: 8888
# - name: rpc
# protocol: TCP
# port: 8887
# targetPort: 8887

# You can only enable an ingress or route, if you are using OpenShift cluster
# Also note that in order to enable ingress or route you need to use ClusterIP service
Expand Down
Loading