You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
#### English | [简体中文](https://github.com/opencurve/curve-operator/tree/master/docs/readme_cn.md)
2
+
1
3
# Curve-Operator
2
4
3
5
## What's curve-operator
4
6
5
-
The curve-operator project helps deploy a Curve cluster orchestrated by Kubernetes.(Currently only supports CurveBS cluster deployment)
7
+
curve-operator manages [Curve](https://github.com/opencurve/curve) cluster on [Kuberentes](https://kubernetes.io/docs/home/). It make Curve a truly cloud-native distributed storage system.
* Three-nodes cluster(stand-alone deployment is not supported at persent)
13
+
### 0.Prerequisite
13
14
14
-
## Install and Deploy
15
+
* Kubernetes 1.19,1.20
15
16
16
-
### 1.Install Operator
17
+
### 1.Install Operator
17
18
18
19
The first step is to install the Curve Operator.
19
20
@@ -23,126 +24,35 @@ $ cd curve-operator
23
24
$ kubectl apply -f config/deploy/
24
25
```
25
26
26
-
verify the curve-operator is in the `Running` state in `curve` namespace.
27
+
verify the curve-operator on the `Running` state in `curve` namespace.
27
28
28
29
```shell
29
30
$ kubectl get pod -n curve
30
-
31
31
NAME READY STATUS RESTARTS AGE
32
32
curve-operator-69bc69c75d-jfsjg 1/1 Running 0 7s
33
33
```
34
34
35
-
### 2. Deploy Curve cluster
35
+
### 2. Deploy Curve Cluster
36
36
37
-
#### Modify the cluster Declarative yaml file
37
+
Operator deploys Curve cluster based on declarative API. You can get and modify customized the cluster yaml file in [config/sample.](https://github.com/opencurve/curve-operator/tree/master/config/samples)
# Curve operator is deployed in this namespace,Do not modify if not necessary
50
-
namespace: curve
51
-
spec:
52
-
# The container image used to launch the Curve daemon pods(etcd, mds, chunkserver, snapshotclone).
53
-
# v1.2 is Pacific and v1.3 is not tested.
54
-
curveVersion:
55
-
image: opencurvedocker/curvebs:v1.2.6
56
-
# Container image pull policy,
57
-
# By default the pull policy of all containers in that pod will be set to IfNotPresent if it is not explicitly specified and no modification necessary.
58
-
imagePullPolicy: IfNotPresent
59
-
# The K8s cluster nodes name in cluster that prepare to deploy Curve daemon pods(etcd, mds, snapshotclone).
60
-
# For stand-alone deploy, set one node here and see bscluster-onehost.yaml
61
-
# - node1 -> etcd-a, mds-a, snapshotclone-a
62
-
# - node2 -> etcd-b, mds-b, snapshotclone-b
63
-
# - node3 -> etcd-c, mds-c, snapshotclone-c
64
-
nodes:
65
-
- curve-operator-node1
66
-
- curve-operator-node2
67
-
- curve-operator-node3
68
-
# hostDataDir where data files and log files will be persisted on host machine. Must be specified.
69
-
# If you reinstall the cluster, make surce that you delete this directory from each host.
70
-
hostDataDir: /curvebs
71
-
etcd:
72
-
# Port for listening to partner communication.
73
-
# Etcd member accept incoming requests from its peers on a specific scheme://IP:port combination and the IP is host ip because we use hostnetwork:true.
74
-
peerPort: 23800
75
-
# clientPort for listening server port.
76
-
clientPort: 23790
77
-
mds:
78
-
port: 6700
79
-
dummyPort: 7700
80
-
storage:
81
-
# useSelectedNodes is to control whether to use individual nodes and their configured devices can be specified as well.
82
-
# This field is not implemented at present and is must set false here.
83
-
# You can refer following selectoedNodes setting commented if the function is completed later.
84
-
# But the func is not implemented yet.
85
-
useSelectedNodes: false
86
-
# The hosts specified to deployment chunkserver as storage resource.
87
-
# And you can configure the same nodes above configure that deploy etcd, mds and snapshotclone service.
88
-
nodes:
89
-
- curve-operator-node1
90
-
- curve-operator-node2
91
-
- curve-operator-node3
92
-
port: 8200
93
-
copySets: 100
94
-
# Make sure the devices configured are available on hosts above.
95
-
devices:
96
-
- name: /dev/vdc
97
-
mountPath: /data/chunkserver0
98
-
percentage: 80
99
-
# Not implement yet
100
-
#selectedNodes:
101
-
#- node: curve-operator-node1
102
-
# - devices:
103
-
# name: /dev/vdd
104
-
# mountPath: /data/chunkserver1
105
-
# percentage: 90
106
-
# - devices:
107
-
# name: /dev/vdf
108
-
# mountPath: /data/chunkserver2
109
-
# percentage: 80
110
-
#- node: curve-operator-node2
111
-
# - devices:
112
-
# name:
113
-
# mountPath:
114
-
# percentage:
115
-
snapShotClone:
116
-
# set false if there is no S3 service available temporarily or don't need to use the snapshot clone service
117
-
# Make sure s3 service exist if enable is set true
118
-
enable: false
119
-
port: 5555
120
-
dummyPort: 8800
121
-
proxyPort: 8900
122
-
s3Config:
123
-
# Access Key for the S3 service. Uploading snapshots
124
-
ak: <>
125
-
# Access Key for the S3 service. Uploading snapshots
126
-
sk: <>
127
-
# S3 service address
128
-
nosAddress: <>
129
-
# S3 service bucket name to store snapshots
130
-
bucketName: <>
131
-
```
41
+
[CurveBS three replicas deployment](https://github.com/opencurve/curve-operator/blob/master/config/samples/cluster.yaml)
[CurveFS three replicase deployment](https://github.com/opencurve/curve-operator/blob/master/config/samples/fscluster.yaml)
134
46
135
-
The specific meaning of each configuration item is already in the comments above `cluster.yaml`, which can be modified according to the instructions. Then you can deploy it in cluster.
47
+
Here we take the deployment of a three-node `CurveBS` cluster as an example. The yaml file is [cluster.yaml](https://github.com/opencurve/curve-operator/blob/master/config/samples/cluster.yaml) and you can learn more about how to modify this configuration file through the comments in it.
136
48
137
-
Create the cluster:
49
+
Type the command to create the cluster:
138
50
139
51
```shell
140
52
$ kubectl apply -f config/samples/cluster.yaml
141
-
142
-
// or you can deploy stand-alone cluster using `bscluster-onhost.yaml`
143
53
```
144
54
145
-
using `kubectl` to list pods in the curve namespace. You should be able to see the following pods once they are all running. The chunkserver numbers of will depend on the number of nodes in the cluster and the the number of devices configured.
> Tips: The chunkserver pods may not start immediately, because the disk needs to be formatted in the background(`prepare-chunkfile` jobs), so it may take a while to see the chunkserver pod. The waiting time is determined according to the number and percentage of configured disks, and it may be a long time.
169
79
170
-
To verify that the cluster is in healthy state, enter one curve-chunkserver pod and `curve_ops_tools status` command to check.
80
+
### 3. Check cluster health
81
+
82
+
To verify that the cluster is in healthy state, enter one `curve-chunkserver` pod and type `curve_ops_tools status` command to check.
171
83
172
84
```shell
173
85
$ kubectl exec -it <any one chunkserver pod> -- bash
The cluster is deploymented completed and successfully if you see `cluster is healthy` prompt.
101
+
The cluster deployment completed and successfully if you see `cluster is healthy` prompt.
190
102
191
-
### 3. Use Curve CSI
103
+
## Curve CSI
192
104
193
-
After installing the operator and curve cluster, you can create a PVC that to use curvebs as pod storage.
105
+
Create a PVC that to use curvebs as pod storage.
194
106
195
-
Refer to the documentation [deploy curve-csi](https://ask.opencurve.io/t/topic/89) to deploy curve-csi and use curve as backend storage of pod.
107
+
you can deploy and get more details from [curve-csi](https://github.com/opencurve/curve-csi) project that dock `curvebs` cluster or [curvefs-csi](https://github.com/opencurve/curvefs-csi) project that dock `curvefs` cluster.
196
108
197
-
More details can see `curve-csi` project at [curve-csi github](https://github.com/opencurve/curve-csi).
109
+
## Remove
198
110
199
-
## Uninstall curve cluster
111
+
Remove curve cluster deployed already and clean up data on host.
200
112
201
-
You can uninstall curve cluster deployed and clean up data on host.
202
-
203
-
### 1. Delete the `Curvecluster` CR:
113
+
### 1.Delete the cluster cr
204
114
205
115
```shell
206
116
$ kubectl -n curve delete curvecluster my-cluster
207
117
```
208
118
209
-
Verify that the cluster CR has been deleted before continuing to the next step.
119
+
Verify the cluster CR has been deleted before continuing to the next step.
210
120
211
121
```shell
212
122
$ kubectl -n curve get curvecluster
213
123
```
214
124
215
-
### 2. Delete the Operator and related Resources
216
-
217
-
This will begin the process of the curve-operator and all other resources being cleaned up.
125
+
### 2.Delete the Operator and related Resources
218
126
219
127
```shell
220
128
$ kubectl delete -f config/deploy/
221
129
```
222
130
223
131
### 3. Delete data and log on host
224
132
225
-
The final cleanup setp requires deleting files on each host in the cluster. All files under the `hostDataDir` property specified in the cluster CRD will need to be deleted. Otherwise, inconsistent state will remain when a new cluster is started.
133
+
The final cleanup step requires deleting files on each host in the cluster. All files under the `hostDataDir` property specified in the cluster CRD will need to be deleted. Otherwise, inconsistent state will remain when a new cluster is started.
226
134
227
135
Connect to each machine and delete `/curvebs`, or the path specified by the `dataDirHostPath` and `logDirHostPath`.
228
136
229
137
```shell
230
138
$ rm -rf /curvebs
231
139
```
232
140
233
-
In the future this setp will not neccssary that we can delete it by running job on cluster if `cleanUpConfirm` is set.
141
+
## Contributing
142
+
143
+
We welcome help in any form, including but not limited to improving documentation, asking questions, fixing bugs, and adding features.
144
+
145
+
## Meeting
146
+
147
+
We have an online community meeting every two weeks which talk about what `Curve` is doing and planning to do. You can view meeting minutes and agenda here [Double Week Meetings](https://github.com/opencurve/curve-meetup-slides/tree/main/2023/Double%20Week%20Meetings).
148
+
149
+
## License
150
+
151
+
You are required to comply with the [CNCF](https://www.cncf.io/) Code of Conduct while participating in this project.
152
+
153
+
## Contact
154
+
155
+
If you encounter any problems during use, please submit an [Issue](https://github.com/opencurve/curve-operator/issues) for feedback. You can also scan [the WeChat QR code](https://github.com/opencurve/curve-operator/tree/master/docs/images/curve-wechat.jpeg) to join the technical exchange group.
0 commit comments