Skip to content

Commit

Permalink
Update Redis docs (#57)
Browse files Browse the repository at this point in the history
  • Loading branch information
acelyc111 authored Jan 11, 2024
1 parent 25be6b4 commit 78a0e7d
Show file tree
Hide file tree
Showing 3 changed files with 227 additions and 15 deletions.
214 changes: 213 additions & 1 deletion _docs/en/api/redis.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,4 +2,216 @@
permalink: api/redis
---

TRANSLATING
# Redis adaptation

## Architecture

After deploying Redis Proxy to Pegasus, users can directly access the proxy through the Redis protocol, thereby indirectly accessing Pegasus services. The overall architecture is as follows:

![redis_proxy_arch.png](/assets/images/redis_proxy_arch.png){:class="img-responsive"}

Using the [Redis Protocol](https://redis.io/topics/protocol) between Redis Client and Redis Proxy. Currently, thr proxy supports all [RESP2 protocols](https://redis.io/docs/reference/protocol-spec/) data types (i.e. Simple Strings, Errors, Integers, Bulk Strings and Arrays).

Redis Proxy uses the Pegasus protocol to communicate with the Pegasus cluster. Here, the proxy acts as a regular Pegasus client, querying routing table from Meta Server and reading and writing user data with Replica Server.

## The form of providing services

Like the Redis service, the proxy instances are provided in the form of `host:port`. If the service pressure is high, multiple proxy instances can be provided to improve service throughput through horizontal scaling.

Proxy is stateless, and multiple proxy instances share the same backend Pegasus service. Load balancing can be achieved through methods such as round-robin or hash.

> The executable binary of Proxy is named `pegasus_rproxy`, [packed](/docs/build/compile-by-docker/#packaging) by `./run.sh pack_tools`
## Configuration

The configuration file rules for Redis Proxy follow [Configurations](/administration/config), referring to the [example](https://github.com/apache/incubator-pegasus/blob/master/src/redis_protocol/proxy/config.ini).

There are several unique configuration items in Proxy that need to be noted:

```
[apps.proxy]
name = proxy
type = proxy
; which pegasus cluster and table dose this proxy redirect to
; - 'onebox': the cluster name which will be used in the next section
; - 'temp': the table name in the cluster
arguments = onebox temp
; if using GEO APIs, an extra table name which will store geo index data
; should be appended, i.e.
; arguments = onebox temp temp_geo
; port serve for redis clients
ports = 6379
pools = THREAD_POOL_DEFAULT
run = true
[pegasus.clusters]
; meta serer list the proxy redirect to
onebox = 127.0.0.1:34601,127.0.0.1:34602,127.0.0.1:34603
```

## APIs

The native commands for Redis can be found [here](https://redis.io/commands)

The following interfaces are compatible with Redis native commands, but may support fewer parameters than Redis.

> The following documents provide all the parameters currently supported by Pegasus Redis Proxy, and those that are not provided are currently not supported.
### Protocol

#### Strings API

For Redis [strings](https://redis.io/docs/data-types/strings/) commands, the key corresponds to the hashkey in Pegasus, while the sortkey in Pegasus is set to an empty string `""`.

The supported commands are `SET`, `GET`, `TTL`, `INCR`, etc.

#### GEO API

[The Implementation Principle of GEO Commands](geo)

In Redis, the data operated by [GEO](https://redis.io/docs/data-types/geospatial/) commands are added by [GEOADD](https://redis.io/commands/geoadd/) commands, i.e. `GEOADD key longitude latitude member`. Here, the `key` has a "namespace" concept, but not the `key` operated by `SET` command.

In Pegasus Proxy, due to the difference of underlying implementation principles,the data operated by `GEO*` commands are added by `SET` command instead. The `key` of `SET` command corresponds to the `member` of `GEO*` commands, and the `key` of `GEO*` must be empty string `""`.

That is to say, in Pegasus's Redis GEO data, there is no longer the concept of "namespace", and all data is in the same space. To distinguish namespaces, a new table can be created in the Pegasus to achieve this.

The format of the `value` of `SET` commands refers to [Value Extrator](/api/geo#value-extrator)

### SET

```
SET key value [EX seconds]
```

### GET

```
GET key
```

### DEL

```
DEL key
```

**Note:**

When a key does not exist, the return value of Pegasus Proxy is slightly different from Redis:
- Redis: The interface returns 0, indicating that no valid data has been deleted this time
- Pegasus Proxy: Since there is no distinction between non-existent and successfully deleted, return a unified 1 in both cases

### SETEX

```
SETEX key seconds value
```

### TTL

```
TTL key
```

### PTTL

```
PTTL key
```

### INCR

```
INCR key
```

### INCRBY

```
INCRBY key increment
```

### DECR

```
DECR key
```

### DECRBY

```
DECRBY key decrement
```

### GEODIST

```
GEODIST key member1 member2 [unit]
```

**Note:**

- The key rule follows the key rule of the Pegasus Proxy `GEO*` commands, which means that the `key` can only be an empty string `""`, and here the `member` corresponds to the `key` during the `SET` command

### GEORADIUS

```
GEORADIUS key longitude latitude radius m|km|ft|mi [WITHCOORD][WITHDIST] [WITHHASH][COUNT count] [ASC|DESC]
```

**Note:**
- The key rule follows the key rule of the Pegasus Proxy `GEO*` commands, which means that the `key` can only be an empty string `""`, and here the `member` corresponds to the `key` during the `SET` command
- Pegasus Proxy has modified the mean of `WITHHASH` parameter of Redis, using it will return the `value` of that `member`

### GEORADIUSBYMEMBER

```
GEORADIUSBYMEMBER key member radius m|km|ft|mi [WITHCOORD][WITHDIST] [WITHHASH][COUNT count] [ASC|DESC]
```

**Note:**
- The key rule follows the key rule of the Pegasus Proxy `GEO*` commands, which means that the `key` can only be an empty string `""`, and here the `member` corresponds to the `key` during the `SET` command
- Pegasus Proxy has modified the mean of `WITHHASH` parameter of Redis, using it will return the `value` of that `member`

## Sample

```
// Strings commands sample
127.0.0.1:6379> SET abc 1 EX 60
OK
127.0.0.1:6379> GET abc
"1"
127.0.0.1:6379> TTL abc
(integer) 52
127.0.0.1:6379> INCR abc
(integer) 2
// GEO commands sample
127.0.0.1:6379> SET 1cc0001000010290050356f "1cc0001000010290050356f|2018-06-10 23:59:59|2018-06-11 13:00:00|wx5j5ff05|116.886447|40.269031|4.863045|20.563248|0|-1"
OK
127.0.0.1:6379> SET 2cc0001000010290050356f "2cc0001000010290050356f|2018-06-10 23:59:59|2018-06-11 13:00:00|wx5j5ff05|115.886447|41.269031|4.863045|20.563248|0|-1"
OK
127.0.0.1:6379> GEORADIUS "" 116.889137 40.261774 1000 m COUNT 100 ASC WITHDIST WITHCOORD
1) 1) "1cc0001000010290050356f"
2) "838.600772"
3) 1) "116.886447"
2) "40.269031"
4) "1cc0001000010290050356f|2018-06-10 23:59:59|2018-06-11 13:00:00|wx5j5ff05|116.886447|40.269031|4.863045|20.563248|0|-1"
127.0.0.1:6379> GEORADIUSBYMEMBER "" 1cc0001000010290050356f 1000 m WITHCOORD WITHDIST WITHHASH
1) 1) "1cc0001000010290050356f"
2) "0.000000"
3) 1) "116.886447"
2) "40.269031"
4) "1cc0001000010290050356f|2018-06-10 23:59:59|2018-06-11 13:00:00|wx5j5ff05|116.886447|40.269031|4.863045|20.563248|0|-1"
127.0.0.1:6379> GEODIST "" 1cc0001000010290050356f 2cc0001000010290050356f m
"139483.293598"
```
2 changes: 1 addition & 1 deletion _docs/zh/api/geo.md
Original file line number Diff line number Diff line change
Expand Up @@ -235,7 +235,7 @@ max_level = 16
min_level = 12
```

## 自定义extrator
## Value Extrator

目前 Pegasus 支持从固定格式的 value 中解析出经纬度。经纬度以字符串形式嵌入在 value 中,以`|`分割。

Expand Down
26 changes: 13 additions & 13 deletions _docs/zh/api/redis.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,9 +20,9 @@ Redis Proxy 与 Pegasus 集群之间使用 Pegasus 的协议,Proxy 在这里

跟 Redis 服务一样,Proxy 实例以 `host:port` 形式提供。如果服务压力大,可以提供多个 Proxy 实例,通过水平扩展的方式来提升服务吞吐量。

Proxy 是无状态的,多个 Proxy 实例共享同一个后端 Pegasus 服务。可以采用round robin, hash等方式进行负载均衡。
Proxy 是无状态的,多个 Proxy 实例共享同一个后端 Pegasus 服务。可以采用round-robin, hash等方式进行负载均衡。

> Proxy 的可执行文件为 `pegasus_rproxy`, `./run.sh pack_tools` [打包](/docs/build/compile-by-docker/#packaging)生成。
> Proxy 的可执行文件为 `pegasus_rproxy``./run.sh pack_tools` [打包](/docs/build/compile-by-docker/#编译打包)生成。
## 配置

Expand Down Expand Up @@ -57,7 +57,7 @@ onebox = 127.0.0.1:34601,127.0.0.1:34602,127.0.0.1:34603

Redis 的原生命令请见[这里](https://redis.io/commands)

以下接口都兼容 Redis 原生命令,但支持的参数可能少于 Redis。
以下命令都兼容 Redis 原生命令,但支持的参数可能少于 Redis。

> 以下文档中都给出了目前 Pegasus Redis Proxy 所支持的所有参数,未给出的目前不支持。
Expand All @@ -71,13 +71,15 @@ Redis 的原生命令请见[这里](https://redis.io/commands) 。

#### GEO API

在 Redis 中,[GEO](https://redis.io/docs/data-types/geospatial/) 接口操作的数据是通过 [GEOADD](https://redis.io/commands/geoadd/),即 `GEOADD key longitude latitude member`,添加到数据库中的。此处的 `key` 是一个 namespace 的概念,而不是 `SET` 操作时的 key。
[GEO命令的实现原理](geo)

而在 Pegasus Proxy 中,由于底层的实现原理不同,他的 `GEO*` 接口操作的数据是通过 `SET` 接口添加到数据库中的,`SET`key 对应于 `GEO*` 接口的 member,而 `GEO*` 接口的 key 则只能是空串 `""`
在 Redis 中,[GEO](https://redis.io/docs/data-types/geospatial/) 命令操作的数据是通过 [GEOADD](https://redis.io/commands/geoadd/),即 `GEOADD key longitude latitude member`,添加到数据库中的。此处的 `key` 是一个 namespace 的概念,而不是 `SET` 操作时的 key

也就是说,在 Pegasus 的 Redis GEO 数据中,不再有 namespace 的概念,全部数据在同一空间 `""` 下。若要区分 namespace,可以在 Pegasus 层创建新的表来实现
而在 Pegasus Proxy 中,由于底层的实现原理不同,他的 `GEO*` 命令操作的数据是通过 `SET` 命令添加到数据库中的,`SET` 的 key 对应于 `GEO*` 命令的 member,而 `GEO*` 命令的 key 则只能是空串 `""`

`SET` 的 value 格式参考[这里](https://pegasus.apache.org/zh/api/geo#%E8%87%AA%E5%AE%9A%E4%B9%89extrator)
也就是说,在 Pegasus 的 Redis GEO 数据中,不再有 namespace 的概念,全部数据在同一空间 `""` 下。若要区分 namespace,可以在 Pegasus 中创建新的表来实现。

`SET` 的 value 格式参考[Value Extrator](/api/geo#value-extrator)

### SET

Expand All @@ -99,9 +101,9 @@ DEL key

**注意:**

这里的接口返回值和 Redis 的定义略有不同
- 当 key 不存在时,Redis 接口返回 0,表示本次没有删除有效数据
- Pegasus Proxy 由于没有对不存在和删除成功做区别,都统一返回的 1
当一条 key 不存在时,Pegasus Proxy 的返回值和 Redis 略有不同
- Redis:返回 0,表示本次没有删除有效数据
- Pegasus Proxy由于没有对不存在和删除成功做区别,都统一返回的 1

### SETEX

Expand Down Expand Up @@ -147,8 +149,6 @@ DECRBY key decrement

### GEODIST

[GEO接口的实现原理](geo)

```
GEODIST key member1 member2 [unit]
```
Expand All @@ -164,7 +164,7 @@ GEORADIUS key longitude latitude radius m|km|ft|mi [WITHCOORD][WITHDIST] [WITHHA
```

**注意:**
- key 规则遵循 GEO API 的 key 规则,即 key 只能是空串 `""`,而这里 member 对应于 `SET` 操作时的 key
- key 规则遵循上述 Pegasus Proxy 的 GEO 命令的 key 规则,即 key 只能是空串 `""`,而这里 member 对应于 `SET` 操作时的 key
- Pegasus 对 Redis 的 `WITHHASH` 参数进行了修改,使用它将会返回该 member 的 value 值

### GEORADIUSBYMEMBER
Expand Down

0 comments on commit 78a0e7d

Please sign in to comment.