Skip to content

Commit 29a1de3

Browse files
Add vsr (#61)
* add node not found error * demux example * feat(application): add basicVSR style(basicVSR): lint improvement(vsr): update result Co-authored-by: MegEngine <[email protected]>
1 parent 44151c2 commit 29a1de3

File tree

16 files changed

+480
-19
lines changed

16 files changed

+480
-19
lines changed

docs/download-models.zh.md

+6-7
Original file line numberDiff line numberDiff line change
@@ -1,16 +1,15 @@
1-
# 模型下载
1+
# 模型和测试数据下载
22

3-
MegFlow 所有模型都压缩成了单个 models.zip 。
4-
5-
| 云盘 | google drive |
6-
| - | - |
7-
| 链接: https://pan.baidu.com/s/1ZLVBR0igJ1hL6PoYQDtByA 提取码: 8ajf | [google](https://drive.google.com/file/d/1EwMJFjNp2kuNglutoleZOVsqccSOW2Z4/view?usp=sharing) |
3+
| 应用名称 | 云盘 | google drive |
4+
| - | - | - |
5+
| 猫猫围栏、电动车检测 | 链接: https://pan.baidu.com/s/1ZLVBR0igJ1hL6PoYQDtByA 提取码: 8ajf | [google](https://drive.google.com/file/d/1EwMJFjNp2kuNglutoleZOVsqccSOW2Z4/view?usp=sharing) |
6+
| 视频超分 | 链接: https://pan.baidu.com/s/131Ul2A9DNxTXbatO1SGKFg?pwd=viy5 提取码: viy5 | [google](https://drive.google.com/file/d/1oyrVL20MODJOSf7BJ9T5OioE-ZaARDBC/view?usp=sharing) |
87

98
取最新的 models_xxx.zip,解压、软链为 examples/models
109

1110
```bash
1211
$ wget ${URL}/modes.zip
13-
$ cd flow-python/examples
12+
$ cd flow-python/examples/application
1413
$ ln -s ${DOWNLOAD_DIR}/models models
1514
```
1615

flow-python/examples/application/cat_finder/README.md

+3-2
Original file line numberDiff line numberDiff line change
@@ -95,7 +95,9 @@ $ megflow_run -c cat_finder/video_cpu.toml -p cat_finder # 无 GPU 的设备
9595

9696
开启一路视频流解析需要流的 url,这里有两种方法:
9797

98-
1)准备一个 rtsp 视频流地址,做测试输入(流地址部署不方便,也可以直接用离线文件的绝对路径代替)。模型包目录提供了测试视频,在 `models/cat_finder_testdata`,需要用户自行部署 live555 服务。最直接的办法:
98+
1)离线文件方式。也就是在 WebUI 中输入类似`/mnt/data/stream/file.ts` 的路径。需要自行保证服务器可访问这个文件、并且格式是可以被 ffmpeg 解析的(例如 .ts/.mp4/.h264/.h265)。
99+
100+
2)在线流方式。准备一个 rtsp 视频流地址,做测试输入。模型包目录提供了测试视频,在 `models/cat_finder_testdata`,需要用户自行部署 live555 服务。最直接的办法:
99101
```bash
100102
$ wget https://github.com/aler9/rtsp-simple-server/releases/download/v0.17.2/rtsp-simple-server_v0.17.2_linux_amd64.tar.gz
101103
$
@@ -108,7 +110,6 @@ $ ffmpeg -re -stream_loop -1 -i ${models}/cat_finder_testdata/test1.ts -c copy -
108110

109111
相关教程已整合在 [如何生成自己的 rtsp 流地址](../../../../docs/02-how-to-run/generate-rtsp.zh.md)
110112

111-
2)如果 rtsp 流地址部署不方便,也可以直接用离线文件的绝对路径代替,也就是在 WebUI 中输入类似`/mnt/data/stream/file.ts` 的路径。需要自行保证服务器可访问这个文件、并且格式是可以被 ffmpeg 解析的(例如 .ts/.mp4/.h264/.h265)。
112113

113114
## 六、视频识别 FAQ
114115

Original file line numberDiff line numberDiff line change
@@ -0,0 +1,32 @@
1+
# 视频实时超分
2+
3+
## 一、[下载模型和测试数据](../../../../docs/download-models.zh.md)
4+
5+
## 二、运行
6+
7+
模型软链后,使用`megflow_run`运行
8+
```bash
9+
$ cd ${path/to/MegFlow}/flow-python/examples/application # 这行必须
10+
$ megflow_run -c video_super_resolution/config.toml -p video_super_resolution
11+
```
12+
13+
浏览器打开 [8087 端口](http://10.122.101.175:8087/docs#/default/post_start__url_),try it out。
14+
15+
1)上传测试数据中的 `a.mp4`,获取 stream_id
16+
17+
2)用 stream_id 查询结果存到了哪个文件,例如 kj2WAS.flv
18+
19+
## 三、如何使用自己的模型
20+
21+
超分模型使用 [basicVSR_mge](https://github.com/Feynman1999/basicVSR_mge) 做训练,炼丹结束后请 cherry-pick [这个 PR](https://github.com/Feynman1999/basicVSR_mge/pull/6),然后把模型 jit.trace 成纯推理格式。
22+
23+
```bash
24+
$ cd ${path/to/basicVSR_mge}
25+
$ python3 tools/dump.py configs/restorers/BasicVSR/basicVSR_test_valid.py
26+
```
27+
28+
## 四、其他
29+
构造测试输入需**注意视频质量**,例如
30+
```bash
31+
$ ffmpeg -i images/%08d.png -q:v 2 -vcodec mpeg4 -r 20 a.mp4
32+
```
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
# MegFlow is Licensed under the Apache License, Version 2.0 (the "License")
2+
#
3+
# Copyright (c) 2019-2021 Megvii Inc. All rights reserved.
4+
#
5+
# Unless required by applicable law or agreed to in writing,
6+
# software distributed under the License is distributed on an
7+
# "AS IS" BASIS, WITHOUT ARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
8+
9+
#!/usr/bin/env python
10+
# coding=utf-8
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,38 @@
1+
main = "vsr"
2+
3+
[[graphs]]
4+
name = "subgraph"
5+
inputs = [{ name = "inp", cap = 128, ports = ["model:inp"] }]
6+
outputs = [{ name = "out", cap = 128, ports = ["save:out"] }]
7+
connections = [
8+
{ cap = 128, ports = ["model:out", "save:inp"] }
9+
]
10+
11+
[[graphs.nodes]]
12+
name = "model"
13+
ty = "Model"
14+
dir = "models/vsr_models"
15+
device = "gpu"
16+
device_id = 0
17+
18+
[[graphs.nodes]]
19+
name = "save"
20+
ty = "Save"
21+
path = "./"
22+
23+
[[graphs]]
24+
name = "vsr"
25+
connections = [
26+
{ cap = 16, ports = ["source:out", "destination:inp"] },
27+
{ cap = 16, ports = ["source:inp", "destination:out"] }
28+
]
29+
30+
[[graphs.nodes]]
31+
name = "source"
32+
ty = "VideoServer"
33+
port = 8087
34+
response = "json"
35+
36+
[[graphs.nodes]]
37+
name = "destination"
38+
ty = "subgraph"
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,194 @@
1+
# MegFlow is Licensed under the Apache License, Version 2.0 (the "License")
2+
#
3+
# Copyright (c) 2019-2021 Megvii Inc. All rights reserved.
4+
#
5+
# Unless required by applicable law or agreed to in writing,
6+
# software distributed under the License is distributed on an
7+
# "AS IS" BASIS, WITHOUT ARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
8+
9+
#!/usr/bin/env python
10+
# coding=utf-8
11+
12+
import argparse
13+
import time
14+
import os
15+
import cv2
16+
import numpy as np
17+
import megenginelite as mgelite
18+
19+
20+
class PredictorLite:
21+
def load(self, fullpath, config, device_id):
22+
net = mgelite.LiteNetwork(config=config)
23+
net.device_id = device_id
24+
print(fullpath)
25+
net.load(fullpath)
26+
return net
27+
28+
def __init__(
29+
self,
30+
path,
31+
device="gpu",
32+
device_id=0,
33+
):
34+
35+
if "gpu" in device.lower():
36+
device_type = mgelite.LiteDeviceType.LITE_CUDA
37+
else:
38+
device_type = mgelite.LiteDeviceType.LITE_CPU
39+
net_config = mgelite.LiteConfig(device_type=device_type)
40+
41+
self.flownet = self.load(os.path.join(path, "flownet.mge"), net_config,
42+
device_id)
43+
self.gen = self.load(os.path.join(path, "generator.mge"), net_config,
44+
device_id)
45+
self.upsample = self.load(os.path.join(path, "upsample.mge"),
46+
net_config, device_id)
47+
self.HIDDEN_CHANNELS = 96
48+
print("basicVSR model loaded.")
49+
50+
def get_bilinear(self, image):
51+
B, T, C, h, w = image.shape
52+
image = image.reshape(-1, C, h, w)
53+
ret = np.zeros((image.shape[0], C, 4 * h, 4 * w), dtype=np.float32)
54+
for i in range(image.shape[0]):
55+
chw = image[i:i + 1].reshape(C, h, w)
56+
hwc = np.transpose(chw, (1, 2, 0))
57+
hwc = cv2.resize(hwc, (w * 4, h * 4))
58+
ret[i:i + 1] = np.transpose(hwc, (2, 0, 1))
59+
ret = ret.reshape(B, T, C, h * 4, w * 4)
60+
return ret
61+
62+
def inference_flownet(self, now_frame, ref):
63+
begin = time.time()
64+
65+
data0 = self.flownet.get_io_tensor("tenFirst")
66+
data0.set_data_by_share(now_frame)
67+
68+
data1 = self.flownet.get_io_tensor("tenSecond")
69+
data1.set_data_by_share(ref)
70+
self.flownet.forward()
71+
self.flownet.wait()
72+
73+
tensor = self.flownet.get_io_tensor(
74+
self.flownet.get_all_output_name()[0])
75+
timecost = time.time() - begin
76+
print(f"flownet timecost {timecost} ms")
77+
return tensor.to_numpy()
78+
79+
def inference_gen(self, hidden, flow, nowFrame):
80+
begin = time.time()
81+
82+
data0 = self.gen.get_io_tensor("hidden")
83+
data0.set_data_by_share(hidden)
84+
85+
data1 = self.gen.get_io_tensor("flow")
86+
data1.set_data_by_share(flow)
87+
88+
data2 = self.gen.get_io_tensor("nowFrame")
89+
data2.set_data_by_share(nowFrame)
90+
91+
self.gen.forward()
92+
self.gen.wait()
93+
94+
tensor = self.gen.get_io_tensor(self.gen.get_all_output_name()[0])
95+
timecost = time.time() - begin
96+
print(f"gen timecost {timecost} ms")
97+
return tensor.to_numpy()
98+
99+
def inference_upsample(self, forward_hidden, backward_hidden):
100+
begin = time.time()
101+
102+
data0 = self.upsample.get_io_tensor("forward_hidden")
103+
data0.set_data_by_share(forward_hidden)
104+
105+
data1 = self.upsample.get_io_tensor("backward_hidden")
106+
data1.set_data_by_share(backward_hidden)
107+
108+
self.upsample.forward()
109+
self.upsample.wait()
110+
111+
tensor = self.upsample.get_io_tensor(
112+
self.upsample.get_all_output_name()[0])
113+
timecost = time.time() - begin
114+
print(f"upsample timecost {timecost} ms")
115+
return tensor.to_numpy()
116+
117+
# shape [batch, 3, H, W]
118+
def inference(self, input):
119+
input = input.astype(np.float32) / 255.0
120+
input = np.expand_dims(input, axis=0) # [1,100,3,180,320]
121+
122+
image = np.ascontiguousarray(input, np.float32)
123+
124+
B, T, _, h, w = image.shape
125+
biup = self.get_bilinear(image)
126+
forward_hiddens = []
127+
backward_hiddens = []
128+
res = []
129+
hidden = np.zeros((2 * B, self.HIDDEN_CHANNELS, h, w),
130+
dtype=np.float32)
131+
for i in range(T):
132+
now_frame = np.concatenate(
133+
[image[:, i, ...], image[:, T - i - 1, ...]], axis=0)
134+
if i == 0:
135+
flow = self.inference_flownet(now_frame, now_frame)
136+
else:
137+
ref = np.concatenate(
138+
[image[:, i - 1, ...], image[:, T - i, ...]], axis=0)
139+
flow = self.inference_flownet(now_frame, ref)
140+
141+
hidden = self.inference_gen(hidden, flow, now_frame)
142+
forward_hiddens.append(hidden[0:B, ...])
143+
backward_hiddens.append(hidden[B:2 * B, ...])
144+
145+
for i in range(T):
146+
res.append(
147+
self.inference_upsample(forward_hiddens[i],
148+
backward_hiddens[T - i - 1]))
149+
150+
res = np.stack(res, axis=1) # [B,T,3,H,W]
151+
HR_G = res + biup
152+
HR_G = (np.clip(HR_G, a_min=0.0, a_max=1.0) * 255.0).round().astype(
153+
np.uint8)
154+
155+
ret = []
156+
for i in range(T):
157+
x = HR_G[0, i, ...]
158+
img_np = np.transpose(x[[2, 1, 0], :, :], (1, 2, 0))
159+
ret.append(img_np)
160+
return ret
161+
162+
163+
def make_parser():
164+
parser = argparse.ArgumentParser("ModelServing Demo!")
165+
parser.add_argument("--model",
166+
default=None,
167+
type=str,
168+
help=".mge for eval")
169+
return parser
170+
171+
172+
# if __name__ == "__main__":
173+
174+
# batchdata = None
175+
# imagelist = []
176+
# for parent, _, filenames in os.walk("images"):
177+
# filenames.sort()
178+
# for filename in filenames:
179+
# mat = cv2.imread(os.path.join(parent, filename))
180+
# print(filename)
181+
# mat = cv2.cvtColor(mat, cv2.COLOR_BGR2RGB)
182+
# mat = np.transpose(mat, (2, 0, 1))
183+
# mat = np.expand_dims(mat, axis=0)
184+
# if mat is not None:
185+
# imagelist.append(mat[...])
186+
# batchdata = np.concatenate(imagelist, axis=0)
187+
# assert(batchdata is not None)
188+
189+
# predictor = PredictorLite("./")
190+
# result = predictor.inference(batchdata)
191+
# assert(len(result) == batchdata.shape[0])
192+
193+
# for idx, image in enumerate(result):
194+
# cv2.imwrite(f"{idx}.jpg", image)

0 commit comments

Comments
 (0)