专栏算法工具链提速开发环境推理：horizon_nn_gpu使用&对比

提速开发环境推理：horizon_nn_gpu使用&对比

Huanghui2024-10-16
130
0

1. 说明和使用方法

  • 工具链的OE包默认使用的horizon_nn，使用horizon_nn_gpu时先卸载horizon_nn。

  • horizon_nn_gpu版本的位置
    /open_exploror/ddk/package/host/ai_toolchain下的horizon_nn_gpu-XXX-py3-none-any.whl

  • Gpu版本启用情况验证

2. GPU推理前后性能对比

测试项目

加速效果

结果提取分析

horizon_nn（0.16.3）

horizon-nn-gpu（0.16.3）

模型解析

无

2023-04-20 00:11:40,189 INFO [Thu Apr 20 00:11:40 2023] Start to parse the onnx model.
2023-04-20 00:11:44,056 INFO [Thu Apr 20 00:11:44 2023] End to parse the onnx model.
00:11:44 - 00:11:40 = 00:00:04
2023-04-19 23:13:15,171 INFO [Wed Apr 19 23:13:15 2023] Start to parse the onnx model.
2023-04-19 23:13:18,850 INFO [Wed Apr 19 23:13:18 2023] End to parse the onnx model.
23:13:18 - 23:13:15 = 00:00:03

模型优化

无

2023-04-20 00:11:47,917 INFO [Thu Apr 20 00:11:47 2023] Start to optimize the model.
2023-04-20 00:12:31,594 INFO [Thu Apr 20 00:12:31 2023] End to optimize the model.
00:12:31 - 00:11:47 = 00:00:44
2023-04-19 23:13:22,524 INFO [Wed Apr 19 23:13:22 2023] Start to optimize the model.
2023-04-19 23:14:05,682 INFO [Wed Apr 19 23:14:05 2023] End to optimize the model.
23:14:05 - 23:13:22 = 00:00:43

数据校准

有

2023-04-20 00:12:41,718 INFO Run calibration model with kl method.
2023-04-20 00:18:53,303 INFO [Thu Apr 20 00:18:53 2023] End to calibrate the model.
00:18:53 - 00:12:41 = 00:06:12
2023-04-19 15:13:54,189 INFO Run calibration model with default calibration method.
2023-04-19 15:14:42,363 INFO [Wed Apr 19 15:14:42 2023] End to calibrate the model.
23:14:46 - 23:14:16 = 00:00:30

量化模型生成

无

2023-04-20 00:18:53,303 INFO [Thu Apr 20 00:18:53 2023] Start to quantize the model.
2023-04-20 00:21:26,285 INFO [Thu Apr 20 00:21:26 2023] End to quantize the model.
00:21:26 - 00:18:53 = 00:02:33
2023-04-19 23:14:46,865 INFO [Wed Apr 19 23:14:46 2023] Start to quantize the model.
2023-04-19 23:16:13,883 INFO [Wed Apr 19 23:16:13 2023] End to quantize the model.
23:16:13 - 23:14:46 = 00:02:27

模型编译

无

2023-04-20 00:21:47,712 INFO [Thu Apr 20 00:21:47 2023] Start to compile the model with march bayes.
2023-04-20 00:22:11,635 INFO [Thu Apr 20 00:22:11 2023] End to compile the model with march bayes.
00:22:11 - 00:21:47 = 00:00:24
2023-04-19 23:16:34,754 INFO [Wed Apr 19 23:16:34 2023] Start to compile the model with march bayes.
2023-04-19 23:16:59,268 INFO [Wed Apr 19 23:16:59 2023] End to compile the model with march bayes.
23:16:59 - 23:16:34 = 00:00:25

精度测试

有
Origin
2023-04-20 00:25:19,737 INFO Feed batch 1/5000
2023-04-20 02:45:20,633 INFO Feed batch 5000/5000
02:45:20 - 00:25:19 = 02:20:01
loading annotations into memory...
Done   (t=0.91s)
creating index...
index created!
2023-04-20 02:45:22,884 INFO start to calc eval info...
Loading and preparing results...
DONE   (t=6.05s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type bbox
DONE   (t=77.95s).
Accumulating evaluation results...
DONE   (t=9.73s).
2023-04-19 23:21:40,172 INFO Feed batch 1/5000
2023-04-19 23:42:30,582 INFO Feed batch 5000/5000
23:42:30 - 23:21:40 = 00:20:50
loading annotations into memory...
Done   (t=0.81s)
creating index...
index created!
2023-04-19 23:42:31,594 INFO start to calc eval info...
Loading and preparing results...
DONE   (t=5.89s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type bbox
DONE   (t=76.13s).
Accumulating evaluation results...
DONE   (t=9.04s).
有
Quantized
2023-04-20 02:47:34,563 INFO Init 10 processes
2023-04-20 02:47:35,362 INFO Feed batch 1/5000
2023-04-20 09:37:45,254 INFO Feed batch 3204/5000
09:37:45 - 02:47:35 = 06:50:10
only 1 processes on GPU mode
2023-04-19 23:44:51,076 INFO Feed batch 1/5000
2023-04-20 00:06:14,301 INFO Feed batch 5000/5000
00:06:14 - 23:44:51 = 00:21:23

3. 遇到的问题 & 注意事项

算法工具链
官方教程
评论0
0/1000