点云3D目标检测 pointpillars 在X3开发板上实现

已解决

hjw2020-11-23

660

2020年6月，地平线在Google Waymo自动驾驶算法大赛中，取得4项全球第一，1项全球第二，所以地平线在这一领域肯定有很多积累，目前我这里在移植pointpillars到X3的过程中，遇到几个问题：

1、pfe.onnx模型使用工具链检测失败，目前不知道问题

2、pfe.onnx和rpn.onnx这两部分如何最高效率的衔接起立

算法工具链

0/1000

anming
Lv.1
下载最新版本工具，再试试
2020-11-23
0
10
- hjw回复anming:
  原来使用的是 x3_tc.1.1.12 报错如上面所述，然后换了 x3_tc.1.1.17e 报错如下：
  1、执行check报错：
  [root@ce390fd69118 mapper]#
  [root@ce390fd69118 mapper]# sh 01_check.sh
  2020-11-23 20:55:32,257 INFO Start hb_mapper....
  2020-11-23 20:55:32,257 INFO hb_mapper version 1.1.24
  2020-11-23 20:55:32,470 INFO Model type: onnx
  2020-11-23 20:55:32,471 INFO output file: ./hjw_demo_checker.log
  2020-11-23 20:55:32,471 INFO input names []
  2020-11-23 20:55:32,471 INFO input shapes {}
  2020-11-23 20:55:32,472 INFO Begin model checking....
  2020-11-23 20:55:32,472 INFO [Mon Nov 23 20:55:32 2020] Start to Horizon NN Model Convert.
  2020-11-23 20:55:32,472 INFO The input parameter is not specified, convert with default parameters.
  2020-11-23 20:55:32,473 INFO The hbdk parameter is not specified, and the submodel will be compiled with the default parameter.
  2020-11-23 20:55:32,473 INFO HorizonNN version: 0.7.6
  2020-11-23 20:55:32,473 INFO HBDK version: 3.12.9
  2020-11-23 20:55:32,473 INFO [Mon Nov 23 20:55:32 2020] Start to parse the onnx model.
  2020-11-23 20:55:32,480 INFO ONNX model info:
  ONNX IR version: 6
  Opset version: 10
  Input name: input.1, [8235, 32, 10]
  2020-11-23 20:55:32,481 INFO [Mon Nov 23 20:55:32 2020] End to parse the onnx model.
  2020-11-23 20:55:32,481 INFO Model input names: ['input.1']
  2020-11-23 20:55:32,508 INFO Saving the original float model: ./.hb_check/original_float_model.onnx.
  2020-11-23 20:55:32,509 INFO [Mon Nov 23 20:55:32 2020] Start to optimize the model.
  2020-11-23 20:55:32,510 INFO [Mon Nov 23 20:55:32 2020] End to optimize the model.
  2020-11-23 20:55:32,533 INFO Saving the optimized model: ./.hb_check/optimized_float_model.onnx.
  2020-11-23 20:55:32,534 INFO [Mon Nov 23 20:55:32 2020] Start to calibrate the model.
  2020-11-23 20:55:32,535 INFO [Mon Nov 23 20:55:32 2020] End to calibrate the model.
  2020-11-23 20:55:32,535 INFO [Mon Nov 23 20:55:32 2020] Start to quantize the model.
  2020-11-23 20:55:32,536 INFO [Mon Nov 23 20:55:32 2020] End to quantize the model.
  2020-11-23 20:55:32,559 INFO Saving the quantized model: ./.hb_check/quantized_model.onnx.
  2020-11-23 20:55:32,560 INFO [Mon Nov 23 20:55:32 2020] Start to compile the model with march bernoulli2.
  2020-11-23 20:55:32,562 ERROR The model does not have an ir_version set properly.
  2020-11-23 20:55:32,563 INFO [Mon Nov 23 20:55:32 2020] End to compile the model with march bernoulli2.
  2020-11-23 20:55:32,563 INFO [Mon Nov 23 20:55:32 2020] End to Horizon NN Model Convert.
  [root@ce390fd69118 mapper]#
  2、执行 build 报错：
  [root@ce390fd69118 mapper]#
  [root@ce390fd69118 mapper]# sh 02_build.sh
  2020-11-23 20:59:49,751 INFO Start hb_mapper....
  2020-11-23 20:59:49,752 INFO hb_mapper version 1.1.24
  2020-11-23 20:59:49,983 INFO norm_types[i]: no_preprocess
  2020-11-23 20:59:49,988 INFO Working dir: /horizon_x3_tc/horizon_x3_tc_1.1.17e/samples/05_miscellaneous/10_test/mapper/model_output
  2020-11-23 20:59:49,988 INFO Start Model Convert....
  2020-11-23 20:59:50,000 INFO [Mon Nov 23 20:59:49 2020] Start to Horizon NN Model Convert.
  2020-11-23 20:59:50,000 INFO Parsing the input parameter:{'input.1': {'input_shape': [8235, 32, 10]}}
  2020-11-23 20:59:50,000 INFO Parsing the calibration parameter
  2020-11-23 20:59:50,015 ERROR cannot reshape array of size 2635200 into shape (32,10)
  2020-11-23 20:59:50,015 INFO [Mon Nov 23 20:59:50 2020] End to Horizon NN Model Convert.
  [root@ce390fd69118 mapper]#
  2020-11-23
  0
  回复
- 芯链情报局回复hjw:
  你好，这里看到报错信息是维度不匹配：
  ERROR cannot reshape array of size 2635200 into shape (32,10)，正好少了一个 8235
  可以截图看下 yaml 是如何配置的吗？
  2020-12-07
  0
  回复
- hjw回复芯链情报局:
  # Copyright (c) 2020 Horizon Robotics.All Rights Reserved.
  #
  # The material in this file is confidential and contains trade secrets
  # of Horizon Robotics Inc. This is proprietary information owned by
  # Horizon Robotics Inc. No part of this work may be disclosed,
  # reproduced, copied, transmitted, or used in any way for any purpose,
  # without the express written permission of Horizon Robotics Inc.
  # 模型转化相关的参数
  model_parameters:
  # Caffe浮点网络数据模型文件
  caffe_model: ''
  # Caffe网络描述文件
  prototxt: ''
  # Onnx 浮点用户模型文件
  onnx_model: 'hjw_demo.onnx'
  # 指定模型转换过程中是否输出各层的中间结果，如果为True，则输出所有层的中间输出结果，
  layer_out_dump: False
  # 日志文件的输出控制参数，
  # debug输出模型转换的详细信息
  # info只输出关键信息
  # warn输出警告和错误级别以上的信息
  log_level: 'debug'
  # 模型转换输出的结果的存放目录
  working_dir: 'model_output'
  # 模型转换输出的用于上板执行的模型文件
  output_model_file_prefix: 'hjw_demo'
  # 模型输入相关参数
  input_parameters:
  #模型输入的节点名称, 此名称应与模型文件中的名称一致, 否则会报错
  input_name: 'input.1'
  # 网络实际执行时，输入给网络的数据格式，包括 nv12/featuremap/rgbp/bgrp,
  # 如果输入的数据为yuv444, 模型训练用的是rgb，则hb_mapper将自动插入YUV到RGB转化操作
  input_type_rt: 'featuremap'
  # 网络训练时输入的图像格式，可选的值为rgbp、bgrp
  input_type_train: 'featuremap'
  # 网络输入的预处理方法，主要有以下五种：
  # no_preprocess 不做任何操作
  # mean_file 图像均值文件
  # data_scale 对图像像素乘以data_scale
  # mean_file_and_scale 减去均值后再乘以scale
  norm_type: 'no_preprocess'
  # 经过resize和crop后，输入到网络的大小
  input_shape: '8235x32x10'
  calibration_parameters:
  # 模型量化的参考图像输入，图片格式支持Jpeg、Bmp等格式，输入的图片
  # 应该是使用的典型场景，一般是从测试集中选择20~50张图片，另外输入
  # 的图片要覆盖典型场景，不要是偏僻场景，如过曝光、饱和、模糊、
  # 纯黑、纯白等图片
  cal_data_dir: '../calibration_data_feature'
  # 如果输入的图片文件尺寸和模型训练的尺寸不一致时，并且pre_process_on为true，则将采用默认预处理方法，
  # 将输入图片缩放或者裁减到指定尺寸，否则，需要用户提前把图像处理为训练时的尺寸
  preprocess_on: False
  # 模型量化的算法类型，支持kl、max和promoter，通常采用KL即可满足要求
  calibration_type: 'kl'
  # 模型的量化校准方法设置为promoter，mapper会根据calibraion的数据对模型进行微调从而提高精度，
  # promoter_level的级别，可选的参数为0到2，建议按照0到2的顺数实验，满足精度即可停止实验
  # 0：表示对模型进行轻微调节，精度提高比较小
  # 1：表示相对1对模型调节幅度稍大，精度提高也比较多
  # 2：表示调节比较激进，可能造成精度的大幅提高也可能造成精度下降
  promoter_level: -1
  # 编译器相关参数
  compiler_parameters:
  # 编译策略，支持 bandwidth和latency两种优化模式，bandwidth以优化ddr的访问带宽为目标；
  # latency以优化推理时间为目标
  compile_mode: 'latency'
  # 设置debug为True将打开编译器的debug模式，能够输出性能仿真的相关信息，如帧率、DDR带宽占用等
  debug: True
  # 编译模型指定核数，不指定默认编译单核模型, 若编译双核模型，将下边注释打开即可
  # core_num: 2
  optimize_level: 'O3'
  2020-12-07
  0
  回复
- blackfeather回复hjw:
  你好，input_shape: '8235x32x10'这个地方，是否少些了batch_size的大小，请尝试改成input_shape: '1x8235x32x10'试一下
  2020-12-07
  0
  回复
- hjw回复blackfeather:
  你好，工具链：x3_tc.1.1.17e，将 input_shape: 由 '8235x32x10' 改为 '1x8235x32x10' ，报错如下：
  1、执行check报错：
  [root@ce390fd69118 mapper]# sh 01_check.sh
  2020-12-07 16:14:58,718 INFO Start hb_mapper....
  2020-12-07 16:14:58,719 INFO hb_mapper version 1.1.24
  2020-12-07 16:14:59,204 INFO Model type: onnx
  2020-12-07 16:14:59,204 INFO output file: ./hjw_demo_checker.log
  2020-12-07 16:14:59,204 INFO input names []
  2020-12-07 16:14:59,205 INFO input shapes {}
  2020-12-07 16:14:59,205 INFO Begin model checking....
  2020-12-07 16:14:59,205 INFO [Mon Dec 7 16:14:59 2020] Start to Horizon NN Model Convert.
  2020-12-07 16:14:59,205 INFO The input parameter is not specified, convert with default parameters.
  2020-12-07 16:14:59,205 INFO The hbdk parameter is not specified, and the submodel will be compiled with the default parameter.
  2020-12-07 16:14:59,205 INFO HorizonNN version: 0.7.6
  2020-12-07 16:14:59,205 INFO HBDK version: 3.12.9
  2020-12-07 16:14:59,205 INFO [Mon Dec 7 16:14:59 2020] Start to parse the onnx model.
  2020-12-07 16:14:59,218 INFO ONNX model info:
  ONNX IR version: 6
  Opset version: 10
  Input name: input.1, [8235, 32, 10]
  2020-12-07 16:14:59,219 INFO [Mon Dec 7 16:14:59 2020] End to parse the onnx model.
  2020-12-07 16:14:59,219 INFO Model input names: ['input.1']
  2020-12-07 16:14:59,310 INFO Saving the original float model: ./.hb_check/original_float_model.onnx.
  2020-12-07 16:14:59,310 INFO [Mon Dec 7 16:14:59 2020] Start to optimize the model.
  2020-12-07 16:14:59,312 INFO [Mon Dec 7 16:14:59 2020] End to optimize the model.
  2020-12-07 16:14:59,338 INFO Saving the optimized model: ./.hb_check/optimized_float_model.onnx.
  2020-12-07 16:14:59,338 INFO [Mon Dec 7 16:14:59 2020] Start to calibrate the model.
  2020-12-07 16:14:59,339 INFO [Mon Dec 7 16:14:59 2020] End to calibrate the model.
  2020-12-07 16:14:59,340 INFO [Mon Dec 7 16:14:59 2020] Start to quantize the model.
  2020-12-07 16:14:59,340 INFO [Mon Dec 7 16:14:59 2020] End to quantize the model.
  2020-12-07 16:14:59,364 INFO Saving the quantized model: ./.hb_check/quantized_model.onnx.
  2020-12-07 16:14:59,364 INFO [Mon Dec 7 16:14:59 2020] Start to compile the model with march bernoulli2.
  2020-12-07 16:14:59,371 ERROR The model does not have an ir_version set properly.
  2020-12-07 16:14:59,371 INFO [Mon Dec 7 16:14:59 2020] End to compile the model with march bernoulli2.
  2020-12-07 16:14:59,371 INFO [Mon Dec 7 16:14:59 2020] End to Horizon NN Model Convert.
  2、执行 build 报错：
  [root@ce390fd69118 mapper]# sh 02_build.sh
  2020-12-07 16:15:53,155 INFO Start hb_mapper....
  2020-12-07 16:15:53,156 INFO hb_mapper version 1.1.24
  2020-12-07 16:15:53,418 INFO norm_types[i]: no_preprocess
  2020-12-07 16:15:53,423 INFO Working dir: /horizon_x3_tc/horizon_x3_tc_1.1.17e/samples/05_miscellaneous/10_test/mapper/model_output
  2020-12-07 16:15:53,424 INFO Start Model Convert....
  2020-12-07 16:15:53,424 WARNING user input_shape: [1, 8235, 32, 10], proto input_shape: [8235, 32, 10].
  2020-12-07 16:15:53,436 INFO [Mon Dec 7 16:15:53 2020] Start to Horizon NN Model Convert.
  2020-12-07 16:15:53,436 INFO Parsing the input parameter:{'input.1': {'input_shape': [1, 8235, 32, 10]}}
  2020-12-07 16:15:53,436 INFO Parsing the calibration parameter
  2020-12-07 16:15:53,530 INFO Parsing the hbdk parameter:{'compile_mode': 'latency', 'debug': True, 'optimize_level': 'O3', 'input_source': {'input.1': 'ddr'}}
  2020-12-07 16:15:53,531 INFO HorizonNN version: 0.7.6
  2020-12-07 16:15:53,531 INFO HBDK version: 3.12.9
  2020-12-07 16:15:53,531 INFO [Mon Dec 7 16:15:53 2020] Start to parse the onnx model.
  2020-12-07 16:15:53,533 ERROR Input shape of input.1 has a dimension [1, 8235, 32, 10] different from [8235, 32, 10] which is parsered from onnx.
  2020-12-07 16:15:53,534 INFO [Mon Dec 7 16:15:53 2020] End to parse the onnx model.
  2020-12-07 16:15:53,534 INFO [Mon Dec 7 16:15:53 2020] End to Horizon NN Model Convert.
  2020-12-07
  0
  回复
- blackfeather回复hjw:
  你好，谢谢回复，想确认一下batch size相关的事宜，这个模型的batch_size是1呢还是8235呢？也就是输入是四维NCHW，还是只有三维的输入比如NHW这样？
  如果是四维输入，那模型在转到onnx时应该是有一些问题的，转到onnx时input_shape需要设置是四维的。如果是三维输入，麻烦将yaml中的input_shape: '8235x32x10'改成input_shape: '1x32x10'再试一下（yaml配置input_shape时batch_size直接设为1即可）
  2020-12-07
  0
  回复
- hjw回复blackfeather:
  你好，batch size为1，我pytorch训练推理的时候，pfe.onnx 的输入是 8235*32*10 的tensor，输出是 8235*1*16的 tensor，使用 torch.onnx.export 导出的 onnx
  按照你说的两种情况，均测试过了，pfe.onnx 和 rpn.onnx 也发给 Bill 了；
  第一种情况（如果是四维输入）前面已经贴了错误结果；
  第二种情况（如果是三维输入，改input_shape: '1x32x10'）的错误如下：
  1、执行check报错：
  [root@ce390fd69118 mapper]# sh 01_check.sh
  2020-12-07 20:34:15,955 INFO Start hb_mapper....
  2020-12-07 20:34:15,955 INFO hb_mapper version 1.1.24
  2020-12-07 20:34:16,347 INFO Model type: onnx
  2020-12-07 20:34:16,347 INFO output file: ./hjw_demo_checker.log
  2020-12-07 20:34:16,347 INFO input names []
  2020-12-07 20:34:16,348 INFO input shapes {}
  2020-12-07 20:34:16,348 INFO Begin model checking....
  2020-12-07 20:34:16,348 INFO [Mon Dec 7 20:34:16 2020] Start to Horizon NN Model Convert.
  2020-12-07 20:34:16,348 INFO The input parameter is not specified, convert with default parameters.
  2020-12-07 20:34:16,348 INFO The hbdk parameter is not specified, and the submodel will be compiled with the default parameter.
  2020-12-07 20:34:16,349 INFO HorizonNN version: 0.7.6
  2020-12-07 20:34:16,349 INFO HBDK version: 3.12.9
  2020-12-07 20:34:16,349 INFO [Mon Dec 7 20:34:16 2020] Start to parse the onnx model.
  2020-12-07 20:34:16,373 INFO ONNX model info:
  ONNX IR version: 6
  Opset version: 10
  Input name: input.1, [8235, 32, 10]
  2020-12-07 20:34:16,375 INFO [Mon Dec 7 20:34:16 2020] End to parse the onnx model.
  2020-12-07 20:34:16,375 INFO Model input names: ['input.1']
  2020-12-07 20:34:16,444 INFO Saving the original float model: ./.hb_check/original_float_model.onnx.
  2020-12-07 20:34:16,444 INFO [Mon Dec 7 20:34:16 2020] Start to optimize the model.
  2020-12-07 20:34:16,448 INFO [Mon Dec 7 20:34:16 2020] End to optimize the model.
  2020-12-07 20:34:16,483 INFO Saving the optimized model: ./.hb_check/optimized_float_model.onnx.
  2020-12-07 20:34:16,484 INFO [Mon Dec 7 20:34:16 2020] Start to calibrate the model.
  2020-12-07 20:34:16,485 INFO [Mon Dec 7 20:34:16 2020] End to calibrate the model.
  2020-12-07 20:34:16,485 INFO [Mon Dec 7 20:34:16 2020] Start to quantize the model.
  2020-12-07 20:34:16,486 INFO [Mon Dec 7 20:34:16 2020] End to quantize the model.
  2020-12-07 20:34:16,511 INFO Saving the quantized model: ./.hb_check/quantized_model.onnx.
  2020-12-07 20:34:16,512 INFO [Mon Dec 7 20:34:16 2020] Start to compile the model with march bernoulli2.
  2020-12-07 20:34:16,516 ERROR The model does not have an ir_version set properly.
  2020-12-07 20:34:16,517 INFO [Mon Dec 7 20:34:16 2020] End to compile the model with march bernoulli2.
  2020-12-07 20:34:16,517 INFO [Mon Dec 7 20:34:16 2020] End to Horizon NN Model Convert.
  2、执行 build 报错：
  [root@ce390fd69118 mapper]# sh 02_build.sh
  2020-12-07 20:34:34,537 INFO Start hb_mapper....
  2020-12-07 20:34:34,538 INFO hb_mapper version 1.1.24
  2020-12-07 20:34:34,789 INFO norm_types[i]: no_preprocess
  2020-12-07 20:34:34,795 INFO Working dir: /horizon_x3_tc/horizon_x3_tc_1.1.17e/samples/05_miscellaneous/10_test/mapper/model_output
  2020-12-07 20:34:34,795 INFO Start Model Convert....
  2020-12-07 20:34:34,795 WARNING user input_shape: [1, 32, 10], proto input_shape: [8235, 32, 10].
  2020-12-07 20:34:34,807 INFO [Mon Dec 7 20:34:34 2020] Start to Horizon NN Model Convert.
  2020-12-07 20:34:34,808 INFO Parsing the input parameter:{'input.1': {'input_shape': [1, 32, 10]}}
  2020-12-07 20:34:34,808 INFO Parsing the calibration parameter
  2020-12-07 20:34:34,890 ERROR cannot reshape array of size 2635200 into shape (32,10)
  2020-12-07 20:34:34,890 INFO [Mon Dec 7 20:34:34 2020] End to Horizon NN Model Convert.
  2020-12-07
  0
  回复
- blackfeather回复hjw:
  好的我们内部先复现一下问题然后给出结论，谢谢反馈
  2020-12-08
  0
  回复
- hjw回复blackfeather:
  好的，等你们的好消息?
  2020-12-08
  0
  回复
- anming回复hjw:
  已转合作项目 FAE跟进
  2021-01-09
  0
  回复