专栏算法工具链模型量化没有报错，但是输出和onnx很不一样

模型量化没有报错，但是输出和onnx很不一样

已解决
fa1232022-12-06
94
15

我的模型转onnx是成功的，也验证过了

现在onnx转 .bin文件失败

遇到的问题：

1.我的模型输入是mel频谱图 ， 输入是float32 , 值域不在0-255

请问efficientnet_lite0_config.yaml （见附件）应该怎么改？

2.或者，能否不进行量化，直接将onnx转 .bin文件

03.build.sh 输出的过程如下， 貌似转成功了，但是输出和onnx 的很不一样，无法使用

022-12-06 14:52:46,872 INFO Reset batch_size=1 and execute calibration again...

2022-12-06 14:58:15,422 INFO Select max method.

2022-12-06 14:58:15,776 INFO [Tue Dec 6 14:58:15 2022] End to calibrate the model.

2022-12-06 14:58:15,777 INFO [Tue Dec 6 14:58:15 2022] Start to quantize the model.

2022-12-06 14:58:37,202 INFO [Tue Dec 6 14:58:37 2022] End to quantize the model.

2022-12-06 14:58:42,803 INFO Saving the quantized model: efficientnet_lite0_224x224_nv12_quantized_model.onnx.

2022-12-06 14:58:52,130 INFO [Tue Dec 6 14:58:52 2022] Start to compile the model with march bernoulli2.

2022-12-06 14:58:56,280 INFO Compile submodel: torch-jit-export_subgraph_0

2022-12-06 14:59:02,053 INFO hbdk-cc parameters:['--core-num', '1', '--fast', '--O3', '--input-layout', 'NHWC', '--output-layout', 'NCHW', '--input-source', 'ddr']

2022-12-06 14:59:02,111 INFO INFO: "-j" or "--jobs" is not specified, launch 16 threads for optimization

[==================================================] 100%

2022-12-06 14:59:06,362 INFO consumed time 4.27701

2022-12-06 14:59:06,838 INFO FPS=32.75, latency = 30534.8 us (see torch-jit-export_subgraph_0.html)

2022-12-06 14:59:06,844 INFO Compile submodel: torch-jit-export_subgraph_1

2022-12-06 14:59:07,695 INFO hbdk-cc parameters:['--core-num', '1', '--fast', '--O3', '--input-layout', 'NHWC', '--output-layout', 'NHWC', '--input-source', 'ddr']

2022-12-06 14:59:07,728 INFO INFO: "-j" or "--jobs" is not specified, launch 16 threads for optimization

[==================================================] 100%

2022-12-06 14:59:07,872 INFO consumed time 0.146057

2022-12-06 14:59:07,952 INFO FPS=1798.18, latency = 556.1 us (see torch-jit-export_subgraph_1.html)

2022-12-06 14:59:09,823 INFO [Tue Dec 6 14:59:09 2022] End to compile the model with march bernoulli2.

2022-12-06 14:59:09,825 INFO The converted model node information:

==================================================================================================================================

Node ON Subgraph Type Cosine Similarity Threshold

-----------------------------------------------------------------------------------------------------------------------------------

Transpose_0 CPU -- Transpose

BatchNormalization_1 CPU -- BatchNormalization 1.000000 --

Conv_3 BPU id(0) HzSQuantizedConv 0.994158 4.039474

Conv_5 BPU id(0) HzSQuantizedConv 0.988520 73.934135

AveragePool_8 BPU id(0) HzSQuantizedConv 0.991906 21.153534

Conv_9 BPU id(0) HzSQuantizedConv 0.983432 8.645608

Conv_11 BPU id(0) HzSQuantizedConv 0.984774 31.801315

AveragePool_14 BPU id(0) HzSQuantizedConv 0.987688 22.935102

Conv_15 BPU id(0) HzSQuantizedConv 0.988757 13.219603

Conv_17 BPU id(0) HzSQuantizedConv 0.986877 15.482847

AveragePool_20 BPU id(0) HzSQuantizedConv 0.987965 17.494753

Conv_21 BPU id(0) HzSQuantizedConv 0.986419 14.433695

Conv_23 BPU id(0) HzSQuantizedConv 0.984016 24.406654

AveragePool_26 BPU id(0) HzSQuantizedConv 0.980193 28.904232

Conv_27 BPU id(0) HzSQuantizedConv 0.982191 26.671112

Conv_29 BPU id(0) HzSQuantizedConv 0.980930 21.382399

AveragePool_32 BPU id(0) HzSQuantizedConv 0.982146 20.134457

Conv_33 BPU id(0) HzSQuantizedConv 0.964356 14.805241

Conv_35 BPU id(0) HzSQuantizedConv 0.980975 58.668331

AveragePool_38 BPU id(0) HzSQuantizedConv 0.980975 0.960493

ReduceMean_39 CPU -- ReduceMean 0.983716 --

ReduceMean_39_reshape CPU -- Reshape

ReduceMax_40 CPU -- ReduceMax 0.989400 --

ReduceMean_41 CPU -- ReduceMean 0.986529 --

Add_42 CPU -- Add 0.989427 --

Gemm_43_pre_reshape CPU -- Reshape

Gemm_43 BPU id(1) HzSQuantizedConv 0.998293 0.857610

Gemm_45 BPU id(1) HzSQuantizedConv 0.999911 5.329660

Gemm_45_NHWC2NCHW_LayoutConvert_Output0_reshape CPU -- Reshape

2022-12-06 14:59:09,825 INFO The quantify model output:

==========================================================================

Node Cosine Similarity L1 Distance L2 Distance Chebyshev Distance

--------------------------------------------------------------------------

Gemm_45 0.999911 0.107928 0.005794 0.441083

2022-12-06 14:59:09,827 INFO [Tue Dec 6 14:59:09 2022] End to Horizon NN Model Convert.

2022-12-06 14:59:12,512 INFO start convert to *.bin file....

2022-12-06 14:59:12,548 INFO ONNX model output num : 1

2022-12-06 14:59:12,549 INFO ############# model deps info #############

2022-12-06 14:59:12,549 INFO hb_mapper version : 1.9.9

2022-12-06 14:59:12,550 INFO hbdk version : 3.37.2

2022-12-06 14:59:12,550 INFO hbdk runtime version: 3.14.14

2022-12-06 14:59:12,551 INFO horizon_nn version : 0.14.0

2022-12-06 14:59:12,551 INFO ############# model_parameters info #############

2022-12-06 14:59:12,551 INFO onnx_model : /data/samples/ai_toolchain/horizon_model_convert_sample/03_classification/05_efficientnet_lite0_onnx/mapper/super_resolution.onnx

2022-12-06 14:59:12,551 INFO BPU march : bernoulli2

2022-12-06 14:59:12,552 INFO layer_out_dump : False

2022-12-06 14:59:12,552 INFO log_level : DEBUG

2022-12-06 14:59:12,552 INFO working dir : /data/samples/ai_toolchain/horizon_model_convert_sample/03_classification/05_efficientnet_lite0_onnx/mapper/model_output

2022-12-06 14:59:12,553 INFO output_model_file_prefix: efficientnet_lite0_224x224_nv12

2022-12-06 14:59:12,553 INFO ############# input_parameters info #############

2022-12-06 14:59:12,553 INFO ------------------------------------------

2022-12-06 14:59:12,554 INFO ---------input info : input ---------

2022-12-06 14:59:12,554 INFO input_name : input

2022-12-06 14:59:12,554 INFO input_type_rt : featuremap

2022-12-06 14:59:12,555 INFO input_space&range : regular

2022-12-06 14:59:12,555 INFO input_layout_rt : NHWC

2022-12-06 14:59:12,555 INFO input_type_train : featuremap

2022-12-06 14:59:12,555 INFO input_layout_train : NCHW

2022-12-06 14:59:12,556 INFO norm_type : no_preprocess

2022-12-06 14:59:12,556 INFO input_shape : 1x1x201x64

2022-12-06 14:59:12,556 INFO cal_data_dir : /data/samples/ai_toolchain/horizon_model_convert_sample/03_classification/05_efficientnet_lite0_onnx/mapper/calibration_data_rgb_f32

2022-12-06 14:59:12,557 INFO ---------input info : input end -------

2022-12-06 14:59:12,557 INFO ------------------------------------------

2022-12-06 14:59:12,557 INFO ############# calibration_parameters info #############

2022-12-06 14:59:12,558 INFO preprocess_on : False

2022-12-06 14:59:12,558 INFO calibration_type: : default

2022-12-06 14:59:12,558 INFO cal_data_type : float32

2022-12-06 14:59:12,559 INFO ############# compiler_parameters info #############

2022-12-06 14:59:12,559 INFO hbdk_pass_through_params: --core-num 1 --fast --O3

2022-12-06 14:59:12,559 INFO input-source : {'input': 'ddr', '_default_value': 'ddr'}

2022-12-06 14:59:12,588 WARNING Except for Quanitze/Dequantize and transpose, all other op's are treated as nchw layout by default!!!

2022-12-06 14:59:12,588 INFO NCHW to NHWC transpose node will be added for input 0 in bin model

2022-12-06 14:59:13,697 INFO Convert to runtime bin file sucessfully!

2022-12-06 14:59:13,698 INFO End Model Convert

附件:
算法工具链
评论4
0/1000
  • 颜值即正义
    Lv.2

    你好，可以使用featuremap形式的输入。转换的参考示例在ddk/samples/ai_toolchain/horizon_model_convert_sample/05_miscellaneous/02_resnet50_feature，部署的参考示例在ddk/samples/ai_toolchain/horizon_runtime_sample/code/03_misc/resnet_feature

    2022-12-07
    0
    11
    • fa123回复颜值即正义:

      收到

      2022-12-07
      1
    • fa123回复颜值即正义:

      大佬，能否帮忙解答一下 ，我的问题可能在哪？我觉得我的模型转onnx 都成功了， 后面的应该没什么问题的 ，结果。。。

      2022-12-07
      0
    • fa123回复颜值即正义:

      请问可以直接onnx转bin文件吗？ 不进行其他操作，比如量化之类的。 我的模型转onnx是完全没有问题的，我只是想要bpu加速而已。现在我转好的bin文件 输出和onnx输出差很多。

      如果不行，请问有没有更详细的资料？

      2022-12-07
      0
    • 颜值即正义回复fa123:

      您好，BPU上加速的话，是需要进行模型转换量化的，建议您尝试使用featuremap输入，资料的话，参考我们的产品手册以及OE包中的示例去开发会快很多，经过多轮迭代，工具链精度不会有问题哒~

      2022-12-07
      0
    • fa123回复颜值即正义:

      请问为什么我的配置文件也没有什么错， 量化模型的时候余弦相似度也很高， 但是最后生成的.bin文件和onnx文件输出（验证过没有问题的） 很不一样

      2022-12-08
      0
    • fa123回复fa123:

      示例和手册都看了，我都已经找不到其他方向 查找问题了，能提示一下吗？

      2022-12-08
      0
    • fa123回复fa123:

      关于输入时featuremap 的示例貌似不是很详细，请问没有能转成功的例子吗？

      2022-12-08
      0
    • fa123回复颜值即正义:

      这样的警告用不用管的？

      2022-12-08
      0
    • 颜值即正义回复fa123:

      建议您参考这个帖子进行排查哈：https://developer.horizon.ai/forumDetail/71036815603174578

      2022-12-08
      0
    • 颜值即正义回复fa123:

      是的，这个警告不用管哒~

      2022-12-08
      0
    • 颜值即正义回复fa123:

      其实本质上都是一样的哈，只是输入有一些变化而已

      2022-12-08
      0
  • fa123
    Lv.1

    比如，某一个输入是这样的：

    float32

    shape：(1, 201, 64, 1)

    最大值：23.625504 ，最小值-47.673203

    2022-12-06
    0
    0
  • fa123
    Lv.1

    onnx转.bin ，是成功的，没有报错，但是转出来的.bin文件预测时输出不对。目前怀疑是量化过程有问题。

    2022-12-06
    0
    0
  • fa123
    Lv.1

    下面的这段是从论坛里面找的，感觉会不会是padding的问题？

    我的模型输入是shape：(1, 201, 64, 1)

    配置文件也选了 featuremap

    -------------------来自论坛里面的-----------------------

    其中，featuremap输入时较为特殊，由于预测库不会对featuremap数据做padding操作，因此当您的模型输入为featuremap时，需在预处理时完成数据对齐，参考代码如下

    if (input_w == out_w) {

    memcpy(out, input, static_cast(input_h * input_w) * data_size);

    } else {

    for (int i = 0; i < input_h; i++) {

    memcpy(out, input, static_cast(input_w) * data_size);

    input += input_w;

    out += out_w;

    }

    }

    2022-12-06
    0
    0