专栏算法工具链转换百度ocr rec 模块后，预测输出都是 blank

转换百度ocr rec 模块后，预测输出都是 blank

已解决
sunkw2021-07-19
174
7
你好，希望使用x3做ocr预测，先使用 paddle2onnx https://github.com/PaddlePaddle/Paddle2ONNX 将 ch_ppocr_mobile_v2.0_rec_infer https://github.com/PaddlePaddle/PaddleOCR 转换为 onnx 模型，使用 hb_mapper checker 检查没有错误或警告提示，使用 makebin 程序几个警告，但还是生成了 .bin 文件，看最后一层的相似度也很高，但预测时，返回的 ctc 解码结果总是 blank。希望知道在 makebin 中的那几个警告是否对最终结果产生了影响？

ocr 的 det 模型预测结果是对的，能准确的定位文字的位置。

原模型中的 HardSigmoid 不支持，在转换到 onnx 时，转换为 Sigmoid 了。

以下为 makebin 的输出：
2021-07-19 09:38:14,549 INFO Start hb_mapper....

2021-07-19 09:38:14,549 INFO log will be stored in /media/nas/work/horizon/horizon_x3_tc_1.1.19e/samples/09_zonekey/00_models/ocr/rec/hb_mapper_makertbin.log

2021-07-19 09:38:14,549 INFO hbdk version 3.19.5

2021-07-19 09:38:14,549 INFO horizon_nn version 0.10.2

2021-07-19 09:38:14,549 INFO hb_mapper version 1.3.8

2021-07-19 09:38:14,549 INFO Start Model Convert....

2021-07-19 09:38:14,557 INFO validating model_parameters...

2021-07-19 09:38:14,557 INFO validating model_parameters finished

2021-07-19 09:38:14,558 INFO validating input_parameters...

2021-07-19 09:38:14,558 INFO input num is set to 1 according to input_type_rt

2021-07-19 09:38:14,558 INFO validating input_parameters finished

2021-07-19 09:38:14,558 INFO validating calibration_parameters...

2021-07-19 09:38:14,558 INFO validating calibration_parameters finished

2021-07-19 09:38:14,558 INFO validating custom_op...

2021-07-19 09:38:14,558 INFO validating custom_op finished

2021-07-19 09:38:14,558 INFO setting log file level to INFO

2021-07-19 09:38:14,571 INFO Model has 1 inputs: ['x']

2021-07-19 09:38:14,571 INFO model input shape missing, using shape from model file: [1, 3, 32, 100]

2021-07-19 09:38:14,574 INFO *******************************************

2021-07-19 09:38:14,577 INFO First calibration picture name: {052B82B9-040F-4B29-BFD8-2B32DB382A6E}

2021-07-19 09:38:14,578 INFO *******************************************

2021-07-19 09:38:14,626 INFO [Mon Jul 19 09:38:14 2021] Start to Horizon NN Model Convert.

2021-07-19 09:38:14,627 INFO Parsing the input parameter:{'x': {'input_shape': [1, 3, 32, 100], 'original_input_layout': 'NCHW'}}

2021-07-19 09:38:14,627 INFO Parsing the calibration parameter

2021-07-19 09:38:14,627 INFO Parsing the hbdk parameter:{'hbdk_pass_through_params': '--core-num 1 --fast --O2', 'input-source': {'x': 'ddr', '_default_value': 'ddr'}}

2021-07-19 09:38:14,627 INFO HorizonNN version: 0.10.2

2021-07-19 09:38:14,627 INFO HBDK version: 3.19.5

2021-07-19 09:38:14,628 INFO [Mon Jul 19 09:38:14 2021] Start to parse the onnx model.

2021-07-19 09:38:14,672 INFO ONNX model info:

ONNX IR version: 7

Opset version: 11

Input name: x, [1, 3, 32, 100]

2021-07-19 09:38:14,724 INFO [Mon Jul 19 09:38:14 2021] End to parse the onnx model.

2021-07-19 09:38:14,725 INFO Model input names: ['x']

2021-07-19 09:38:15,137 INFO Saving the original float model: baidu_ocr_rec_original_float_model.onnx.

2021-07-19 09:38:15,138 INFO [Mon Jul 19 09:38:15 2021] Start to optimize the model.

2021-07-19 09:38:15,493 INFO [Mon Jul 19 09:38:15 2021] End to optimize the model.

2021-07-19 09:38:15,941 INFO Saving the optimized model: baidu_ocr_rec_optimized_float_model.onnx.

2021-07-19 09:38:15,941 INFO [Mon Jul 19 09:38:15 2021] Start to calibrate the model.

2021-07-19 09:38:15,986 INFO There are 9 samples in the calibration data set.

2021-07-19 09:38:16,447 INFO Run calibration model with kl method.

2021-07-19 09:38:21.775200861 [E:onnxruntime:, sequential_executor.cc:183 Execute] Non-zero status code returned while running Reshape node. Name:'reshape_LSTM_0_853' Status Message: /home/jenkins/workspace/model_convert/onnxruntime/onnxruntime/core/providers/cpu/tensor/reshape_h

elper.h:43 onnxruntime::ReshapeHelper::ReshapeHelper(const onnxruntime::TensorShape&, std::vector<long int>&) gsl::narrow_cast<int64_t>(input_shape.Size()) == size was false. The input tensor cannot be reshaped to the requested shape. Input shape:{400,48,1,1}, requested shape:{1,96,1,25}

2021-07-19 09:38:21,775 INFO Reset batch_size=1 and execute calibration again...

2021-07-19 09:39:22,859 WARNING find illegal threshold in relu_2.tmp_0, it will be set default.

2021-07-19 09:39:22,915 INFO [Mon Jul 19 09:39:22 2021] End to calibrate the model.

2021-07-19 09:39:22,919 INFO [Mon Jul 19 09:39:22 2021] Start to quantize the model.

2021-07-19 09:39:30,031 INFO [Mon Jul 19 09:39:30 2021] End to quantize the model.

2021-07-19 09:39:31,223 INFO Saving the quantized model: baidu_ocr_rec_quantized_model.onnx.

2021-07-19 09:39:31,224 INFO [Mon Jul 19 09:39:31 2021] Start to compile the model with march bernoulli2.

2021-07-19 09:39:31,445 INFO Compile submodel: paddle-onnx_subgraph_0

2021-07-19 09:39:31,637 INFO hbdk-cc parameters:['--core-num', '1', '--fast', '--O2', '--input-layout', 'NHWC', '--output-layout', 'NCHW', '--input-source', 'ddr']

[==================================================] 100%

2021-07-19 09:40:13,299 INFO Compile submodel: paddle-onnx_subgraph_1

2021-07-19 09:40:13,471 INFO hbdk-cc parameters:['--core-num', '1', '--fast', '--O2', '--input-layout', 'NHWC', '--output-layout', 'NCHW', '--input-source', 'ddr']

[==================================================] 100%

2021-07-19 09:40:55,382 INFO [Mon Jul 19 09:40:55 2021] End to compile the model with march bernoulli2.

2021-07-19 09:40:55,404 INFO The converted model node information:

======================================================================================================================================

Node ON Subgraph Type Cosine Similarity Threshold

--------------------------------------------------------------------------------------------------------------------------------------

Conv_0 BPU id(0) HzSQuantizedConv 0.999107 0.299316

Mul_0 BPU id(0) HzLut 0.998763 4.166700

Conv_1 BPU id(0) HzSQuantizedConv 0.997243 4.166700

Conv_2 BPU id(0) HzSQuantizedConv 0.995672 2.583919

GlobalAveragePool_0_SPLIT_WITH_DEPTHWISE_CONV_0 BPU id(0) HzSQuantizedConv 0.999357 5.099117

GlobalAveragePool_0_SPLIT_WITH_DEPTHWISE_CONV_1 BPU id(0) HzSQuantizedConv 0.999610 3.494113

GlobalAveragePool_0 BPU id(0) HzSQuantizedConv 0.999615 3.011065

Conv_3 BPU id(0) HzSQuantizedConv 1.000000 2.996608

Conv_4 BPU id(0) HzSQuantizedConv 0.999897 1.000000

Sigmoid_0 BPU id(0) HzLut 0.999988 5.498970

....

....

reshape_LSTM_4_853 CPU -- Reshape

Transpose_5 CPU -- Transpose

Reshape_19 CPU -- Reshape

Transpose_6 CPU -- Transpose

MatMul_0 CPU -- MatMul

Add_43 CPU -- Add 0.870400 --

Softmax_0 CPU -- Softmax 0.999749 --

2021-07-19 09:40:55,421 INFO [Mon Jul 19 09:40:55 2021] End to Horizon NN Model Convert.

2021-07-19 09:40:55,528 INFO start convert to *.bin file....

2021-07-19 09:40:55,614 INFO ONNX model output num: 1

2021-07-19 09:40:55,614 INFO ########################################

2021-07-19 09:40:55,614 INFO ----------- dependency info ------------

2021-07-19 09:40:55,614 INFO hb mapper version: 1.3.8

2021-07-19 09:40:55,614 INFO hbdk version: 3.19.5

2021-07-19 09:40:55,614 INFO hbdk runtime version: 3.12.6

2021-07-19 09:40:55,614 INFO horizon_nn version: 0.10.2

2021-07-19 09:40:55,615 INFO -------- model parameters info ---------

2021-07-19 09:40:55,615 INFO onnx_model: /media/nas/work/horizon/horizon_x3_tc_1.1.19e/samples/09_zonekey/00_models/ocr/rec/baidu_ocr_rec.onnx

2021-07-19 09:40:55,615 INFO march: bernoulli2

2021-07-19 09:40:55,615 INFO layer_out_dump: False

2021-07-19 09:40:55,615 INFO log_level: INFO

2021-07-19 09:40:55,615 INFO working_dir: /media/nas/work/horizon/horizon_x3_tc_1.1.19e/samples/09_zonekey/00_models/ocr/rec/bin

2021-07-19 09:40:55,615 INFO model_prefix: baidu_ocr_rec

2021-07-19 09:40:55,615 INFO output_nodes: []

2021-07-19 09:40:55,615 INFO -------- input_parameters info ---------

2021-07-19 09:40:55,615 INFO ------------ input info : x -----------

2021-07-19 09:40:55,615 INFO --input_name: x

2021-07-19 09:40:55,615 INFO --input_type_rt: featuremap

2021-07-19 09:40:55,615 INFO --input_type_train: featuremap

2021-07-19 09:40:55,615 INFO --input_layout_rt: NCHW

2021-07-19 09:40:55,615 INFO --input_layout_train: NCHW

2021-07-19 09:40:55,615 INFO --norm_type: no_preprocess

2021-07-19 09:40:55,616 INFO --input_shape: 1x3x32x100

2021-07-19 09:40:55,616 INFO ----------------------------------

2021-07-19 09:40:55,616 INFO -------- calibration parameters info ---------

2021-07-19 09:40:55,616 INFO calibration_data_dir: /media/nas/work/horizon/horizon_x3_tc_1.1.19e/samples/09_zonekey/00_models/ocr/rec/feats

2021-07-19 09:40:55,616 INFO preprocess_on: False

2021-07-19 09:40:55,616 INFO calibration_type: kl

2021-07-19 09:40:55,616 INFO per_channel: False

2021-07-19 09:40:55,616 INFO max_percentile: 1.0

2021-07-19 09:40:55,616 INFO ------------ compiler_parameters info -------------

2021-07-19 09:40:55,616 INFO hbdk_pass_through_params: --core-num 1 --fast --O2

2021-07-19 09:40:55,616 INFO input-source: {'x': 'ddr', '_default_value': 'ddr'}

2021-07-19 09:40:55,616 INFO ########################################

2021-07-19 09:40:57,396 INFO Convert to runtime bin file sucessfully!

2021-07-19 09:40:57,396 INFO End Model Convert

算法工具链
评论3
0/1000
  • admin
    Lv.1

    1.转换后的onnx模型是否可识别出具体的文字？

    2.校验数据如何选取的呢？可以参考我们的FAQ玩文档进行精度调优。

    2021-07-19
    0
    3
    • sunkw回复admin:

      我测试一下转换后的 onnx，稍等

      2021-07-19
      0
    • sunkw回复sunkw:
      import cv2
      import onnxruntime as nxrun
      import numpy as np

      img = cv2.imread("./slice1.jpg", 1)
      img = cv2.resize(img, (100, 32))
      img = img.astype(np.float32)
      img /= 255.0
      img -= 0.5
      img /= 0.5
      im = np.transpose(img, (2, 0, 1))
      im = im[NULL, ...]
      sess = nxrun.InferenceSession("./baidu_ocr_rec.onnx")

      inp_name = sess.get_inputs()[0].name
      out_name = sess.get_outputs()[0].name

      result = sess.run(NULL, {inp_name: im})
      out = result[0]
      out = np.argmax(out, axis=2)
      out = out[np.where(out > 0)]
      print(out)

      是可以预测得到 slices1.jpg 中的文本的

      2021-07-19
      0
    • sunkw回复sunkw:

      slice1.jpg

      2021-07-19
      0
  • sunkw
    Lv.1

    已发邮件，包含百度转换后的 onnx，.yaml，和 checker, makebin 的日志文件，谢谢 ：）

    2021-07-19
    0
    1
    • admin回复sunkw:

      收到，会尽快转给研发，让其分析并回复。

      2021-07-19
      0
  • admin
    Lv.1
    您好，可以把相关的工程文件打包好，发送到邮箱forum-bd@horizon.ai，感谢您提供的资料
    2021-07-19
    0
    0