Quanti onnx and Arm result Strict check FAILED

2022-10-28 14:00:35,535 INFO Model Verifier Starts...

2022-10-28 14:00:35,535 INFO verifier tool version 1.9.9

2022-10-28 14:00:35,535 INFO PLEASE NOTE: Verifier tool only works for single input model

2022-10-28 14:00:35,535 INFO PLEASE NOTE: Verifier tool will only check the first model output

2022-10-28 14:00:35,697 WARNING Please note: this model has been modified, comparison results may not be correct

2022-10-28 14:00:35,697 INFO check dnn tool version on host machine

2022-10-28 14:00:36,057 INFO check dnn tool version on arm board

The authenticity of host '192.168.200.10 (192.168.200.10)' can't be established.

ECDSA key fingerprint is SHA256:8q+y1ZW0IgoK2LdZJFs/yKcbzyE5J/7gQWu58mBozgQ.

Are you sure you want to continue connecting (yes/no/[fingerprint])? yes

Warning: Permanently added '192.168.200.10' (ECDSA) to the list of known hosts.

root@192.168.200.10's password:

2022-10-28 14:00:45,493 INFO ================== input info parsed =============================

2022-10-28 14:00:45,493 INFO quantized onnx model: ./model_output/pp_nv12_quantized_model.onnx

2022-10-28 14:00:45,493 INFO bin model: ./model_output/pp_nv12.bin

2022-10-28 14:00:45,493 INFO march: bernoulli2

2022-10-28 14:00:45,494 INFO actual input type : nv12

2022-10-28 14:00:45,494 INFO actual input layout: NHWC

2022-10-28 14:00:45,494 INFO model train input data type: rgb

2022-10-28 14:00:45,494 INFO model train input data layout: NCHW

2022-10-28 14:00:45,494 INFO input shape: ['?', 384, 640, 3]

2022-10-28 14:00:45,494 INFO compare digits: 5

2022-10-28 14:00:45,494 INFO ================== input info end ==========================

2022-10-28 14:00:45,610 INFO Quanti infer starts

2022-10-28 14:00:50,346 INFO Quanti infer finish

2022-10-28 14:00:50,346 INFO Arm infer starts

2022-10-28 14:00:50,404 INFO shell file generated

root@192.168.200.10's password:

2022-10-28 14:01:10,181 INFO ================== Arm infer log start ==========================

2022-10-28 14:01:10,182 INFO + dirname /userdata/model_verifier_test/infer.sh

+ cd /userdata/model_verifier_test

+ hrt_model_exec infer --model_file=pp_nv12.bin --input_file=input.bin --enable_dump=true --dump_format=txt --dump_precision=5

hrt_model_exec infer --model_file=pp_nv12.bin --input_file=input.bin --enable_dump=true --dump_format=txt --dump_precision=5

I0000 00:00:00.000000 3468 vlog_is_on.cc:197] RAW: Set VLOG level for "*" to 3

[BPU_PLAT]BPU Platform Version(1.3.1)!

[HBRT] set log level as 0. version = 3.14.14

[DNN] Runtime version = 1.9.9_(3.14.14 HBRT)

Load model to DDR cost 159.912ms.

file length: 368640

I1028 14:01:07.655822 3468 main.cpp:758] get model handle success

I1028 14:01:07.656008 3468 main.cpp:764] get model input count success

I1028 14:01:07.656612 3468 main.cpp:780] read file success!

I1028 14:01:07.656678 3468 main.cpp:786] get model output count success

I1028 14:01:07.656823 3468 main.cpp:792] prepare output tensor success

I1028 14:01:07.657794 3468 main.cpp:812] infer success

---------------------Frame 0 begin---------------------

I1028 14:01:07.690109 3468 main.cpp:818] task done

Infer time: 33.195 ms

---------------------Frame 0 end---------------------

I1028 14:01:10.038221 3468 main.cpp:887] Dump path: ./model_infer_input_0.txt

I1028 14:01:10.103890 3468 main.cpp:971] Dump path: ./model_infer_output_0.txt

2022-10-28 14:01:10,183 INFO ================== Arm infer log end ==========================

root@192.168.200.10's password:

2022-10-28 14:01:12,787 INFO Arm infer finish

2022-10-28 14:01:12,788 INFO Sim infer starts

2022-10-28 14:01:12,805 INFO shell file generated

2022-10-28 14:01:18,383 INFO ================== Sim infer log start ==========================

2022-10-28 14:01:18,384 INFO ++ dirname /data/ping/model_verifier_output/sim_model/infer.sh

+ cd /data/ping/model_verifier_output/sim_model

+ hrt_model_exec infer --model_file=pp_nv12.bin --input_file=input.bin --enable_dump=true --dump_format=txt --dump_precision=5

hrt_model_exec infer --model_file=pp_nv12.bin --input_file=input.bin --enable_dump=true --dump_format=txt --dump_precision=5

I0000 00:00:00.000000 173 vlog_is_on.cc:197] RAW: Set VLOG level for "*" to 3

core[0] open!

core[1] open!

[HBRT] set log level as 0. version = 3.14.14

[DNN] Runtime version = 1.9.9_(3.14.14 HBRT)

Load model to DDR cost 129.673ms.

I1028 14:01:13.106722 173 main.cpp:758] get model handle success

I1028 14:01:13.106810 173 main.cpp:764] get model input count success

file length: 368640

I1028 14:01:13.107219 173 main.cpp:780] read file success!

I1028 14:01:13.107228 173 main.cpp:786] get model output count success

I1028 14:01:13.107270 173 main.cpp:792] prepare output tensor success

I1028 14:01:13.107301 173 main.cpp:812] infer success

WARNING: cnn_core_fc_avl_cap is ignored in pseudo_firmware

I1028 14:01:17.601486 173 main.cpp:818] task done

---------------------Frame 0 begin---------------------

Infer time: 4494.19 ms

---------------------Frame 0 end---------------------

I1028 14:01:18.355247 173 main.cpp:887] Dump path: ./model_infer_input_0.txt

I1028 14:01:18.376161 173 main.cpp:971] Dump path: ./model_infer_output_0.txt

2022-10-28 14:01:18,384 INFO ================== Sim infer log end ==========================

2022-10-28 14:01:18,384 INFO Sim infer finish

2022-10-28 14:01:18,384 INFO Result compare starts

2022-10-28 14:01:19,794 INFO mismatch line num: 10079

2022-10-28 14:01:19,794 INFO total line num: 10080

2022-10-28 14:01:19,794 INFO mismatch rate: 1.0

2022-10-28 14:01:19,794 WARNING Quanti onnx and Arm result Strict check FAILED

2022-10-28 14:01:21,167 INFO mismatch line num: 9920

2022-10-28 14:01:21,168 INFO total line num: 10080

2022-10-28 14:01:21,168 INFO mismatch rate: 0.984

2022-10-28 14:01:21,168 WARNING Quanti onnx and Sim result Strict check FAILED

2022-10-28 14:01:21,406 INFO mismatch line num: 1348

2022-10-28 14:01:21,407 INFO total line num: 10080

2022-10-28 14:01:21,407 INFO mismatch rate: 0.134

2022-10-28 14:01:21,407 WARNING Arm and Sim result Strict check FAILED

2022-10-28 14:01:21,407 INFO Result compare finish