专栏算法工具链量化Warning:形状不同,跳过计算

量化Warning:形状不同,跳过计算

已解决
error2025-01-02
67
3

芯片型号:J6

天工开物开发包:J6_OE_3.0.22

问题定位:量化Warning

问题具体描述:

规划算法PLUTO部署工作,

其中attention模块在量化过程中显示张量形状不匹配,跳过该张量的计算。但是量化后的ONNX模型中看着张量形状没问题。

 

2025-01-02 09:16:43,730 WARNING [/agent_encoder/history_encoder/levels.0/blocks.0/attn/Gather_output_0] Different shape, skip this tensor calculation.model shape: (20, 20, 32) vs (32, 20, 20) 2025-01-02 09:16:43,731 WARNING [/agent_encoder/history_encoder/levels.0/blocks.0/attn/Mul_output_0] Different shape, skip this tensor calculation.model shape: (20, 20, 32) vs (32, 20, 20) 2025-01-02 09:16:43,732 WARNING [/agent_encoder/history_encoder/levels.0/blocks.0/attn/attn/MatMul_1_/agent_encoder/history_encoder/levels.0/blocks.0/attn/attn/MatMul_1_/agent_encoder/history_encoder/levels.0/blocks.0/attn/attn/Transpose_1_output_0_reshape_in_transpose_in] Different shape, skip this tensor calculation.model shape: (1, 32, 20, 20) vs (20, 32, 20, 1) 2025-01-02 09:16:43,732 WARNING [/agent_encoder/history_encoder/levels.0/blocks.0/attn/attn/MatMul_2_/agent_encoder/history_encoder/levels.0/blocks.0/attn/attn/MatMul_2_/agent_encoder/history_encoder/levels.0/blocks.0/attn/attn/Transpose_2_output_0_reshape_in_transpose_in] Different shape, skip this tensor calculation.model shape: (1, 32, 20, 20) vs (20, 32, 20, 1) 2025-01-02 09:16:43,734 WARNING [/agent_encoder/history_encoder/levels.0/blocks.0/attn/attn/MatMul_4_output_0] Different shape, skip this tensor calculation.model shape: (40, 20, 16) vs (1, 20, 640) 2025-01-02 09:16:43,739 WARNING [/agent_encoder/history_encoder/levels.0/blocks.1/attn/Gather_output_0] Different shape, skip this tensor calculation.model shape: (20, 20, 32) vs (32, 20, 20) 2025-01-02 09:16:43,739 WARNING [/agent_encoder/history_encoder/levels.0/blocks.1/attn/Mul_output_0] Different shape, skip this tensor calculation.model shape: (20, 20, 32) vs (32, 20, 20) 2025-01-02 09:16:43,740 WARNING [/agent_encoder/history_encoder/levels.0/blocks.1/attn/attn/MatMul_1_/agent_encoder/history_encoder/levels.0/blocks.1/attn/attn/MatMul_1_/agent_encoder/history_encoder/levels.0/blocks.1/attn/attn/Transpose_1_output_0_reshape_in_transpose_in] Different shape, skip this tensor calculation.model shape: (1, 32, 20, 20) vs (20, 32, 20, 1) 2025-01-02 09:16:43,740 WARNING [/agent_encoder/history_encoder/levels.0/blocks.1/attn/attn/MatMul_2_/agent_encoder/history_encoder/levels.0/blocks.1/attn/attn/MatMul_2_/agent_encoder/history_encoder/levels.0/blocks.1/attn/attn/Transpose_2_output_0_reshape_in_transpose_in] Different shape, skip this tensor calculation.model shape: (1, 32, 20, 20) vs (20, 32, 20, 1) 2025-01-02 09:16:43,742 WARNING [/agent_encoder/history_encoder/levels.0/blocks.1/attn/attn/MatMul_4_output_0] Different shape, skip this tensor calculation.model shape: (40, 20, 16) vs (1, 20, 640) 2025-01-02 09:16:43,750 WARNING [/agent_encoder/history_encoder/levels.1/blocks.0/attn/Gather_output_0] Different shape, skip this tensor calculation.model shape: (20, 10, 64) vs (64, 20, 10) 2025-01-02 09:16:43,750 WARNING [/agent_encoder/history_encoder/levels.1/blocks.0/attn/Mul_output_0] Different shape, skip this tensor calculation.model shape: (20, 10, 64) vs (64, 20, 10) 2025-01-02 09:16:43,751 WARNING [/agent_encoder/history_encoder/levels.1/blocks.0/attn/attn/MatMul_1_/agent_encoder/history_encoder/levels.1/blocks.0/attn/attn/MatMul_1_/agent_encoder/history_encoder/levels.1/blocks.0/attn/attn/Transpose_1_output_0_reshape_in_transpose_in] Different shape, skip this tensor calculation.model shape: (1, 64, 10, 20) vs (10, 64, 20, 1) 2025-01-02 09:16:43,751 WARNING [/agent_encoder/history_encoder/levels.1/blocks.0/attn/attn/MatMul_2_/agent_encoder/history_encoder/levels.1/blocks.0/attn/attn/MatMul_2_/agent_encoder/history_encoder/levels.1/blocks.0/attn/attn/Transpose_2_output_0_reshape_in_transpose_in] Different shape, skip this tensor calculation.model shape: (1, 64, 10, 20) vs (10, 64, 20, 1) 2025-01-02 09:16:43,753 WARNING [/agent_encoder/history_encoder/levels.1/blocks.0/attn/attn/MatMul_4_output_0] Different shape, skip this tensor calculation.model shape: (40, 10, 32) vs (1, 10, 1280) 2025-01-02 09:16:43,758 WARNING [/agent_encoder/history_encoder/levels.1/blocks.1/attn/Gather_output_0] Different shape, skip this tensor calculation.model shape: (20, 10, 64) vs (64, 20, 10) 2025-01-02 09:16:43,758 WARNING [/agent_encoder/history_encoder/levels.1/blocks.1/attn/Mul_output_0] Different shape, skip this tensor calculation.model shape: (20, 10, 64) vs (64, 20, 10) 2025-01-02 09:16:43,759 WARNING [/agent_encoder/history_encoder/levels.1/blocks.1/attn/attn/MatMul_1_/agent_encoder/history_encoder/levels.1/blocks.1/attn/attn/MatMul_1_/agent_encoder/history_encoder/levels.1/blocks.1/attn/attn/Transpose_1_output_0_reshape_in_transpose_in] Different shape, skip this tensor calculation.model shape: (1, 64, 10, 20) vs (10, 64, 20, 1) 2025-01-02 09:16:43,760 WARNING [/agent_encoder/history_encoder/levels.1/blocks.1/attn/attn/MatMul_2_/agent_encoder/history_encoder/levels.1/blocks.1/attn/attn/MatMul_2_/agent_encoder/history_encoder/levels.1/blocks.1/attn/attn/Transpose_2_output_0_reshape_in_transpose_in] Different shape, skip this tensor calculation.model shape: (1, 64, 10, 20) vs (10, 64, 20, 1) 2025-01-02 09:16:43,762 WARNING [/agent_encoder/history_encoder/levels.1/blocks.1/attn/attn/MatMul_4_output_0] Different shape, skip this tensor calculation.model shape: (40, 10, 32) vs (1, 10, 1280) 2025-01-02 09:16:43,770 WARNING [/agent_encoder/history_encoder/levels.2/blocks.0/attn/Gather_output_0] Different shape, skip this tensor calculation.model shape: (20, 5, 128) vs (128, 20, 5) 2025-01-02 09:16:43,770 WARNING [/agent_encoder/history_encoder/levels.2/blocks.0/attn/Mul_output_0] Different shape, skip this tensor calculation.model shape: (20, 5, 128) vs (128, 20, 5) 2025-01-02 09:16:43,771 WARNING [/agent_encoder/history_encoder/levels.2/blocks.0/attn/attn/MatMul_1_/agent_encoder/history_encoder/levels.2/blocks.0/attn/attn/MatMul_1_/agent_encoder/history_encoder/levels.2/blocks.0/attn/attn/Transpose_1_output_0_reshape_in_transpose_in] Different shape, skip this tensor calculation.model shape: (1, 128, 5, 20) vs (5, 128, 20, 1) 2025-01-02 09:16:43,771 WARNING [/agent_encoder/history_encoder/levels.2/blocks.0/attn/attn/MatMul_2_/agent_encoder/history_encoder/levels.2/blocks.0/attn/attn/MatMul_2_/agent_encoder/history_encoder/levels.2/blocks.0/attn/attn/Transpose_2_output_0_reshape_in_transpose_in] Different shape, skip this tensor calculation.model shape: (1, 128, 5, 20) vs (5, 128, 20, 1) 2025-01-02 09:16:43,774 WARNING [/agent_encoder/history_encoder/levels.2/blocks.0/attn/attn/MatMul_4_output_0] Different shape, skip this tensor calculation.model shape: (40, 5, 64) vs (1, 5, 2560) 2025-01-02 09:16:43,779 WARNING [/agent_encoder/history_encoder/levels.2/blocks.1/attn/Gather_output_0] Different shape, skip this tensor calculation.model shape: (20, 5, 128) vs (128, 20, 5) 2025-01-02 09:16:43,779 WARNING [/agent_encoder/history_encoder/levels.2/blocks.1/attn/Mul_output_0] Different shape, skip this tensor calculation.model shape: (20, 5, 128) vs (128, 20, 5) 2025-01-02 09:16:43,780 WARNING [/agent_encoder/history_encoder/levels.2/blocks.1/attn/attn/MatMul_1_/agent_encoder/history_encoder/levels.2/blocks.1/attn/attn/MatMul_1_/agent_encoder/history_encoder/levels.2/blocks.1/attn/attn/Transpose_1_output_0_reshape_in_transpose_in] Different shape, skip this tensor calculation.model shape: (1, 128, 5, 20) vs (5, 128, 20, 1) 2025-01-02 09:16:43,781 WARNING [/agent_encoder/history_encoder/levels.2/blocks.1/attn/attn/MatMul_2_/agent_encoder/history_encoder/levels.2/blocks.1/attn/attn/MatMul_2_/agent_encoder/history_encoder/levels.2/blocks.1/attn/attn/Transpose_2_output_0_reshape_in_transpose_in] Different shape, skip this tensor calculation.model shape: (1, 128, 5, 20) vs (5, 128, 20, 1) 2025-01-02 09:16:43,783 WARNING [/agent_encoder/history_encoder/levels.2/blocks.1/attn/attn/MatMul_4_output_0] Different shape, skip this tensor calculation.model shape: (40, 5, 64) vs (1, 5, 2560) 2025-01-02 09:16:43,790 WARNING [/agent_encoder/history_encoder/Slice_output_0] Different shape, skip this tensor calculation.model shape: (20, 128, 10, 1) vs (1, 10, 128, 20) 2025-01-02 09:16:43,791 WARNING [/agent_encoder/history_encoder/Slice_1_output_0] Different shape, skip this tensor calculation.model shape: (20, 128, 20, 1) vs (1, 20, 128, 20) 2025-01-02 09:16:43,794 WARNING [/agent_encoder/ego_state_emb/attn/Slice_4_output_0] Different shape, skip this tensor calculation.model shape: (1, 6, 1, 128) vs (6, 1, 1, 128) 2025-01-02 09:16:43,874 WARNING [/encoder_blocks.0/attn/Slice_1_output_0] Different shape, skip this tensor calculation.model shape: (1, 240, 1, 128) vs (240, 1, 1, 128) 2025-01-02 09:16:43,882 WARNING [/encoder_blocks.0/attn/MatMul_2_output_0] Different shape, skip this tensor calculation.model shape: (8, 240, 16) vs (1, 240, 128) 2025-01-02 09:16:43,894 WARNING [/encoder_blocks.1/attn/Slice_1_output_0] Different shape, skip this tensor calculation.model shape: (1, 240, 1, 128) vs (240, 1, 1, 128) 2025-01-02 09:16:43,903 WARNING [/encoder_blocks.1/attn/MatMul_2_output_0] Different shape, skip this tensor calculation.model shape: (8, 240, 16) vs (1, 240, 128) 2025-01-02 09:16:43,913 WARNING [/encoder_blocks.2/attn/Slice_1_output_0] Different shape, skip this tensor calculation.model shape: (1, 240, 1, 128) vs (240, 1, 1, 128) 2025-01-02 09:16:43,920 WARNING [/encoder_blocks.2/attn/MatMul_2_output_0] Different shape, skip this tensor calculation.model shape: (8, 240, 16) vs (1, 240, 128) 2025-01-02 09:16:43,930 WARNING [/encoder_blocks.3/attn/Slice_1_output_0] Different shape, skip this tensor calculation.model shape: (1, 240, 1, 128) vs (240, 1, 1, 128) 2025-01-02 09:16:43,937 WARNING [/encoder_blocks.3/attn/MatMul_2_output_0] Different shape, skip this tensor calculation.model shape: (8, 240, 16) vs (1, 240, 128) 2025-01-02 09:16:43,961 WARNING tensor /planning_decoder/Sin_output_0 consine compute error. 2025-01-02 09:16:43,961 WARNING tensor /planning_decoder/Unsqueeze_1_output_0 consine compute error. 2025-01-02 09:16:43,998 WARNING [/planning_decoder/decoder_blocks.0/r2r_attn/Slice_1_output_0] Different shape, skip this tensor calculation.model shape: (1, 6, 12, 128) vs (12, 128, 1, 6) 2025-01-02 09:16:44,001 WARNING [/planning_decoder/decoder_blocks.0/r2r_attn/MatMul_2_output_0] Different shape, skip this tensor calculation.model shape: (96, 6, 16) vs (1, 6, 1536) 2025-01-02 09:16:44,012 WARNING [/planning_decoder/decoder_blocks.0/m2m_attn/MatMul_4_output_0] Different shape, skip this tensor calculation.model shape: (48, 12, 16) vs (1, 12, 768) 2025-01-02 09:16:44,018 WARNING [/planning_decoder/decoder_blocks.0/cross_attn/Slice_4_output_0] Different shape, skip this tensor calculation.model shape: (1, 240, 1, 128) vs (240, 1, 1, 128) 2025-01-02 09:16:44,022 WARNING [/planning_decoder/decoder_blocks.0/cross_attn/MatMul_3_output_0] Different shape, skip this tensor calculation.model shape: (8, 72, 16) vs (1, 72, 128) 2025-01-02 09:16:44,032 WARNING [/planning_decoder/decoder_blocks.1/r2r_attn/Slice_1_output_0] Different shape, skip this tensor calculation.model shape: (1, 6, 12, 128) vs (12, 128, 1, 6) 2025-01-02 09:16:44,035 WARNING [/planning_decoder/decoder_blocks.1/r2r_attn/MatMul_2_output_0] Different shape, skip this tensor calculation.model shape: (96, 6, 16) vs (1, 6, 1536) 2025-01-02 09:16:44,046 WARNING [/planning_decoder/decoder_blocks.1/m2m_attn/MatMul_4_output_0] Different shape, skip this tensor calculation.model shape: (48, 12, 16) vs (1, 12, 768) 2025-01-02 09:16:44,052 WARNING [/planning_decoder/decoder_blocks.1/cross_attn/Slice_4_output_0] Different shape, skip this tensor calculation.model shape: (1, 240, 1, 128) vs (240, 1, 1, 128) 2025-01-02 09:16:44,056 WARNING [/planning_decoder/decoder_blocks.1/cross_attn/MatMul_3_output_0] Different shape, skip this tensor calculation.model shape: (8, 72, 16) vs (1, 72, 128) 2025-01-02 09:16:44,067 WARNING [/planning_decoder/decoder_blocks.2/r2r_attn/Slice_1_output_0] Different shape, skip this tensor calculation.model shape: (1, 6, 12, 128) vs (12, 128, 1, 6) 2025-01-02 09:16:44,070 WARNING [/planning_decoder/decoder_blocks.2/r2r_attn/MatMul_2_output_0] Different shape, skip this tensor calculation.model shape: (96, 6, 16) vs (1, 6, 1536) 2025-01-02 09:16:44,082 WARNING [/planning_decoder/decoder_blocks.2/m2m_attn/MatMul_4_output_0] Different shape, skip this tensor calculation.model shape: (48, 12, 16) vs (1, 12, 768) 2025-01-02 09:16:44,088 WARNING [/planning_decoder/decoder_blocks.2/cross_attn/Slice_4_output_0] Different shape, skip this tensor calculation.model shape: (1, 240, 1, 128) vs (240, 1, 1, 128) 2025-01-02 09:16:44,092 WARNING [/planning_decoder/decoder_blocks.2/cross_attn/MatMul_3_output_0] Different shape, skip this tensor calculation.model shape: (8, 72, 16) vs (1, 72, 128) 2025-01-02 09:16:44,103 WARNING [/planning_decoder/decoder_blocks.3/r2r_attn/Slice_1_output_0] Different shape, skip this tensor calculation.model shape: (1, 6, 12, 128) vs (12, 128, 1, 6) 2025-01-02 09:16:44,107 WARNING [/planning_decoder/decoder_blocks.3/r2r_attn/MatMul_2_output_0] Different shape, skip this tensor calculation.model shape: (96, 6, 16) vs (1, 6, 1536) 2025-01-02 09:16:44,119 WARNING [/planning_decoder/decoder_blocks.3/m2m_attn/MatMul_4_output_0] Different shape, skip this tensor calculation.model shape: (48, 12, 16) vs (1, 12, 768) 2025-01-02 09:16:44,126 WARNING [/planning_decoder/decoder_blocks.3/cross_attn/Slice_4_output_0] Different shape, skip this tensor calculation.model shape: (1, 240, 1, 128) vs (240, 1, 1, 128) 2025-01-02 09:16:44,130 WARNING [/planning_decoder/decoder_blocks.3/cross_attn/MatMul_3_output_0] Different shape, skip this tensor calculation.model shape: (8, 72, 16) vs (1, 72, 128)

算法工具链
征程6
评论3
0/1000
  • Huanghui
    Lv.5

    你好,下次反馈问题时请说明一下问题产生的背景哈,就是你在那个版本上做什么操作时遇到的问题,然后把能提供的日志、文件附上。

    针对当前问题,目前的判断是:

    1. 该问题是在使用hb_verifer验证两个模型的一致性是产生的。

    2. 问题原因是针对参与对比的两个模型的不同输出,进行同名输出的tensor余弦相似度对比时发现, 同名输出的shape不一致。你可以可视化看看参与对比的两个模型的同名输出的shape情况。

    2025-01-03
    0
    0
  • Huanghui
    Lv.5

    以下是从代码中提取到的参与对比的同名输出的对比时的一些特点可供参考:

    1.另个模型中name和name + "_quantized"可以认为是同名。

    2.对于同名输出如果一个模型的输出是[1, sub_shape],另一个模型是[n, sub_shape]的情况,对比之前会先进行数据扩展 [1, sub_shape] ->[n, sub_shape] ,然后再进行对比。

    3.对于同名输出如果存在layout不一致的情况,工具会先进行layout的变换在进行比较,工具是支持的。

    2025-01-03
    0
    0
  • Huanghui
    Lv.5

    客户您好,长时间未收到你的答复,相信问题已解。如对此尚存疑问欢迎新帖讨论,感谢您的参与!

    2025-01-10
    0
    0