一、 算子对比
Transfrom架构中，常用的layernorm算子，可以加速网络收敛，提出的dyt算子不仅
可以实现layernorm的这个归一化过程，同时在模型精度上会更加优秀，计算性能也更少，相关关于模型精度问题，请查阅相关论文与实验指标对比。
在J6系列中，layernorm算子被拆分如下小算子：
Dyt算子组合如下：
Layernorm默认就是限制int16量化类型，为了对齐实验，当前dyt也采用int16做量化，同时从整体来说，可以看到dyt算子很简洁，对性能是有大幅度提升的。
二、 关键代码
1. 代码模块与网络构建
三、 性能对比
1、 LayerNorm性能测试
2、 Dyt性能测试
3、对比两个性能报告，可以看到dyt的耗时降低0.7ms，当前网络的norm层不算多，对于整个模型架构来说，性能收益会更加明显，模块改造进入整体网络得工作量也不太大，如果压缩模型耗时同时不修改模型整体结构，可以从这点入手
'/%3e%3cpath%20d='M8%200.5C12.1421%200.5%2015.5%203.85786%2015.5%208C15.5%2012.1421%2012.1421%2015.5%208%2015.5C3.85786%2015.5%200.5%2012.1421%200.5%208C0.5%203.85786%203.85786%200.5%208%200.5Z'%20fill='url(%23paint1_linear_0_45845)'%20stroke='white'/%3e%3cpath%20d='M10.0597%204.5332C10.031%204.5332%2010.0036%204.54817%209.98859%204.57313L8.09724%207.79442C8.02613%207.90046%207.84897%208.23731%207.5483%208.7875C7.37863%209.0969%207.25387%209.32771%207.17152%209.48615C7.1341%209.55727%207.0293%209.53855%207.01807%209.45871C6.97191%209.11936%206.89456%208.6066%206.78727%207.91793L6.2483%204.61804C6.24207%204.57812%206.20713%204.54942%206.16721%204.54942H4.28085C4.2297%204.54942%204.18977%204.59683%204.201%204.64798L5.71059%2012.1959C5.71808%2012.2346%205.75176%2012.2621%205.79044%2012.2621H7.46845C7.49715%2012.2621%207.52335%2012.2471%207.53832%2012.2234L12.1869%204.65796C12.2205%204.60307%2012.1806%204.5332%2012.117%204.5332H10.0597Z'%20fill='white'/%3e%3cdefs%3e%3clinearGradient%20id='paint0_linear_0_45845'%20x1='8.99711'%20y1='15.3526'%20x2='47.3575'%20y2='15.3526'%20gradientUnits='userSpaceOnUse'%3e%3cstop%20stop-color='%23AF86FF'/%3e%3cstop%20offset='1'%20stop-color='%23774EFF'/%3e%3c/linearGradient%3e%3clinearGradient%20id='paint1_linear_0_45845'%20x1='1.1152'%20y1='15.1368'%20x2='15.3888'%20y2='15.1368'%20gradientUnits='userSpaceOnUse'%3e%3cstop%20stop-color='%23AF86FF'/%3e%3cstop%20offset='1'%20stop-color='%23774EFF'/%3e%3c/linearGradient%3e%3c/defs%3e%3c/svg%3e)