组会-CBAM--transformer--ViT-DETR注意力机制.pptxVIP

注意力Attention 目录CV中的Attention机制——简单而有效的CBAM模块NLP中的self-Attention transformer将NLP的结构应用到CV中的 ViT DETR CV中的Attention机制——简单而有效的CBAM模块 1. 什么是注意力机制？注意力机制（Attention Mechanism）是机器学习中的一种数据处理方法，广泛应用在自然语言处理、图像识别及语音识别等各种不同类型的机器学习任务中。从实现的角度来讲：注意力机制通过神经网络的操作生成一个掩码mask, mask上的值一个打分，评价当前需要关注的点的评分注意力机制可以分为：通道注意力机制：对通道生成掩码mask，进行打分，代表是SEnet, Channel Attention Module空间注意力机制：对空间进行掩码的生成，进行打分，代表是Spatial Attention Module混合域注意力机制：同时对通道注意力和空间注意力进行评价打分，代表的有BAM, CBAM 2. CBAM模块的实现网络架构也很简单，一个是通道注意力模块，另一个是空间注意力模块，CBAM就是先后集成了通道注意力模块和空间注意力模块。2.1 通道注意力机制 2.2 空间注意力机制 2.3 Convolutional bottleneck attention module 3. 在什么情况下可以使用？CBAM integrated with a ResBlock in ResNet 消融实验如何更有效地计算channel attention?2.如何更有效地计算spatial attention?3.如何组织这两个部分？ 在MSCOCO数据及使用了ResNet50,ResNet101为backbone, Faster RCNN为检测器的模型进行目标检测目标检测结果 在VOC2007数据集中采用了StairNet进行了测试目标检测结果 NLP中的self-Attention transformer SequenceHard to parallel !????????Previous layerNext layer????Using CNN to replace RNN ………………Sequence Hard to parallel????????Previous layerNext layer????(CNN can parallel)……????Filters in higher layer can consider longer sequence Using CNN to replace RNN Self-Attention ????????????????Self-Attention Layer??You can try to replace any thing that has been done by RNN with self-attention. ????????????????????Self-attention???????/abs/1706.03762Attention is all you need. ???????????????????Scaled Dot-Product Attention:Self-attention??dot product?????拿每个 query q 去对每个 key k 做 attention ??????????????????Self-attention??Soft-max????????? ????????????????Self-attention?????????Considering the whole sequence? ????????????????Self-attention??????????拿每個 query q 去對每個 key k 做 attention ????Self-attention????????Self-Attention Layer? Multi-head Self-attention????(2 heads as example)???????????????????? Multi-head Self-attention????(2 heads as example)????????????????????? Multi-head Self-attention????(2 heads as example)????????????????????=??? Positional Encoding ???????+?100…………i-th dim???????+=???? source of image: http://jalam