前言本文介绍了局部像素关系对齐上采样模块LPRMAlignUpModule在YOLO11中的结合应用。该模块通过压缩特征预测局部像素关系并利用不同膨胀率的动态关系对跨尺度特征进行对齐与细化增强上采样过程中的局部结构表达能力。我们将LPRMAlignUpModule集成到YOLO11的Neck多尺度特征融合路径中以替换对应的Concat融合位置并进行相关注册和配置用于改善上采样特征与侧向特征之间的空间对齐和细节融合。文章目录 YOLO11改进大全卷积层、轻量化、注意力机制、损失函数、Backbone、SPPF、Neck、检测头全方位优化汇总专栏链接: YOLO11改进专栏介绍高空间频率信息包括纹理等细节对语义分割的准确性有重要贡献。然而根据奈奎斯特—香农采样定理在步幅卷积等下采样层传播时高频成分容易受到混叠或失真的影响。为此我们提出了一种新的空间频率调制SFM方法在下采样前将高频特征调制到较低频率并在上采样过程中将其重新解调回来。具体而言我们通过自适应重采样ARS实现调制并设计了一种轻量级附加模块对高频区域进行密集采样以放大信号从而依据频率缩放特性降低其频率。我们还提出了多尺度自适应上采样MSAU通过非均匀上采样对调制后的特征进行解调并恢复高频信息。该模块进一步通过显式利用不同尺度下密集重采样区域与稀疏重采样区域之间的信息交互来改善分割效果。上述两个模块都能够无缝集成到不同架构中适用范围从卷积神经网络扩展到Transformer。特征可视化与分析表明我们的方法能够有效缓解混叠同时在解调后成功保留细节。最后我们将SFM的适用性和有效性扩展验证到图像分类、对抗鲁棒性、实例分割和全景分割等任务中。代码已公开在https://github.com/Linwei-Chen/SFM。文章链接论文地址https://arxiv.org/pdf/2507.11893代码地址https://github.com/search?qrepo%3ALinwei-Chen%2FSFMLPRMAlignUpModuletypecode基本原理1. 解决的关键问题YOLO11的Neck通常通过上采样与Concat完成不同尺度特征的融合。普通双线性或最近邻上采样使用固定空间采样规则难以针对局部纹理、边缘和目标轮廓进行自适应细化当高层语义特征与低层细节特征进行跨尺度融合时还可能出现空间错位、细节稀释和局部结构恢复不足的问题。LPRMAlignUpModule针对这一过程引入由压缩特征动态预测的局部像素关系使输出不再只是固定插值或直接拼接而是根据当前特征内容重新聚合邻域信息。2. 整体架构模块接收两路Neck特征低分辨率路径特征和高分辨率侧向引导特征。首先使用1×1卷积将两路输入投影到目标输出通道并把空间尺寸对齐到高分辨率特征随后分别压缩两路特征并相加形成用于预测局部像素关系的条件特征。条件特征经过关系建模后先后送入两个不同膨胀率的局部像素关系细化单元对低分辨率路径进行动态邻域重采样最后与高分辨率引导特征进行残差式融合并通过输出投影得到YOLO11后续C3k2所需的特征。在YOLO11中模块放置在Neck的四个跨尺度融合位置P5到P4、P4到P3、P3回流到P4以及P4回流到P5。这样既保留了YOLO11的自顶向下和自底向上路径又把原来的Concat替换为带局部关系建模的对齐融合操作。3. 技术原理模块使用一个小通道压缩分支生成关系条件图再由3×3卷积预测每个空间位置的k²个邻域权重并在邻域维度进行Softmax归一化。对于输入特征的每个位置使用这些权重对膨胀邻域展开结果进行加权求和从而完成动态局部像素关系聚合。第一阶段使用膨胀率1建模近邻关系第二阶段使用膨胀率2扩大局部感受野使模块能够同时利用细粒度边缘信息和稍大范围的上下文信息。与固定插值相比这种方式能够根据局部内容改变采样邻域的贡献与直接Concat相比它先完成通道投影、空间对齐和关系细化再输出统一通道数的融合特征。当前YOLO11适配版保留了LPRM的动态局部关系核心并将原论文面向非均匀坐标的上采样过程改造成适合YOLO11多尺度Neck输入输出契约的可运行模块。核心代码classLPRMAlignUpModule(nn.Module):def__init__(self,low_res_channels,high_res_channels,kernel_size3,align_cornersFalse,compress_ratio4):super().__init__()self.align_cornersalign_corners predictor_in_channelslow_res_channelshigh_res_channels compressed_channelspredictor_in_channels//compress_ratio self.compress_conv_lownn.Sequential(nn.Conv2d(low_res_channels,compressed_channels,1,biasTrue))self.compress_conv_highnn.Sequential(nn.Conv2d(high_res_channels,compressed_channels,1,biasTrue))self.lprmLPRM_efficient(channelscompressed_channels,kernel_sizekernel_size,upscale2)self.lpr_convnn.Conv2d(compressed_channels,kernel_size**2,3,padding1)defforward(self,x_low,guidance_high_aligned):_,_,target_h,target_wguidance_high_aligned.shape compressed_lowself.compress_conv_low(x_low)compressed_highself.compress_conv_high(guidance_high_aligned)lpr_featF.interpolate(self.lpr_conv(compressed_low),scale_factor2,modebilinear,align_cornersself.align_corners)F.interpolate(self.lpr_conv(compressed_high),size(x_low.size(-2)*2,x_low.size(-1)*2),modebilinear,align_cornersself.align_corners)x_lowself.lprm(lpr_feat,x_low)returnF.interpolate(x_low,size(target_h,target_w),modebilinear,align_cornersself.align_corners)YOLO11引入代码在根目录下的ultralytics/nn/目录新建一个featureFusion目录然后新建一个以LPRMAlignUpModule.py为文件名的py文件 把代码拷贝进去。from__future__importannotationsfromcollections.abcimportSequenceimporttorchimporttorch.nnasnnimporttorch.nn.functionalasFfromultralytics.nn.modules.convimportConvclass_LocalPixelRelation(nn.Module):Apply dynamic k-by-k local relations to a BCHW feature map.def__init__(self,channels:int,kernel_size:int3)-None:super().__init__()ifkernel_size1orkernel_size%20:raiseValueError(kernel_size must be a positive odd integer)self.channelsint(channels)self.kernel_sizeint(kernel_size)self.paddingself.kernel_size//2defforward(self,relation_logits:torch.Tensor,value:torch.Tensor)-torch.Tensor:ifrelation_logits.ndim!4orvalue.ndim!4:raiseValueError(relation_logits and value must be 4D BCHW tensors)b,_,h,wvalue.shape k2self.kernel_size*self.kernel_sizeifrelation_logits.shape[1]!k2:raiseValueError(fexpected{k2}relation channels, got{relation_logits.shape[1]})ifrelation_logits.shape[-2:]!(h,w):relation_logitsF.interpolate(relation_logits,size(h,w),modebilinear,align_cornersFalse)weightsrelation_logits.reshape(b,1,k2,h,w).softmax(dim2)paddedF.pad(value,(self.padding,)*4,modereplicate)patchesF.unfold(padded,kernel_sizeself.kernel_size,padding0,stride1,)patchespatches.reshape(b,self.channels,k2,h,w)return(patches*weights).sum(dim2)classLPRMAlignUpModule(nn.Module):def__init__(self,in_channels:Sequence[int],out_channels:int,kernel_size:int3,align_corners:boolFalse,compress_ratio:int4,)-None:super().__init__()low_res_channels,high_res_channels(int(c)forcinin_channels)self.low_res_channelslow_res_channels self.high_res_channelshigh_res_channels self.out_channelsint(out_channels)self.align_cornersbool(align_corners)predictor_in_channelslow_res_channelshigh_res_channels compressed_channelsmax(8,predictor_in_channels//int(compress_ratio))self.compressed_channelscompressed_channels self.compress_conv_lownn.Conv2d(low_res_channels,compressed_channels,kernel_size1,biasTrue)self.compress_conv_highnn.Conv2d(high_res_channels,compressed_channels,kernel_size1,biasTrue)self.lpr_convnn.Conv2d(compressed_channels,kernel_size*kernel_size,kernel_size3,padding1)self.lprm_LocalPixelRelation(low_res_channels,kernel_sizekernel_size)self.guidance_to_low(nn.Identity()ifhigh_res_channelslow_res_channelselsenn.Conv2d(high_res_channels,low_res_channels,kernel_size1,biasFalse))self.conv_finalConv(low_res_channels,self.out_channels,1)defforward(self,inputs:Sequence[torch.Tensor])-torch.Tensor:x_low,x_highinputs target_sizex_high.shape[-2:]relation_lowself.lpr_conv(self.compress_conv_low(x_low))relation_highself.lpr_conv(self.compress_conv_high(x_high))ifrelation_low.shape[-2:]!target_size:relation_lowF.interpolate(relation_low,sizetarget_size,modebilinear,align_cornersself.align_corners)relation_logitsrelation_lowrelation_highifx_low.shape[-2:]!target_size:x_lowF.interpolate(x_low,sizetarget_size,modebilinear,align_cornersself.align_corners)aligned_lowself.lprm(relation_logits,x_low)guidanceself.guidance_to_low(x_high)fusedaligned_lowguidancereturnself.conv_final(fused)注册在ultralytics/nn/tasks.py中进行如下操作步骤1:fromultralytics.nn.neck.LPRMAlignUpModuleimportLPRMAlignUpModule步骤2修改def parse_model(d, ch, verboseTrue):elifmisLPRMAlignUpModule:c1[ch[x]forxinf]c2make_divisible(min(args[0],max_channels)*width,8)args[c1,c2,*args[1:]]配置yolo11-LPRMAlignUpModule.yaml# Ultralytics YOLO , AGPL-3.0 license# YOLO11 object detection model with P3-P5 outputs. For Usage examples see https://docs.ultralytics.com/tasks/detect# Parametersnc:80# number of classesscales:# model compound scaling constants, i.e. modelyolo11n.yaml will call yolo11.yaml with scale n# [depth, width, max_channels]n:[0.50,0.25,1024]s:[0.50,0.50,1024]m:[0.50,1.00,512]l:[1.00,1.00,512]x:[1.00,1.50,512]# YOLO11n backbonebackbone:# [from, repeats, module, args]-[-1,1,Conv,[64,3,2]]# 0-P1/2-[-1,1,Conv,[128,3,2]]# 1-P2/4-[-1,2,C3k2,[256,False,0.25]]-[-1,1,Conv,[256,3,2]]# 3-P3/8-[-1,2,C3k2,[512,False,0.25]]-[-1,1,Conv,[512,3,2]]# 5-P4/16-[-1,2,C3k2,[512,True]]-[-1,1,Conv,[1024,3,2]]# 7-P5/32-[-1,2,C3k2,[1024,True]]-[-1,1,SPPF,[1024,5]]# 9-[-1,2,C2PSA,[1024]]# 10# YOLO11n headhead:-[-1,1,nn.Upsample,[None,2,nearest]]-[[-1,6],1,LPRMAlignUpModule,[512]]# 12: P4 fusion-[-1,2,C3k2,[512,False]]# 13-[-1,1,nn.Upsample,[None,2,nearest]]-[[-1,4],1,LPRMAlignUpModule,[256]]# 15: P3 fusion-[-1,2,C3k2,[256,False]]# 16 (P3/8-small)-[-1,1,Conv,[256,3,2]]-[[-1,13],1,LPRMAlignUpModule,[512]]# 18: head P4 fusion-[-1,2,C3k2,[512,False]]# 19 (P4/16-medium)-[-1,1,Conv,[512,3,2]]-[[-1,10],1,LPRMAlignUpModule,[1024]]# 21: head P5 fusion-[-1,2,C3k2,[1024,True]]# 22 (P5/32-large)-[[16,19,22],1,Detect,[nc]]# Detect(P3, P4, P5)实验脚本importwarnings warnings.filterwarnings(ignore)fromultralyticsimportYOLOif__name____main__:# 修改为自己的配置文件地址modelYOLO(./ultralytics/cfg/models/11/yolo11-LPRMAlignUpModule.yaml)# 修改为自己的数据集地址model.train(data./ultralytics/cfg/datasets/coco8.yaml,cacheFalse,imgsz640,epochs10,single_clsFalse,# 是否是单类别检测batch8,close_mosaic10,workers0,optimizerSGD,ampTrue,projectruns/train,nameLPRMAlignUpModule,)结果
