| 17 | 0 | 21 |
| 下载次数 | 被引频次 | 阅读次数 |
点击率预测作为推荐系统中连接用户行为建模与内容分发的核心环节，直接影响个性化推荐效果以及平台资源分配效率。为解决前期研究融合LightGBM和通道注意力机制的点击率预测模型（LSNN）中挤压激励网络（SENet）因压缩操作导致的信息丢失问题，提出一种融合高效通道注意力网络（ECANet）与多头自注意力机制的LSNN改进模型（LENN）。LENN在LSNN的基础上，将SENet模块替换为ECANet模块，采用一维卷积操作代替压缩操作来保留有效信息。同时，加入多头自注意力机制，进一步增强模型捕捉数据复杂关系和模式的能力。在Criteo和MovieLens＿20M数据集上的实验结果表明，LENN在AUC和Logloss两个指标上均优于LSNN，验证了LENN具有更优的点击率预测性能。
Abstract：Click-through rate(CTR) prediction, the core link connecting user behavior modeling and content distribution in recommender systems, directly affects the personalized recommendation effect and platform resource allocation efficiency.To solve the problem of information loss caused by the compression operation of SENet in the LSNN(LightGBM-senet neural networks) model, an improved LSNN model named LENN(LightGBM-ecanet neural networks), which fuses ECANet and a multi-head self-attention mechanism, is proposed. Building upon LSNN, LENN replaces SENet with ECANet to retain the effective information by using one-dimensional convolution operation instead of compression operation, and incorporates a multi-head self-attention mechanism to strengthen the model′ s ability to capture the complex relationships and patterns of the data. Experimental results on Criteo and MovieLens＿20M datasets show that LENN outperforms LSNN in both AUC and Logloss metrics, verifying that LENN has a stronger performance in predicting CTR.
[1]龚雪鸾，陈艳姣，王帅.在线广告点击率预测方法的研究综述[J].中文信息学报，2023,37(4):1-17.
[2]吴寅琛.基于协同过滤与自注意力机制的推荐算法研究与应用[D].西安：西安电子科技大学，2023.
[3] RENDLE S. Factorization machines[C]//2010 IEEE International Conference on Data Mining. Sydney,NSW,Australia:IEEE,2010:995-1000.
[4] STECK H,BALTRUNAS L,ELAHI E,et al. Deep learning for recommender systems:a netflix case study[J]. AI Magazine,2021,42(3):7-18.
[5] GUO H F,TANG R M,YE Y M,et al. DeepFM:a factorization-machine based neural network for CTR prediction[C]//Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence. Melbourne,Australia:International Joint Conferences on Artificial Intelligence Organization,2017:1725-1731.
[6] WANG R X,FU B,FU G,et al. Deep&cross network for ad click predictions[C]//Proceedings of the ADKDD′17. Halifax,NS,Canada:ACM,2017:1-7.
[7] QU Y R,CAI H,REN K,et al. Product-based neural networks for user response prediction[C]//2016 IEEE 16th International Conference on Data Mining(ICDM). Barcelona,Spain:IEEE,2016:1149-1154.
[8] HUANG T W,ZHANG Z Q,ZHANG J L. FiBiNET:combining feature importance and bilinear feature interaction for click-through rate prediction[C]//Proceedings of the 13th ACM Conference on Recommender Systems. Copenhagen,Denmark:ACM,2019:169-177.
[9] SONG W P,SHI C C,XIAO Z P,et al. AutoInt:automatic feature interaction learning via self-attentive neural networks[C]//Proceedings of the 28th ACM International Conference on Information and Knowledge Management. Beijing China:ACM,2019:1161-1170.
[10]VASWANI A,SHAZEER N,PARMAR N,et al. Attention is all you need[EB/OL].(2017-06-12)[2024-11-02]. https：//arxiv.org/abs/1706.03762.
[11]LI Y S,WANG J P,DAI T,et al. RAT:retrieval-augmented transformer for click-through rate prediction[C]//Proceedings of the ACM Web Conference 2024. Singapore,Singapore:ACM,2024:867-870.
[12] KE G L,MENG Q,FINLEY T,et al. LightGBM:a highly efficient gradient boosting decision tree[C]//Proceedings of the 31st International Conference on Neural Information Processing Systems. Long Beach,California,USA:ACM,2017:3149-3157.
[13] HUJ,SHENL,SUNG.Squeeze-and-excitation networks[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City,UT,USA:IEEE,2018:7132-7141.
[14]高广尚.推荐系统中神经网络结合注意力机制研究综述[J].计算机工程与应用，2024,60(10):47-60.
[15] WANG Q L,WU B G,ZHU P F,et al. ECA-net:efficient channel attention for deep convolutional neural networks[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR). Seattle,WA,USA:IEEE,2020:11531-11539.
[16]LIAN J X,ZHOU X H,ZHANG F Z,et al. xDeepFM:combining explicit and implicit feature interactions for recommender systems[C]//Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery&Data Mining. London United Kingdom:ACM,2018:1754-1763.
[17] CHENG H T,KOC L,HARMSEN J,et al. Wide&deep learning for recommender systems[C]//Proceedings of the 1st Workshop on Deep Learning for Recommender Systems. Boston. MA USA:ACM,2016:7-10.
[18]马占喆.基于异维嵌入与特征交叉的展示广告点击率预估模型[D].北京：北京交通大学，2023.
基本信息:
DOI：10.19573/j.issn2095-0926.202503001
中图分类号:TP391.3
引用信息:
[1]李泽铖,武志峰.融合ECANet和多头自注意力机制的点击率预测模型[J].天津职业技术师范大学学报,2025,35(03):1-8.DOI:10.19573/j.issn2095-0926.202503001.
基金信息:
天津市自然科学基金面上项目（22JCYBJC00470）; 天津市科学普及项目（22KPXMRC00170）