多模态融合的家庭音乐相册自动生成.docVIP

多模态融合的家庭音乐相册自动生成 刘君芳 邵曦 南京邮电大学通信与信息工程学院 X 关注成功！ 加关注后您将方便地在 我的关注中得到本文献的被引频次变化的通知！ 新浪微博 腾讯微博 人人网 开心网 豆瓣网 网易微博 摘????要： 随着大数据以及社交网络的发展, 电子相册与在线服务成为如今人们使用计算机与互联网的基础应用.尤其是近年社交网络的流行, 电子相册的数量得到了爆炸增长, 而如何增强相册的用户体验变得尤为重要.具有某种主题的相册一般都带有一定的情感信息, 因此, 本文研究了基于多模态融合的家庭音乐相册自动生成问题, 旨在使用户能够在享受音乐的同时配以与音乐情感相同的相册图片.针对音乐与图片中所蕴含的情感, 本文在音乐和图像中分别选取能够表达其情感的句子级别的音频特征和图像特征, 然后在图像与音乐之间异构和跨模态的特征融合问题上, 采用局部保持投影 (LPP) 方法, 将图像特征与音乐特征映射到更具情感分类能力的隐式特征空间中, 实现了音乐相册的自动生成.在实验中, 客观评测结果表明, 采用LPP方法在查准率方面高于纯CCA方法;在主观评测中LPP获得72.06%的满意度, 与人工推荐的评价结果 (78.09%) 比较接近, 明显高于随机推荐和CCA方法的满意度. 关键词： 音乐相册; 情感模型; 句子级别; 多模态融合; 隐式空间; 作者简介：刘君芳, 女, 硕士生, 研究方向为多媒体信息系统与多媒体通信.ljf846344673@163.com 作者简介：邵曦 (通信作者) , 男, 博士, 副教授, 主要研究方向为多媒体信息系统与多媒体通信.shaoxi@ 收稿日期：2017-08-28 基金：国家自然科学基金 Automatic generation of family music album based on multi-modal fusion LIU Junfang SHAO Xi College of Telecommunications and Information Engineering, Nanjing University of Posts and Telecommunications; Abstract： With the development of the big data and social network, electronic albums and online services have become basic uses of computers and the Internet.Especially in recent years, the number of electronic albums has exploded with the popularity of social network.So how to improve the user experience of music album becomes particularly important.A photo album with certain topic usually has some emotion information.This paper studies the problem of automatic generation of family music album based on multi-modal fusion, so that users can enjoy music when browsing album photos with matched emotion. According to the emotions in music and images, the representative sentence-level features both for music and images are selected, and the LPP ( Locality Preserving Projection) is employed to study the relevance between the music and the images in the same emotion.The image feature and the music feature are mapped into the latent space with more emotional classification ability to realize the automatic generation of music album.In the