基于多模态特征的视频语义镜头标注-信号与信息处理专业论文.docxVIP

摘 摘 要 摘 要 视频的镜头类型是视频底层特征和高级语义之间的桥梁，能有效弥补两者之 间的鸿沟，有助于视频精彩事件检测，促进创建视频摘要，进而满足用户快速查 询和浏览视频的要求。现有视频语义镜头标注方法大多基于视频的视觉特征，未 充分利用多模态特征，浪费了数据资源，标注效果也有待提升。本文提出了基于 多模态特征的视频语义镜头标注方法，并应用于足球视频，主要工作有： （1）分析并提取了足球视频的多模态特征。包括镜头关键帧的视觉特征—草 色块个数、人脸区域参数、边缘、纹理和镜头的摄像机运动特征；镜头对应音频 信号的平均短时能量、短时过零率、音频帧能量序列的自相关函数特征。同时， 本文提出了累积方差法，并结合连通域分析技术提取了足球视频记分牌中的时钟 数字，采用支持向量机(Support Vector Machine, SVM)对提取出的时钟数字进行了 识别，用关键词匹配方法提取了足球视频网络直播文本的文本事件及对应的时间 标签，并将文本/视频的时间进行了对准。实验结果证明了特征提取方法的正确性。 （2）分别将提取出的一部分视频镜头的视觉特征矢量集和多模态特征矢量集 作为训练数据，输入分类器，采用决策树 C4.5 挖掘特征矢量和镜头类型之间的潜 在关系，即建立语义镜头标注的模型，并用该模型对新的足球视频镜头集进行了 语义标注。实验结果证明，本文基于多模态特征的视频语义镜头标注方法比基于 视觉特征的标注方法性能好。 最后总结了本文的研究内容，展望了未来的研究方向。 关键词：多模态特征 镜头特征矢量 语义镜头标注 文本/视频时间对准 Ab Abstract Abstract Video shot type effectively bridges the gap between low-level features and high- level semantics of the video. It helps to detect the highlights, and create the video summary. With the result that, the users could query and brows the video easily. The most of the existing approaches for annotating video semantic shots are on the basis of visual features, without making full use of multi- modal features of the video. Thus, the data resource is wasted and the annotation result needs to be improved. Therefore, this paper proposes an annotating approach based on the multi- modal features. The main contributions of this paper lie in the following aspects: Analyzing and extracting the visual features of the soccer video, including the grass-block amount, the face parameters, edge pixels rate, texture and camera motion, and calculating the audio features, including average shot-time energy, shot-time zero-crossing rate, and autocorrelation function of the audio frame energy sequence. Detecting the clock number in the soccer video scoreboard by using the connected domain analysis and the accumulation variance concept proposed in this paper, identifying the extracted clock number by using the SVM(Support Vector Machine, SVM), and detecting the text event and its corresponding time tag by m