基于内容的音频信息检索概要.ppt

* Wu Gangshan: Modern Information Retrieval * (3) 音频分割的好处 音频是时基线性媒体。现在我们看到的典型音频播放接口是与磁带录音机相似的界面，具有停止、暂停、播放、快进、倒带等按钮。 为了不丢失其中的重要东西，必须从头到尾听一遍声音文件，这样要花费很多时间，即使使用“快进”，也容易丢失重要的片断，不能满足信息技术的要求。 因此，在分割的基础上，就可以结构化表示音频的内容，建立超越常规的顺序浏览界面和基于内容的音频浏览接口。 * Wu Gangshan: Modern Information Retrieval * 补充:Audio Features Features derived in the time domain: ~average energy ~zero crossing rate (ZCR): It indicates the freq of signal amplitude sign change. ~silence ratio: Thresholding may be tricky. Features derived in the freq domain ~sound spectrum【频谱】 ~bandwidth 【带宽】 : Music usually has a higher bandwidth than speech. ~energy distribution 【能量分布】 : Music usually has more high freq components than speech. =spectral centroid/brightness (midpoint of the spectral energy distribution) ~harmonicity【调和性】: Music is usually more harmonic than other sounds. ~pitch【音质】: Only period sounds give rise to a sensation of pitch. Pitch is subjective, related to but not equivalent to the fundamental freq. Spectrogram【频谱图】 It shows the relation between freq, time and intensity. Music spectrogram is more regular. Subjective features【主观特征】 Pitch【音质】, Timbre【音色】, etc. * Wu Gangshan: Modern Information Retrieval * * Wu Gangshan: Modern Information Retrieval * * Wu Gangshan: Modern Information Retrieval * * Wu Gangshan: Modern Information Retrieval * * Wu Gangshan: Modern Information Retrieval * * Wu Gangshan: Modern Information Retrieval * * Wu Gangshan: Modern Information Retrieval * Audio Classification Here we consider first speech vs. music. There’re of coz other types of sound, but their characteristics vary greatly and are difficult to generalize. Features Speech Music Bandwidth 0-7kHz 0-20kHz Spectral Centroid Low High Silence ratio High Low ZCR More variable Less variable Regular beat Non-existing Often existing * Wu Gangshan: Modern Information Retrieval * Audio Classification (cont’d) Step-by-step classification