audiosignalintro02.ppt.pptVIP

* * * Characteristics of Speech Long-term (sentence level, several seconds) Drastic/irregular changes Short-term (frame level, 20ms or so) Regular periodic changes for voiced sounds Noise-like for unvoiced sounds Hard to recognize without context information Spectrum in Frequency-Domain Three basic characteristics in a spectrum: Timbre: Spectrum after smoothing Pitch: Distance between harmonics Intensity: Magnitude of spectrum Second formant F2 First formant F1 Pitch freq Intensity Timber Demo: Real-time Spectrogram Simulink model for real-time display of spectrogram dspstfft_audio (Before MATLAB R2011a) dspstfft_audioInput (R2012a or later) Spectrogram: Spectrum: Audio Feature Extraction Recog. Frame blocking Frame duration of 20 ms Feature extraction Volume, pitch, MFCC, LPC, etc Endpoint detection Based on volume ZCR Recognition DTW, HMM Example: Audio Feature Extraction 256 points/frame 84 points overlap 11025/(256-84)=64 feature vectors per second Zoom in Overlap Frame Three Basic Acoustic Features Three basic speech features Volume/Energy/Intensity（音量、能量、強度）：Vibration Amplitude Pitch（音高）：Fundamental frequency (which is equal to the reciprocal of the fundamental period) Timbre（音色）：The waveform within a fundamental period These features are perceived subjectively by humans. However, we can use some mathematics to “emulate” human and capture these features. Acoustic Feature: Energy Energy is the square sum of a frame, also known as intensity or volume. Characteristics: Usually noise and fricative have low energy. Energy is influence a lot by microphone setup. If we take log of square sum, and times 10, we have energy in terms of Decibel（分貝） Energy is commonly used in endpoint detection. In embedded system implementation, volume can be computed as the abs. sum of a frame in order to reduce computation. Acoustic Feature: Zero Crossing Rate Zero crossing rate (ZCR) The number of zero crossing in a frame. Characteristics： Noise and unvoiced sou