Create Photo-Realistic Talking Face精品分析.pptVIP

More video results * More video results * Conclusion and Future Work Pose tracking and lip motion tracking Size of the train database Talking face with expression Real-time generation? Fast modeling for different person * Animation * Thank you * Create Photo-Realistic Talking Face Changbo Hu 2001.11.26 *This work was done during visiting Microsoft Research China with Baining Guo and Bo Zhang * Outline Introduction of talking face Motivations System overview Techniques Conclusions * Introduction What is a talking face Face (lip) animation, driven by voice Applications The process of talking face Face model Motion capture Mapping between audio and video Rendering, Photo-realistic? * Literatures Walter,93, DecFace, 2Dwire frame model Terzopoulos,95, Skin and muscle model Breglar,97, Video Rewrite, Sample image based TS Huang,98,Mesh model from range data Poggio,98, MikeTalk, Viseme morphing Guenter,99, Making face, 3D from multicamera Zhengyou Zhang, 00, 3D face modeling from video through epipolar constraint Cosatto,00, Planar quads model * Some Face models * Motivations Aim: a graphics interface for conversation agent Photo-realistic Driven by Chinese Smooth connection between sentences Extended from “Video rewrite” * System overview:Pipeline of the system(1) Video with Sound Images Sound Pose tracking Phoneme segmentation Annotation Lip motion Tracking Train database * System overview: Pipeline of the system(2) New text Wav sound TTS system Triphone sequence Segmentation Synthesized triphone sequence Train database Lip motion sequence Rewrite to faces Background sequence * Techniques Analysis: Audio process Image process Synthesis Lip image Background image Stitch together * Audio part:Sound Segmentation Given the wav file and the script Using HMM to train the segment system Segment wav file to phoneme sequence Example of the segmentation result: SILOPEN 0 23 SILOPEN 24 42 s 43 61 if4 62 74 j 75 80 ia1 81 97 sh 98 109