facial tracking and animation - texas am university.pptVIP

Facial Tracking and Animation Todd Belote Bryan Harris David Brown Brad Busse Problem Background Speech driven facial animation Correlate captured facial movements to audio patterns Capture facial movements Analyze corresponding audio Goals and Objectives Develop an inexpensive, robust real-time system to track facial motion and process corresponding audio. The system must: Cost around $1000 Run on a personal computer Allow for long periods of data acquisition Handle head movements Recover from point occlusion Output only necessary information System Functionality Capture Live Data Record Video Capture From Video Generate FAP from output file Process audio of .wav file System Description Top level system organization Illustrates data flow Functional block division Data Acquisition Video Data – 320x240 at 30 fps (bmp) Audio Data – 16 bit at 16kHz (wav) Point Initialization and Tracking Process frame data Locate all areas in the frame which constitute a point Identify points Initialization – algorithm to determine identity Tracking – algorithm to match current frame points with previous location of identified points Point Transform Approach: Criminisi et al. Maps any arbitrary quadrilateral onto any other This can account for all six degrees of freedom as well as perspective distortion, greatly simplifying the computation required to reorient the face When using an orientation square that encompasses most of the face, this algorithm can be made as accurate as necessary Point Transform: Demo FAP Generation and Audio Processing FAP Generation Process text file of point movement Output FAE Engine compliant file of point displacements Audio Processing LPC MFCC Pitch Power Environmental and Health Considerations All hardware is off the shelf No harm from infrared light No harm from other products Eg. Reflective markers Social, Political and Ethical Considerations Provide low cost audiovisual capture Increase research in field by removing cost barrier Further advances Eg.