Speech driven lip synthesis using viseme based hidden markov models

Vector Quantization: audio features clustered in to a fixed number of prototypes and incoming sound is matched to the closest prototype.

Direct Estimation: directly compute the output features with some math formula.

Visemes: shape of the mouth, used when certain sounds are voiced

• Extract audio features
• Create hidden markov model to correlate audio features with visemes
• Feed visemes to animation system