Blog

Realtime lip-sync code snippet for Flash using AS3

August 17, 2011 Flash, Programming, Tips & Tricks

One of the flash projects I have been working on has required a lip sync talking. Dialogues were dynamic so frame by frame matching was out of the question. So, I ended up with the following action script 3 code which I’m sharing with you here…

It processes the input sound in real time, which returns the different values based on the sound volume and also outputs the sound itself.

You can use those values to match up mouth animation frames as provided in the sample. But you may also simulate the effect using a single mouth image and scaling it based on the values returned by the function…

Usage

Downloads

License

/*
 - Copyright 2011 Pixel Envision (E.Gonenc)
 - http://www.pixelenvision.com/
 - support@pixelenvision.com
*/

var left:Number;

function processSound(event:SampleDataEvent):void
{
    var bytes:ByteArray = new ByteArray();
    playerObject.sourceSnd.extract(bytes, 4096);
		bytes.position = 0;
		while(bytes.bytesAvailable > 0)
    {
		left = bytes.readFloat()*128;
		if (left<0)	{
			left = -left;
		}
		var scale:Number = left*2;
	}
event.data.writeBytes(bytes);

//Define mouth animation here
if (scale<1) {mouth.gotoAndStop(1);}
else if (scale<10) {mouth.gotoAndStop(2);}
else if (scale<25) {mouth.gotoAndStop(3);}
else if (scale<50) {mouth.gotoAndStop(4);}
else {mouth.gotoAndStop(5);}
//Define mouth animation here

trace(scale);
}

var playerObject:Object = new Object(); 
playerObject.sourceSnd = new speech();
playerObject.outputSnd = new Sound();
playerObject.outputSnd.addEventListener(SampleDataEvent.SAMPLE_DATA, processSound);
playerObject.outputSnd.play();
stop();