Flash | Pixel Envision Ltd.

Realtime lip-sync code snippet for Flash using AS3

August 17, 2011 Flash, Programming, Tips & Tricks

One of the flash projects I have been working on has required a lip sync talking. Dialogues were dynamic so frame by frame matching was out of the question. So, I ended up with the following action script 3 code which I’m sharing with you here…

It processes the input sound in real time, which returns the different values based on the sound volume and also outputs the sound itself.

You can use those values to match up mouth animation frames as provided in the sample. But you may also simulate the effect using a single mouth image and scaling it based on the values returned by the function…

Usage

FLA project is compatible with Flash CS4, contains sample speech & mouth animation images.
speech() is the the sound file linked from the library.
mouth is the animation clip that contains mouth drawings.

Downloads

Sample SWF file for preview.
Download FLA project

License

This code is free to use, distribute, modify and study. Mouth animation and the images provided in the sample FLA is NOT free to use in anyway. When referencing please link back to this website / post in any way e.g. direct link, credits etc. If you find this useful, please leave a comment and share using the buttons below!

/*
 - Copyright 2011 Pixel Envision (E.Gonenc)
 - http://www.pixelenvision.com/
 - support@pixelenvision.com
*/

var left:Number;

function processSound(event:SampleDataEvent):void
{
    var bytes:ByteArray = new ByteArray();
    playerObject.sourceSnd.extract(bytes, 4096);
		bytes.position = 0;
		while(bytes.bytesAvailable > 0)
    {
		left = bytes.readFloat()*128;
		if (left<0)	{
			left = -left;
		}
		var scale:Number = left*2;
	}
event.data.writeBytes(bytes);

//Define mouth animation here
if (scale<1) {mouth.gotoAndStop(1);}
else if (scale<10) {mouth.gotoAndStop(2);}
else if (scale<25) {mouth.gotoAndStop(3);}
else if (scale<50) {mouth.gotoAndStop(4);}
else {mouth.gotoAndStop(5);}
//Define mouth animation here

trace(scale);
}

var playerObject:Object = new Object(); 
playerObject.sourceSnd = new speech();
playerObject.outputSnd = new Sound();
playerObject.outputSnd.addEventListener(SampleDataEvent.SAMPLE_DATA, processSound);
playerObject.outputSnd.play();
stop();

- http://www.pixelenvision.com/

- support@pixelenvision.com

var left:Number;

function processSound(event:SampleDataEvent):void

{

var bytes:ByteArray = new ByteArray();

playerObject.sourceSnd.extract(bytes, 4096);

bytes.position = 0;

while(bytes.bytesAvailable > 0)

{

left = bytes.readFloat()*128;

if (left<0) {

left = -left;

}

var scale:Number = left*2;

}

event.data.writeBytes(bytes);

//Define mouth animation here

if (scale<1) {mouth.gotoAndStop(1);}

else if (scale<10) {mouth.gotoAndStop(2);}

else if (scale<25) {mouth.gotoAndStop(3);}

else if (scale<50) {mouth.gotoAndStop(4);}

else {mouth.gotoAndStop(5);}

//Define mouth animation here

trace(scale);

}

var playerObject:Object = new Object();

playerObject.sourceSnd = new speech();

playerObject.outputSnd = new Sound();

playerObject.outputSnd.addEventListener(SampleDataEvent.SAMPLE_DATA, processSound);

playerObject.outputSnd.play();

stop();

Category: Flash

Realtime lip-sync code snippet for Flash using AS3

Usage

Downloads

License