This is the second in a series of tutorials in which we will create a synthesizer based audio engine that can generate sounds for retro-styled games. The audio engine will generate all of the sounds at runtime without the need for any external dependencies such as MP3 files or WAV files. The end result will be a working library that can be dropped effortlessly into your games.
If you have not already read the first tutorial in this series, you should do that before continuing.
The programming language used in this tutorial is ActionScript 3.0 but the techniques and concepts used can easily be translated into any other programming language that provides a low-level sound API.
You should make sure you have Flash Player 11.4 or higher installed for your browser if you want to use the interactive examples in this tutorial.
Audio Engine Demo
By the end of this tutorial all of the core code required for the audio engine will have been completed. The following is a simple demonstration of the audio engine in action.
Only one sound is being played in that demonstration, but the frequency of the sound is being randomised along with its release time. The sound also has a modulator attached to it to produce the vibrato effect (modulate the sound’s amplitude) and the frequency of the modulator is also being randomised.
AudioWaveform Class
The first class that we will create will simply hold constant values for the waveforms that the audio engine will use to generate the audible sounds.
Start by creating a new class package called noise, and then add the following class to that package:
package noise { public final class AudioWaveform { static public const PULSE:int = 0; static public const SAWTOOTH:int = 1; static public const SINE:int = 2; static public const TRIANGLE:int = 3; } } We will also add a static public method to the class that can be used to validate a waveform value, the method will return true or false to indicate whether or not the waveform value is valid.
static public function validate( waveform:int ):Boolean { if( waveform == PULSE ) return true; if( waveform == SAWTOOTH ) return true; if( waveform == SINE ) return true; if( waveform == TRIANGLE ) return true; return false; } Finally, we should prevent the class from being instantiated because there is no reason for anyone to create instances of this class. We can do this within the class constructor:
public function AudioWaveform() { throw new Error( "AudioWaveform class cannot be instantiated" ); } This class is now complete.
Preventing enum-style classes, all-static classes, and singleton classes from being directly instantiated is a good thing to do because these types of class should not be instantiated; there is no reason to instantiate them. Programming languages such as Java do this automatically for most of these class types but currently in ActionScript 3.0 we need to enforce this behaviour manually within the class constructor.
Audio Class
Next on the list is the Audio class. This class in similar in nature to the native ActionScript 3.0 Sound class: every audio engine sound will be represented by an Audio class instance.
Add the following barebones class to the noise package:
package noise { public class Audio { public function Audio() {} } } The first things that need to be added to the class are properties that will tell the audio engine how to generate the sound wave whenever the sound is played. These properties include the type of waveform used by the sound, the frequency and amplitude of the waveform, the duration of the sound, and its release time (how quickly it fades out). All of these properties will be private and accessed via getters/setters:
private var m_waveform:int = AudioWaveform.PULSE; private var m_frequency:Number = 100.0; private var m_amplitude:Number = 0.5; private var m_duration:Number = 0.2; private var m_release:Number = 0.2;
As you can see, we have set a sensible default value for each property. The amplitude is a value in the range 0.0 to 1.0, the frequency is in hertz, and the duration and release times are in seconds.
We also need to add two more private properties for the modulators that can be attached to the sound; again these properties will be accessed via getters/setters:
private var m_frequencyModulator:AudioModulator = null; private var m_amplitudeModulator:AudioModulator = null;
Finally, the Audio class will contain a few internal properties that will only be accessed by the AudioEngine class (we will create that class shortly). These properties do not need to be hidden behind getters/setters:
internal var position:Number = 0.0; internal var playing:Boolean = false; internal var releasing:Boolean = false; internal var samples:Vector.<Number> = null;
The position is in seconds and it allows the AudioEngine class to keep track of the sound’s position while the sound is playing, this is needed to calculate the waveform sound samples for the sound. The playing and releasing properties tell the AudioEngine what state the sound is in, and the samples property is a reference to the cached waveform samples that the sound is using. The use of these properties will become clear when we create the AudioEngine class.
To finish the Audio class we need to add the getters/setters:
Audio.waveform
public final function get waveform():int { return m_waveform; } public final function set waveform( value:int ):void { if( AudioWaveform.isValid( value ) == false ) { return; } switch( value ) { case AudioWaveform.PULSE: samples = AudioEngine.PULSE; break; case AudioWaveform.SAWTOOTH: samples = AudioEngine.SAWTOOTH; break; case AudioWaveform.SINE: samples = AudioEngine.SINE; break; case AudioWaveform.TRIANGLE: samples = AudioEngine.TRIANGLE; break; } m_waveform = value; } Audio.frequency
[Inline] public final function get frequency():Number { return m_frequency; } public final function set frequency( value:Number ):void { // clamp the frequency to the range 1.0 - 14080.0 m_frequency = value < 1.0 ? 1.0 : value > 14080.0 ? 14080.0 : value; } Audio.amplitude
[Inline] public final function get amplitude():Number { return m_amplitude; } public final function set amplitude( value:Number ):void { // clamp the amplitude to the range 0.0 - 1.0 m_amplitude = value < 0.0 ? 0.0 : value > 1.0 ? 1.0 : value; } Audio.duration
[Inline] public final function get duration():Number { return m_duration; } public final function set duration( value:Number ):void { // clamp the duration to the range 0.0 - 60.0 m_duration = value < 0.0 ? 0.0 : value > 60.0 ? 60.0 : value; } Audio.release
[Inline] public final function get release():Number { return m_release; } public function set release( value:Number ):void { // clamp the release time to the range 0.0 - 10.0 m_release = value < 0.0 ? 0.0 : value > 10.0 ? 10.0 : value; } Audio.frequencyModulator
[Inline] public final function get frequencyModulator():AudioModulator { return m_frequencyModulator; } public final function set frequencyModulator( value:AudioModulator ):void { m_frequencyModulator = value; } Audio.amplitudeModulator
[Inline] public final function get amplitudeModulator():AudioModulator { return m_amplitudeModulator; } public final function set amplitudeModulator( value:AudioModulator ):void { m_amplitudeModulator = value; } You no doubt noticed the [Inline] metadata tag bound to a few of the getter functions. That metadata tag is a shiny new feature of Adobe’s latest ActionScript 3.0 Compiler and it does what says on the tin: it inlines (expands) the contents of a function. This is extremely useful for optimisation when used sensibly, and generating dynamic audio at runtime is certainly something that requires optimisation.
AudioModulator Class
The purpose of the AudioModulator is to allow the amplitude and frequency of Audio instances to be modulated to create useful and crazy sound effects. Modulators are actually similar to Audio instances, they have a waveform, an amplitude, and frequency, but they don’t actually produce any audible sound they only modify audible sounds.
First thing first, create the following barebones class in the noise package:
package noise { public class AudioModulator { public function AudioModulator() {} } } Now let’s add the private private properties:
private var m_waveform:int = AudioWaveform.SINE; private var m_frequency:Number = 4.0; private var m_amplitude:Number = 1.0; private var m_shift:Number = 0.0; private var m_samples:Vector.<Number> = null;
If you are thinking this looks very similar to the Audio class then you are correct: everything except for the shift property is the same.
To understand what the shift property does, think of one of the basic waveforms that the audio engine is using (pulse, sawtooth, sine, or triangle) and then imagine a vertical line running straight through the waveform at any position you like. The horizontal position of that vertical line would be the shift value; its a value in the range 0.0 to 1.0 that tells the modulator where to begin reading it’s waveform from and in turn can have a profound affect on the modifications the modulator makes to a sound’s amplitude or frequency.
As an example, if the modulator was using a sine waveform to modulate the frequency of a sound, and the shift was set at 0.0, the sound’s frequency would first rise and then fall due to the curvature of the sine wave. However, if the shift was set at 0.5 the sound’s frequency would first fall and then rise.
Anyway, back to the code. The AudioModulator contains one internal method that is only used by the AudioEngine; the method is as follows:
[Inline] internal final function process( time:Number ):Number { var p:int = 0; var s:Number = 0.0; if( m_shift != 0.0 ) { time += ( 1.0 / m_frequency ) * m_shift; } p = ( 44100 * m_frequency * time ) % 44100; s = m_samples[p]; return s * m_amplitude; } That function is inlined because it is used a lot, and when I say “a lot” I mean 44100 times a second for each sound that is playing that has a modulator attached to it (this is where inlining becomes incredibly valuable). The function simply grabs a sound sample from the waveform the modulator is using, adjusts that sample’s amplitude, and then returns the result.
To finish the AudioModulator class we need to add the getters/setters:
AudioModulator.waveform
public function get waveform():int { return m_waveform; } public function set waveform( value:int ):void { if( AudioWaveform.isValid( value ) == false ) { return; } switch( value ) { case AudioWaveform.PULSE: m_samples = AudioEngine.PULSE; break; case AudioWaveform.SAWTOOTH: m_samples = AudioEngine.SAWTOOTH; break; case AudioWaveform.SINE: m_samples = AudioEngine.SINE; break; case AudioWaveform.TRIANGLE: m_samples = AudioEngine.TRIANGLE; break; } m_waveform = value; } AudioModulator.frequency
public function get frequency():Number { return m_frequency; } public function set frequency( value:Number ):void { // clamp the frequency to the range 0.01 - 100.0 m_frequency = value < 0.01 ? 0.01 : value > 100.0 ? 100.0 : value; } AudioModulator.amplitude
public function get amplitude():Number { return m_amplitude; } public function set amplitude( value:Number ):void { // clamp the amplitude to the range 0.0 - 8000.0 m_amplitude = value < 0.0 ? 0.0 : value > 8000.0 ? 8000.0 : value; } AudioModulator.shift
public function get shift():Number { return m_shift; } public function set shift( value:Number ):void { // clamp the shift to the range 0.0 - 1.0 m_shift = value < 0.0 ? 0.0 : value > 1.0 ? 1.0 : value; } And that wraps up the AudioModulator class.
AudioEngine Class
Now for the big one: the AudioEngine class. This is an all-static class and manages pretty much everything related to Audio instances and sound generation.
Let’s start with a barebones class in the noise package as usual:
package noise { import flash.events.SampleDataEvent; import flash.media.Sound; import flash.media.SoundChannel; import flash.utils.ByteArray; // public final class AudioEngine { public function AudioEngine() { throw new Error( "AudioEngine class cannot be instantiated" ); } } } As mentioned before, all-static classes should not be instantiated, hence the exception that is thrown in the class constructor if someone does try to instantiate the class. The class is also final because there’s no reason to extend an all-static class.
The first things that will be added to this class are internal constants. These constants will be used to cache the samples for each of the four waveforms that the audio engine is using. Each cache contains 44,100 samples which equates to one hertz waveforms. This allows the audio engine to produce really clean low frequency sound waves.
The constants are as follows:
static internal const PULSE:Vector.<Number> = new Vector.<Number>( 44100 ); static internal const SAWTOOTH:Vector.<Number> = new Vector.<Number>( 44100 ); static internal const SINE:Vector.<Number> = new Vector.<Number>( 44100 ); static internal const TRIANGLE:Vector.<Number> = new Vector.<Number>( 44100 );
There are also two private constants used by the class:
static private const BUFFER_SIZE:int = 2048; static private const SAMPLE_TIME:Number = 1.0 / 44100.0;
The BUFFER_SIZE is the number of sound samples that will be passed to the ActionScript 3.0 sound API whenever a request for sound samples is made. This is the smallest number of samples allowed and it results in the lowest possible sound latency. The number of samples could be increased to reduce CPU usage but that would increase the sound latency. The SAMPLE_TIME is the duration of a single sound sample, in seconds.
And now for the private variables:
static private var m_position:Number = 0.0; static private var m_amplitude:Number = 0.5; static private var m_soundStream:Sound = null; static private var m_soundChannel:SoundChannel = null; static private var m_audioList:Vector.<Audio> = new Vector.<Audio>(); static private var m_sampleList:Vector.<Number> = new Vector.<Number>( BUFFER_SIZE );
- The
m_positionis used to keep track of the sound stream time, in seconds. - The
m_amplitudeis a global secondary amplitude for all of theAudioinstances that are playing. - The
m_soundStreamandm_soundChannelshouldn’t need any explanation. - The
m_audioListcontains references to anyAudioinstances that are playing. - The
m_sampleListis a temporary buffer used to store sound samples when they are requested by the ActionScript 3.0 sound API.
Now, we need to initialize the class. There are numerous ways of doing this but I prefer something nice and simple, a static class constructor:
static private function $AudioEngine():void { var i:int = 0; var n:int = 44100; var p:Number = 0.0; // while( i < n ) { p = i / n; SINE[i] = Math.sin( Math.PI * 2.0 * p ); PULSE[i] = p < 0.5 ? 1.0 : -1.0; SAWTOOTH[i] = p < 0.5 ? p * 2.0 : p * 2.0 - 2.0; TRIANGLE[i] = p < 0.25 ? p * 4.0 : p < 0.75 ? 2.0 - p * 4.0 : p * 4.0 - 4.0; i++; } // m_soundStream = new Sound(); m_soundStream.addEventListener( SampleDataEvent.SAMPLE_DATA, onSampleData ); m_soundChannel = m_soundStream.play(); } $AudioEngine(); If you have read the previous tutorial in this series then you will probably see what’s happening in that code: the samples for each of the four waveforms are being generated and cached, and this only happens once. The sound stream is also being instantiated and started and will run continuously until the app is terminated.
The AudioEngine class has three public methods that are used to play and stop Audio instances:
AudioEngine.play()
static public function play( audio:Audio ):void { if( audio.playing == false ) { m_audioList.push( audio ); } // this allows us to know exactly when the sound was started audio.position = m_position - ( m_soundChannel.position * 0.001 ); audio.playing = true; audio.releasing = false; } AudioEngine.stop()
static public function stop( audio:Audio, allowRelease:Boolean = true ):void { if( audio.playing == false ) { // the sound isn't playing return; } if( allowRelease ) { // skip to the end of the sound and flag it as releasing audio.position = audio.duration; audio.releasing = true; return; } audio.playing = false; audio.releasing = false; } AudioEngine.stopAll()
static public function stopAll( allowRelease:Boolean = true ):void { var i:int = 0; var n:int = m_audioList.length; var o:Audio = null; // if( allowRelease ) { while( i < n ) { o = m_audioList[i]; o.position = o.duration; o.releasing = true; i++; } return; } while( i < n ) { o = m_audioList[i]; o.playing = false; o.releasing = false; i++; } } And here come the main audio processing methods, both of which are private:
AudioEngine.onSampleData()
static private function onSampleData( event:SampleDataEvent ):void { var i:int = 0; var n:int = BUFFER_SIZE; var s:Number = 0.0; var b:ByteArray = event.data; // if( m_soundChannel == null ) { while( i < n ) { b.writeFloat( 0.0 ); b.writeFloat( 0.0 ); i++; } return; } // generateSamples(); // while( i < n ) { s = m_sampleList[i] * m_amplitude; b.writeFloat( s ); b.writeFloat( s ); m_sampleList[i] = 0.0; i++; } // m_position = m_soundChannel.position * 0.001; } So, in the first if statement we are checking if the m_soundChannel is still null, and we need to do that because the SAMPLE_DATA event is dispatched as soon as the m_soundStream.play() method is invoked, and before the method gets a chance to return a SoundChannel instance.
The while loop rolls through the sound samples that have been requested by m_soundStream and writes them to the provided ByteArray instance. The sound samples are generated by the following method:
AudioEngine.generateSamples()
static private function generateSamples():void { var i:int = 0; var n:int = m_audioList.length; var j:int = 0; var k:int = BUFFER_SIZE; var p:int = 0; var f:Number = 0.0; var a:Number = 0.0; var s:Number = 0.0; var o:Audio = null; // roll through the audio instances while( i < n ) { o = m_audioList[i]; // if( o.playing == false ) { // the audio instance has stopped completely m_audioList.splice( i, 1 ); n--; continue; } // j = 0; // generate and buffer the sound samples while( j < k ) { if( o.position < 0.0 ) { // the audio instance hasn't started playing yet o.position += SAMPLE_TIME; j++; continue; } if( o.position >= o.duration ) { if( o.position >= o.duration + o.release ) { // the audio instance has stopped o.playing = false; j++; continue; } // the audio instance is releasing o.releasing = true; } // grab the audio instance's frequency and amplitude f = o.frequency; a = o.amplitude; // if( o.frequencyModulator != null ) { // modulate the frequency f += o.frequencyModulator.process( o.position ); } // if( o.amplitudeModulator != null ) { // modulate the amplitude a += o.amplitudeModulator.process( o.position ); } // calculate the position within the waveform cache p = ( 44100 * f * o.position ) % 44100; // grab the waveform sample s = o.samples[p]; // if( o.releasing ) { // calculate the fade-out amplitude for the sample s *= 1.0 - ( ( o.position - o.duration ) / o.release ); } // add the sample to the buffer m_sampleList[j] += s * a; // update the audio instance's position o.position += SAMPLE_TIME; j++; } i++; } } Finally, to finish things off, we need to add the getter/setter for the private m_amplitude variable:
static public function get amplitude():Number { return m_amplitude; } static public function set amplitude( value:Number ):void { // clamp the amplitude to the range 0.0 - 1.0 m_amplitude = value < 0.0 ? 0.0 : value > 1.0 ? 1.0 : value; } And now I need a break!
Coming Up…
In the third and final tutorial in the series we will be adding audio processors the to audio engine. These will allow us to push all of the generated sound samples though processing units such as hard limiters and delays. We will also be taking a look at all of the code to see if anything can be optimised.
All of the source code for this tutorial series will be made available with the next tutorial.
Follow us on Twitter, Facebook, or Google+ to keep up to date with the latest posts.
Entradas
No hay comentarios:
Publicar un comentario