Content Based Soundtracks in Flash

This paper describes a technique for playing back multiple soundtracks in a Flash ® web site. The technique allows the selection of which soundtrack to play to be based on what content is being displayed. It also allows the user to pause the movie or browse the site without interrupting the soundtrack, and to mute the soundtrack without affecting their ability to browse the site while still allowing sound effects like button sounds to play.
This paper deals with advanced Flash ® techniques that include using sound, setting and checking variables, loading and using movies on various levels, and targeting movies and instances on different levels. As this paper describes advanced techniques it assumes that the reader is comfortable working in Flash® and so does not describe every step for everything.

Backgrounder: "What I want are multiple, awesome soundtracks that I have artistic control over, delivered in full stereo and all in a file small enough that it can be downloaded from the web in a reasonable amount of time…" These were the specs I asked of killersound, Inc. to deliver. I got more than I bargained for.

Using the killersound technique™ described in Layering Audio in Flash ® - the killersound technique ™ , I was able to get the soundtracks that I wanted, but now I was faced with delivering them. I was working on a site that used many .swf files loaded on different levels. One level contained the master navigation that was used to view the various main sections of content. This navigational tool was always on stage regardless of what content was being viewed. Another level contained the content for the site, which was delivered in several .swf files on the same level. Each section of content would play and then load the next movie in the sequence to take its place, playing like a presentation. Or they could be called by the user via the main navigation providing interactive navigation and content on demand. Each section of content contained a content sensitive navigational element which allowed the user to move backwards and forwards within the content, pause the content movie, and mute or un-mute the soundtrack. I wanted the user to be able to navigate within the content independent of what was happening with the soundtracks, and to be able to mute or un-mute the soundtrack without affecting the content.

The technique that I developed worked so well that I felt that I should share it. I created a test site that uses two soundtracks mixed using the killersound technique, four sections of "content", and the other movies to load and drive the site and its sound. This paper is a description of these files and the steps used to build the test site. Because my technique is somewhat organic and the parts are very interdependent, if you don’t get everything on the first read don’t worry. Just read it again and it should be clearer.

The Files

First let me describe the files that we are using.

loader.html - This is the HTML file that is created by loader.fla. It is where a user will point their browser to view our test site.
loader.fla - It contains the action scripts used to load all of the movies into their correct levels and get things started. It also contains the menu buttons which let the user navigate between the content sections. This lets us load the navigation for the site only once. This movie will be on level 0.
conductor.fla - It contains the actions scripts that will control the sound. It also will be used to hold variable information about weather or not the user has "muted" the sound, what soundtrack is supposed to be playing, and what soundtrack is currently being played. It also contains text fields that display the current value of these variables. This movie will be on level 2.
section1-4.fla - These 4 movies represent the various content of our test site. They each contain action scripts that control the conductor, a mute button, and a button that allows the user to stop the movie independent of the soundtracks. These movies will all be on level 1, but only one at a time.
test1-2.fla – These movies each contain a soundtrack based on the killersound, inc. technique discussed in "Layering Audio in Flash®; The killersound Technique". These movies will be on levels 3 and 4, one for each soundtrack.

loader.fla

This movie is the base for our test site. We use it to generate the HTML file that a user will browse to see our site. The HTML file contains the code to show the loader.swf file. All the other movies are controlled from within it or the other movies.

There are two layers in this movie, each with only one frame. The first layer is the "actions" layer. It contains the "Load Movie" actions which get everything up and running. (Fig.1) First it loads the "test1.swf" movie on layer 3. This is the soundtrack for sections 1 and 2. It is loaded first so that it is ready to play when the conductor calls on it. In a real project you will want to optimize your loading sequence for your files.

Fig. 1 – Loader frame actions

Next we load the "conductor.swf" movie on layer 2. This gets our sound started. I’ll show you how when we look at the conductor.

Then we load the "section1.swf" movie on layer 1. Now our user will start to see the first content of our site. This is the layer that will contain all the content.

And finally we load the soundtrack for sections 3 and 4, "test2.swf" on layer 4. As the soundtrack movies are likely to be the largest in file size, this lets our users hear and see content while we preload the next soundtrack in the background.

The other layer is the "buttons" layer. There are four buttons which make up the navigation for our site. They allow a user to view whichever content section they want. Each button has only one action: "On Release, Load Movie "sectionX.swf" on Level 1", where X is the section number for the button. (Fig. 2) So the button to load Section 1 would load "section1.swf", the button to load Section 2 would load "section2.swf", and so on.

Fig. 2 – Loader button actions

test1.fla

The two files "test1.fla" and "test2.fla" are the soundtracks for our test site. They both use the killersound method of layering the sound and using Flash^® as a sequencer to get a lot of sound from some very small files. To learn more about the killersound method, read " Layering Audio in Flash®; The killersound Technique ™".

In order to use more than one soundtrack we have to make some structural changes to the files. They are the same for all the soundtracks that we will use, so I’ll just explain test1.fla.

First we add an "actions" layer. Next we create the first of two sections – the "stop" section. (Fig. 3)

Fig. 3 – Soundtrack layers

Give frame 1 a Label of "stop". This is important because it is where I will tell the movie to go when I want it to stop playing. I also give it a "stop" action. This keeps the movie from playing when it is first loaded. We put the "stop" section before the "play" section so that we have control over when the soundtrack is played and it doesn’t just start playing when it is loaded. Because this is where we want to cause the sound to stop playing, on each of the layers which have sound (drums, melody, etc…), create a blank key frame. We add a frame action to stop each sound individually. To do this, double click on the key frame of the first sound layer and select the "Sounds" tab from the Frame Properties dialogue box. In the Sound dropdown box select the sound you are using on that layer (drums.wav on the drums layer, etc…). In the Sync dropdown choose "Stop". (Fig. 4) Repeat this process for each sound layer. The reason for stopping each sound individually and not using a "stop all sounds" is that this way you are only stopping the soundtrack and not any other sounds.

Fig. 4 – The Sync "stop" sound action

Next we create the "play" section. I created a blank key frame in frame 6 of the actions layer and give it a label of "play". Because we will address the label and not the frame number, it doesn’t matter what frame it is. I chose frame 6 so you can see the "stop" label. Now we add the sound. On the first sound layer create a blank key frame in the frame after the "play" frame in the actions layer. In this example it is frame 7. Now add the sound to that layer by double clicking on the frame and selecting the "Sound" tab from the Frame Properties dialogue box. In the Sound dropdown box select the sound you are using on that layer (drums.wav on the drums layer, etc…). In the Sync dropdown choose "Event". (Fig. 5) We add the sound to the frame after the "play" label because of how we will be dealing with the looping of the soundtrack. We choose "Event" because the soundtrack does not need to sync up to the animation, and because we want it to play clearly without dropping frames from our content.

Fig. 5 – Playing sound

Once we tell a soundtrack to "play" the test1.swf movie will go to the frame labeled "play" and play. When it hits the next frame (frame 7 in our example) it will start the different pieces of music on the different layers at the same time and play your mix. But by using very short files that are looped to create the mix we loose the ability to just have the sound start over by using the loop feature, so we have to start the soundtrack over manually. We do this with a frame action placed on the actions layer in the frame that corresponds with the end of your mix. (Fig. 6)

Fig. 6 – Go To and Play action at the end of the mix

Here is an example. Let’s say that you have a sound called "drums". You can find out how long the sample is in frames by double clicking on the frame containing the sound (frame 7 of the "drums" layer in our example) and selecting the Sound tab. Flash^® lets you view your sound on a time line that is either shown in seconds or frames. You choose the seconds by clicking on the icon that looks like a clock, or frames by clicking on the icon that looks like a movie. Choose the frames icon. (Fig. 7) You can then use the zoom tool to get in close enough that you will see the frame numbers. At the end of your sound there will be a vertical line that shows you where it ends. That tells you approximately how many frames your sound will be. When you change the number of "loops" you will see a new wave with another vertical line showing each loop. Lets say that your mix has the drums looping 10 times. To find the ending frame of your mix just scroll down the time line until you see no more music, zoom in and look at the frame number. Now close the Frame Properties dialogue box and back on the main timeline of the movie, scroll to the same frame number. Add 7 for the frames from the start of the movie to the start of the music. Now "insert frame" on each layer of the movie. You will see the end of the drums wave form. On the actions layer insert a blank key frame in the last frame and give it a frame action of "Go to and play" the label "play". This will start the sound over.

Fig. 7 – Length of sound in number of Frames

Now here’s the catch. Because the sounds are Event sounds they will play independent of the time line. But the action in the last frame is dependent on the time line. The time line is dependent on the processor of the users computer, so the sound may end before the users computer can catch up to the end of the movie. You can either fade the sound out leaving some "dead air" before it starts again, just let it keep playing the last bit of your mix, or create a "bridge" sound. We chose the Bridge method. Using the bridge method you would start the bridge sound at the same time as the other sounds, but it’s volume would be "0". Then near the end of your mix you can fade the sound in and just let it loop a very long time at the same volume. In this example if the mix was 10 loops long, we might set our bridge to 1000 loops. When the movie hits the action that sends it back to the "play" label, you will notice one difference in the Bridge layer from the other sound layers. On the bridge layer in the same frame as the "play" label there is a blank key frame that has the same instructions to stop the "Bridge" sound as there are in the "stop" frame. (Fig.8) This means that whenever the sound loops back it will stop the bridge sound and begin playing the soundtrack again, regardless of when that is. To see this in action in our test site, pause the movie anywhere and just listen for the soundtrack to start over. Then try it on a speed different machine…

Fig. 8 – Stopping the "bridge" when the soundtrack plays

section1.fla

For our example each section contains three layers – actions, box, and buttons. The box layer is just a simple animation of a bar that shows the progress of the movie from beginning to end.

The buttons layer contains two buttons. The first one is a pause button that stops and starts the movie. Because we want it to toggle its function it is contained in a movie clip. The first frame of the move contains a label of "stop" and a frame action of "stop" to keep it in this position. (Fig. 9) The button here has a button action which tells the main timeline to "stop" and to move its own time line to the "play" frame. The "play" frame contains a button with the action to "play" the main time line and move its own time line back to the "stop" frame. There is also a different graphic at each label which tells the user what will happen when they click on the button.

Fig. 9 – Content Pause button actions

The other button is the mute button. The mute button allows the user to turn the soundtracks off and on. Because we need it to be either a "mute sound" button if the music is playing or a "play sound" button if the music is muted, and because we need to be able to tell the button which position to be in, the button is contained in a movie clip. We make the instance of the movie clip a target by giving it an instance name of "play-mute". (Fig. 10)

Fig. 10 – Turing the Mute

This move contains the "stop" and "play" positions just like the pause button. But these buttons need to talk to a movie on a different level. Movies can talk to other movies on different levels by using the "tell target" action. In this case we are going to talk to the conductor on level 2, so we use the action "tell target" and use the target "_level2". (Fig. 11) Then they set the variable "muted" on level 2 either to "yes" or "no", and then they tell the conductor to "go to and play" the label "playsound." This is where we will always send the conductor to figure out what action to take regarding the soundtracks.

Fig. 11 – Mute button actions

The actions layer has two frames with actions. The first one is at the beginning of the movie and it has two functions. First it tells the conductor what soundtrack should be played for this section, and second makes sure that the mute button is in the correct position. Lets take a look at these one at a time.

First we set the variable "score" to the soundtrack for this section, "test1". (Fig. 12) Note that we are setting the variable "score" on level 2. This is because although the sections will replace one another, the conductor will always be present, and a variable only exists as long as the movie that contains it. After setting the "score", we tell the conductor to "go to and play" the label "playsound".

Click to enlarge

Fig. 12 – First frame actions

Next we check to see if the user has muted the sound, and if so, put the mute button in the "play sound" mode. The action in this frame checks to see if the value of the variable "muted" on level 2 is equal to "yes". If it’s not, it does nothing because the button starts in the "mute sound" position. If it is, it tells the movie clip instance named "play-mute" to go to and stop at the label "play".

The other frame with actions is at the end of the movie. It quite simply loads the next section of content in the sequence so that the movies all play in order. (Fig. 13) In short, section1.swf loads section2.swf, which loads section3.swf, which loads section4.swf, which loads section1.swf and completes the cycle. Each section loads the next section on to the same level as its self, level 1. Because each level can only contain one movie at a time, when you load a movie on a level that already contains a movie, the new movie replaces the old one. This means that for our test movie there is only one section of content on the stage at a time.

Click to enlarge

Fig. 13 – Last frame actions

conductor.fla

The conductor, as the name implies, is the movie which controls the sound. For the purposes of our test file there are text fields which display the current value of the three variables that the conductor cares about. They are "muted" which says if the user has muted the soundtracks or not, "score" which has the name of the soundtrack that is intended for the current section of content, and "nowplaying" which has the value of what soundtrack is currently being played.

The first section is labeled "setsnd". It contains the actions to set the initial state of the variables and start playing the opening soundtrack. (Fig. 14) It sets "muted" to "no", "score" to "test1" sense we know that section 1 is the first section that the user will see, and sets "nowplaying" to "test1". It then sends the conductor to the section labeled "test1".

Fig. 14 – The "setsnd" section actions

Sections "test1" and "test2" work in basically the same way. They are the triggers which actually cause music to play. The frame actions first set the variable "nowplaying" to "test1" or "test2", respectively. (Fig. 15) Then the appropriate soundtrack is told to "go to and play" the frame labeled "play" via the "tell target" action. Test1 targets level 3, and test2 targets level 4. The final action is a "stop" action which holds the conductor until some other event tries to change the sound.

Fig. 15 – The "test1" section actions

The frame labeled "playsound" is the real decision maker for the conductor. The content sections of the site all talk to this frame. It works as one long "if" statement. First it checks to see if the sound is muted. (Fig. 16)

Fig. 16 – The "playsound" section actions

If "muted" equals "yes" then the conductor goes to the section labeled "mute". The mute section tells both soundtracks to "go to and stop" at the label "stop" which causes both soundtracks to quit playing. (Fig. 17) A "stop" action then holds the conductor until the next sound event.

Fig. 17 – The "mute" section actions

If "muted" does not equal "yes" then the next "if" checks to see if the requested soundtrack is already playing. For example, sections 1 and 2 use the same soundtrack. This lets the soundtrack play uninterrupted when the sections change. This is checked by comparing the values of the requested "score" which comes from the first frame of each of the content sections, to the value of "nowplaying" which is set by the conductor. If they are the same then the requested sound track is already playing and the conductor goes to the frame labeled "continueplay". That frame contains only a "stop" action to hold the conductor until the next sound event.

If the "score" and "nowplaying" values are not the same then we know that we need to change the soundtrack. That is a two step process. First we check to see if "score" equals "test1". If it does then we know we need to tell the soundtrack "test2" to stop playing and to start playing the "test1" soundtrack. We stop "test2" by using a "tell target" with a target of "_level4" and a "go to and stop" at the frame labeled "stop". We then tell the conductor to "go to" the frame labeled "test1" which will start that soundtrack.

If "score" does not equal "test1" then it must be the other soundtrack that we are wanting, so we stop the soundtrack on level 3 and go to the frame labeled "test2".

Conclusion: Well, that’s it. As I stated in the beginning, if it doesn’t sink in on the first read, read it again, open the files and look at what is happening. Because it is a very organic, interdependent system, it may take a few passes to see how it all functions.

There are some other things that you will want to take into consideration. Sound in Flash® is still likely to be the largest files that you will have, so you will want to make a loader that loads you control movies, content movies, and soundtrack movies in the best order for your site. Remember that you can load a soundtrack and it will not play until the conductor tells it to. I found it best to load just the first soundtrack and then the first section so that the remaining files can load while the user is viewing content.