Firelight Technologies FMOD Studio API

Terminology / Basic Concepts

Introduction

Throughout FMOD documentation certain terms and concepts will be used. This section will explain some of these to alleviate confusion.

It is recommended when you see an API function highlighted as a link, that you check the API reference for more detail.

Samples vs bytes vs milliseconds

Within FMOD functions you will see references to PCM samples, bytes and milliseconds.

To understand what the difference is a diagram has been provided to show how raw PCM sample data is stored in FMOD buffers.

• A left/right pair (a sound with 2 channels) is called a sample.
• Because this is made up of 16bit data, 1 sample = 4 bytes.
• If the sample rate, or playback rate is 44.1khz, or 44100 samples per second, then 1 sample is 1/44100th of a second, or 1/44th of a millisecond. Therefore 44100 samples = 1 second or 1000ms worth of data.

To convert between the different terminologies, the following formulas can be used:

• ms = samples * 1000 / samplerate.
• samples = ms * samplerate / 1000.
• samplerate = samples * 1000 / ms.
• bytes = samples * bits * channels / 8.
• samples = bytes * 8 / bits / channels.

Some functions like Sound::getLength provide the length in milliseconds, bytes and samples to avoid needing to do these calculations.

Sounds. Samples vs compressed samples vs streams

When a sound is loaded, it is either decompressed as a static sample into memory as PCM (samples), loaded into memory in its native format and decompressed at runtime (compressed samples), or streamed and decoded in realtime (in chunks) from an external media such as a harddisk or CD (streams).

• "Samples" are good for small sounds that need to be played more than once at a time, for example sound effects. These generally use little or no CPU to play back and can be hardware accelerated. See FMOD_CREATESAMPLE.
• "Streams" are good for large sounds that are too large to fit into memory and need to be streamed from disk into a small ringbuffer that FMOD manages. These take a small amount of CPU and disk bandwidth based on the file format. For example MP3 takes more cpu power to decode in real-time than a PCM decompressed wav file does. A streaming sound can only be played once at a time, they cannot be spawned multiple times at once like a sample, due to it only having 1 file handle per stream and 1 ringbuffer to decode into. See FMOD_CREATESTREAM.
• "Compressed samples" are an option that allows the user to load a certain compressed file format (such as IMA ADPCM, FADPCM, Vorbis, MP2, MP3, AT9 and XMA formats currently). FADPCM/Vorbis/AT9 are only supported through the .FSB container format), and leave them compressed in memory without decompressing them. They are software mixed on the CPU and don't have the 'once only' limitation of streams. They take more cpu than a standard PCM sample, but actually less than a stream due to not doing any disk access and much smaller memory buffers. See FMOD_CREATECOMPRESSEDSAMPLE.

By default System::createSound will want to decode the whole sound fully into memory (ie, as a decompressed sample). To have it stream in realtime and save memory, use the FMOD_CREATESTREAM flag when creating a sound, or use the helper function System::createStream which is essentially the same as System::createSound but just has the FMOD_CREATESTREAM flag added in automatically for you. To make a compressed sample use System::createSound with FMOD_CREATECOMPRESSEDSAMPLE.

Channels and sounds

When you have loaded your sounds, you will want to play them. When you play them you will use System::playSound, which will return you a pointer to a Channel / FMOD_CHANNEL handle.

FMOD will automatically select a channel for the sound to play on, you do not have to manage your own channels.

Sub-mixing and ChannelGroups

Instead of processing or controlling channels individually, channels can be grouped into a 'ChannelGroup'. ChannelGroups allow you to operate on a group of channels at a time, and control the mix graph of the audio.

A ChannelGroup can also be thought of as a 'sub mix', as in the signal chain, this would be the point that the channel signals mix into a single buffer. This buffer can then be processed with a DSP effect (see below) once, rather than once for each channel, saving a lot of CPU time.

2D vs 3D

A 3D sound source is a channel that has a position and a velocity in space. When a 3D channel is playing, its volume, speaker placement and pitch will be affected automatically based on the relation to the listener.
A listener is the player, or the game camera. It has a position and velocity like a sound source, but it also has an orientation.

The listener and the source distance from each other determine the volume.
The listener and the source relative velocity determines the pitch (doppler effect).
The orientation of the listener to the source determines the pan or speaker placement.

A 2D sound is simply different in that it is not affected by the 3D sound listener, and does not have doppler or attenuation or speaker placement affected by it.
A 2D sound can call Channel::setMixLevelsOutput, Channel::setMixMatrix or Channel::setPan, whereas these commands on a 3D sound will not have any effect, unless you call Channel::set3DLevel to alter the 2D component of the 3D sound. You can blend a sound between 3D and 2D using this function.
A 3D sound can call any function with the word 3D in the function name, whereas a 2D sound cannot.

For a more detailed description of 3D sound, read the tutorial in the documentation on 3D sound.

DSP

DSP stands for "Digital Signal Processing", and usually relates to processing raw PCM samples to alter the sound. FMOD provides a suite of DSP effects that can alter the sound in interesting ways to simulate real life or exaggerate a sound.
Examples of this are echo, reverb, lowpass filtering, flange and chorus.
Effects can easily be added to an FMOD channel, or a sub mix, or ChannelGroup with the ChannelControl::addDSP function.
You also have the option of writing your own effects with System::createDSP. See the Digital Signal Processing (DSP) Architecture and Usage tutorial for more.