Using Sound
Overview
Adding sound to your experiences will enhance user interactivity and realism.
In 3DVIA Studio, you will be able to easily add sound to an experience during authoring, and control sound behavior during playtime.
In particular, you will be able to:
- Import compressed sound assets into your project
- Play sound with or without spatialization
- Capture sound during playtime
- Control sound playing during authoring, or at runtime using VSL and events
- Export your sound and publish your experience with sound
If you have the 3DVIA Studio SDK, you will further be able to develop your own sound engine, e.g. to provide extended sound asset support.
The workflow for adding sound to your experiences will be:
- Import a sound asset into your project or use the sound capture preset to capture your sound directly from 3DVIA Studio at runtime.
- Create a sound player and configure its properties to play a particular sound. You may use the Sound Toolbar to test your configuration (e.g. speed or volume).
- Configure the 3D settings of the sound (if sound needs to be spatialized), such as the sound Emission Cone and the used distance model.
- Build, export and publish your experience with sound. Sound assets processing options will be saved on the project configuration file on disk.
Supported import and export sound asset formats and codecs
- WAV/PCM (uncompressed): mono and stereo Microsoft PCM, Microsoft ADPCM, IMA ADPCM
- OGG/Vorbis
XACT files support
XACT sound banks and configuration files are supported on play only (Windows platforms only). They can still be configured but are not suitable for import.
Note that when a project with XACT files is loaded, the XACT sound engine is loaded. Visit the Migrate XACT Sound Assets to WAV/OGG Sound Assets to use the new sound engine.
Import
To import a sound into your project, you must simply drag and drop your sound asset to 3D View or to the Project Editor:
When the drop takes place, the asset import dialog will allow you to choose where to copy the asset to import:
By default, imported sound assets are copied under the project directory (“Sources” > “Sounds” folder). Visit the Asset Import section for general information on asset import.
Once imported, your sound will be available from the Project Editor (in the targeted stage’s “Resources” folder) and from the Resources Explorer (“Sound” category):
You can review the properties of an imported sound directly from the 3DVIA Studio application, by double-clicking on the sound resource or right-clicking on the resource and selecting “Edit”:
These properties include:
- Sampling Rate
- Format (Uncompressed, Vorbis, etc.)
- Size (Kbytes)
- Duration (in ms)
- Number of channels (Mono or Stereo)
and you can also access (read) them usingVSL methods and Building Blocks.
Once imported, you must create a sound player to play your sound. If you want to create a spatialized sound, you will also need a sound listener. The following sections explain how to play a sound.
Play Sound
In order to play your imported sound, you will need to use a sound player.
1. A sound player preset is available from the Libraries view > “Presets” > “Sound” category.
Create a sound player by drag and dropping the “Sound Player” preset from the Libraries view > “Presets” > “Sound” category to the 3D View or to the target stage in the Project Editor:
When the drop takes place, a “New Sound” player actor is created in the targeted stage.
Note: if you want to attach your sound to an existing element (e.g. a 3D element so as to create a spatialized sound), drag and drop the sound player preset to the element in question (see section “Spatialized Sound” below for more details).
2. Configure the sound player by selecting the “New Sound” actor in the Project Editor and, from the Property View, set its “Sound Resource” property to be one of your imported sounds (e.g. the “ring” sound imported in the previous section).
3. Leave the “3D> Node 3D” property set to “null” (which is the default) if you want to play the sound without spatialization.
You can also check the “Loop Mode” check-box if you want to loop the sound, and customize the sound “Speed” (tonality) and” Volume”. Visit the
sound player preset documentation for more information on sound player properties.
4. To play the sound only, use the play button
available from the sound toolbar, or use the project play button available from the playback toolbar to play the whole experience (and therefore the sound too).
Spatialized Sound
To create a spatialized sound, you’ll need an emitter (i.e. a 3D element used to set the sound in space), a sound player (used to play the sound, as explained above) and a sound listener (used as a receiver).
1. Create an emitter by drag and dropping a “3D Entity” preset from the Libraries view > “Presets” > “3D” category to the target stage.
In this example you will be using the resulting 3D actor as emitter. If you want, configure the emitter properties (such as position and orientation) from the Property View.
Note: skip this step if you wish to use one of your scene’s 3D entities as the emitter (for example, the speakers of a hi-fi system).
2. Create a sound player as an element attached to your emitter by drag and dropping the “Sound Player” preset from the Libraries > Presets > Sound category onto your emitter.
3. Configure the sound player: drag and dropping the sound player preset onto a 3D entity attaches a sound player to the 3D entity. The 3D entity is then automatically taken as the emitter for the sound that will be played by the sound player.
You will still need to configure the sound to play (i.e. the sound player’s “Sound Resource” property, as explained above for unspatialized sound), but the “3D” property of the sound player will be automatically set to be the attached 3D entity.
Alternatively, you can create the sound player as a separate entity and set the sound player’s “3D” > “Node 3D” property (i.e. the sound emitter) later during authoring or at runtime using VSL methods.
You can further configure the geometrical properties of your sound, such as:
- The local position of the emitter relative to the Node 3D position.
- The local orientation of the emitter relative to the Node 3D orientation.
- The sound emission cone (inner and outer dimensions) and the sound emission outer volume (volume of the sound when the listener is outside the emission cone) from the EmissionCone. Visit the Emission Cone documentation for a graphical representation of the emission cone.
- The distance or scale factor (e.g. to create an attenuation effect fitting the geometrical scale of the objects in your scene), by defining the roll-off factor and the reference distance to compute the sound attenuation based on distance.
Note: in the case of a hi-fi system, you may want to use several sound players, one attached to each speaker.
Tip: if you have one sound channel per asset, you may be able to simulate a multi-channeled sound system by associating each speaker to a sound player with different sound resource.
4. Create a sound listener: in the Project Editor or in the 3D View, locate the element that will be used as a sound receiver (e.g. a character) and drag and drop the sound listener preset (Libraries view > Presets > Sound) onto this element.
When the drop takes place, the new sound listener is created as a component attached to the targeted element, making the later one an element sensitive to all the sounds present in your experience.
Note: there can only be one active sound listener at a time within a single project. By default, a sound listener is active when created. If however there already is an active sound listener in your project, the sound listener will be deactivated upon creation. To activate it at a later stage, you will first need to deactivate the active sound listener (uncheck the “Active” sound listener property either from the Property View or using VSL to deactivate a sound listener), then activate your to-be-active sound listener by checking its “Active” property.
Sound Manipulators
When configuring the sound player, 3D manipulators let you:
- Modify the inner and outer cone angles
- Modify the sound position relative to its attached 3D node
- Modify the sound direction relative to its 3D point
When configuring the sound listener, 3D manipulators let you:
- Modify the listener position relative to its attached 3D node
- Modify the listener direction relative to its 3D point
For both the player and the listener, manipulators are displayed when:
- Placing the mouse cursor on the player/listener icon (but only for 1 or 2 seconds)
- Clicking the player/listener icon
- Selecting the player/listener component in the Project editor
Export & Publish
Once you have finished assembling and configuring your sound elements, go may want to go to the Build workset and tweak the sound options in the Project Build Explorer before exporting & publishing your experience. Sound assets processing options will be saved on the project configuration file saved on disk.
Visit the Export & Publication documentation for more information on exporting your experiences.
You may also be interested in reading the Vorbis license and the OpenAL license agreements.
Capture Sound
You can capture sound at runtime i.e. while the experience is playing.
To capture sound:
1. Drag and drop the sound capture preset from the Libraries view > Presets > Sound category to the 3D View or to the target stage in the Project Editor:
The capture sound component is capable of capturing the sound from a microphone into a PCM buffer in memory.
2. Select the just created sound capture in the Project Editor and configure the device to use for the sound capture from the Property View > “Source” category > “Audio Source” property:
All the sound capture devices recognized by OpenAL implementation should be available and work.
Note: captured sound is not and cannot be saved, but you can still use the Property View to review the properties of your sound capture.
Play Sound On Event
You can (also) configure your sounds so as to play at runtime on specific events using VSL methods.
VSL Sound Methods
You can use the VSL methods proper to the sound player and sound listener classes to play sound at runtime on specific events.
Check other available sound classes and methods from the Types view > Sound category.
Sound resource (vkSoundResource) VSL methods
- uint32 GetUncompressedSize ( ) const
Retrieve the estimated size (in Bytes) of the uncompressed sound data.
Vorbis sound resource (vkVorbisSoundResource) VSL methods
- const vkString& GetFormat ( ) const
Retrieve the description of the resource sound format (PCM, etc.).- uint32 GetOriginalSize ( ) const
Retrieve the size (in Bytes) of the original sound data (pure sound data, no headers, etc.).
This describes the size occupied by the sound data in memory.- float GetDuration ( ) const
Retrieve the estimated length (in seconds) of the original sound (playing at original speed and frequency).- uint32 GetChannelsCount ( ) const
Retrieve the number of channels of the resource (mono = 1, stereo = 2, etc.).- uint32 GetSampleRate ( ) const
Retrieve the sample rate (in Hertz) of the resource (22050 and 44100 are the most common values).- uint32 GetSampleSize ( ) const
Retrieve the size (in bits) of the resource sound sample.
Sound player (vkSound) VSL methods
- void Play (bool iStoppable)
Play/resume the sound resource attached to the sound player. Do nothing if no resource is attached.- void Stop ( )
Stop the sound.- void Pause ( )
Pause the sound.- void Resume ( )
Resume the sound.- void SetSoundResource (vkSoundResource* iSoundResource)
Set the sound resource to be played.- vkSoundResource* GetSoundResource ( )
Retrieve the sound resource to be played.- void SetSoundEnumValue (vkVariant& iSound)
Set the sound resource to be played based on the GUI enum value.- const vkVariant& GetSoundEnumValue ( ) const
Retrieve the sound resource to be played based on the GUI enum value.- bool32 IsPlaying ( )
Retrieve the play status of the sound. Return True if the sound is playing, false otherwise.- void SetNode3D(vkNode3D* iNode3d)
Define the 3D position of the sound emitter.- const vkNode3D* GetNode3D ( ) const
Retrieve the 3D position of the sound emitter.- vkPoint& GetLocalOrientation()
Retrieve the “delta” orientation of the emitter, relative to the attached 3D Node. This delta position is defined in local coordinates of the attached 3D Node.- void SetLocalOrientation(vkPoint& LocalOrientation)
Define the “delta” orientation of the emitter, relative to the attached 3D Node. This delta position is defined in local coordinates of the attached 3D Node.- const vkPoint& GetLocalPosition() const
Set the “delta” position of the emitter, relative to the attached 3D Node. This delta position is defined in local coordinates of the attached 3D Node.- void SetLocalPosition(vkPoint& LocalPosition)
Retrieve the “delta” position of the emitter, relative to the attached 3D Node. This delta position is defined in local coordinates of the attached 3D Node.- float GetPlayPositionTime();
Retrieve the current play position in seconds.- void SetLoopMode(bool isLooping)
Enable/disable looping mode.- bool GetLoopMode ( )
Retrieve the looping mode status (enabled/disabled).- error SetVolume (float VolumeValue)
Set the volume (must be positive or null and ≤ 1).- float GetVolume ( )
Retrieve the gain value (must be positive or null and ≤ 1).- error SetSpeed (float SpeedValue)
Set the peed (must be positive).- float GetSpeed ( )
Retrieve the speed value.- float GetRollOffFactor ( ) const
Retrieve the Roll-off factor. This coefficient is used to compute the sound attenuation (based on the distance between the listener and the emitter).- void SetRollOffFactor (float RolloffFactor)
Define the Roll-off factor. This coefficient will be used to compute the sound attenuation (based on the distance between the listener and the emitter).- float GetOuterAngle ( ) const
Retrieve the angle of the outer emission cone.- Error SetOuterAngle (float OuterAngle)
Define the angle of the outer emission cone.- float GetInnerAngle ( ) const
Retrieve the angle of the inner emission cone.- void SetInnerAngle(float InnerAngle)
Define the angle of the inner emission cone.- float GetOuterVolume ( ) const
Retrieve the volume of the sound outside the outer cone of the emitter.- void SetOuterVolume (float OuterVolume)
Define the volume of the sound outside the outer cone of the emitter.
Sound listener (vkSound3DListener) VSL methods
- void SetNode3D (vkNode3D* iNode3d)
Define the 3D position of the sound listener.- const vkNode3D* GetNode3D ( ) const
Retrieve the 3D position of the sound listener.- const vkPoint& GetLocalPosition( ) const;
Retrieve the ”delta” position of the listener, relative to the attached 3D Node. This delta position is defined in coordinates local to the attached 3D Node.- void SetLocalPosition (vkPoint& LocalPosition);
Set the ”delta” position of the listener, relative to the attached 3D Node. This delta position is defined in coordinates local to the attached 3D Node.- vkXYZEulerAngles GetlocalOrientation();
Retrieve the ”delta” orientation of the listener, relative to the attached 3D Node. This delta position is defined in coordinates local to the attached 3D Node.- void SetLocalOrientation (vkXYZEulerAngles& iLocalOrientation);
Define the ”delta” orientation of the listener, relative to the attached 3D Node. This delta position is defined in coordinates local to the attached 3D Node
Sound capture (vkSoundCapture) VSL methods
- void Start ( );
Start the Capture: Start to fill the resulting sound buffer.- void Stop ( );
Stop the Capture: Do not fill the resulting sound buffer anymore.
If Start is called after a call to Stop, the attached sound buffer will be overwritten.- void Pause ( );
Pause the Capture: Do not fill the resulting sound resource anymore.
If Resume is called after a call to Pause, the attached sound buffer will be concatenated.- void Resume ( );
Resume the Capture (only if Sound Capture was paused): Concatenate the existing sound resource with the new captured data.- bool32 IsCapturing ( ) const;
Return True if the component is capturing sound. False otherwise.- uint32 GetBufferSize ( ) const
Retrieve the Size (in Bytes) of the buffer to fill with audio PCM data when capturing.- uint32 GetChannelsCount ( ) const
Retrieve the channel number of the captured sound data.- uint32 GetSampleRate ( ) const
Retrieve the sample rate (in Hertz) of the captured sound data.- uint32 GetSampleSize ( ) const
Retrieve the sample size (in bits) of the captured sound data.- float FillSoundResource (vkSoundResourcePtr& oSoundResource) const;
Fill a Sound resource with the whole captured data.- float FillSoundResource (float iBeginTime, float iDuration, vkSoundResourcePtr& oSoundResource) const;
Fill a given Sound resource with a part of the captured sound data (given a start time and a duration in seconds). If the available sound data is not as long as required, fill the resource with the maximum amount of data. Returns the duration (in seconds) of the captured sound data copied in the sound resource.- error FlushBuffer ( );
Reset the captured Buffer. All the captured data will be flushed and the buffer size will be set to zero.- float GetBufferDuration ( ) const;
Retrieve the Size (in Seconds) of the buffer filled with audio data.- error SetAudioSource (EAudioSource iAudioSource)
Set the audio source (i.e. the device on which sound will be captured).- EAudioSource GetAudioSource ( ) const
Retrieve index of the current audio source.
Sound manager (vkSoundManager) VSL methods
- float GetDistanceScale ( )
Retrieve the distance scale. This scale will be used to compute the sound attenuation (based on the distance between the listener and the emitter). Use this value to define the distance unit of the sound engine i.e. to scale all the sounds of your experience(s).
Note: this value will be combined with the roll-off factor (which is defined only for a sound).- void SetDistanceScale (float DistanceScale)
Define the distance scale. This scale will be used to compute the sound attenuation (based on the distance between the listener and the emitter). Use this value to define the distance unit of the sound engine i.e. to scale all the sounds of your experience(s).- uint32 GetAudioInputsCount ( ) const
Retrieve the number of available audio input devices.- error GetAudioInputName (uint32 iIndex, vkString& oDeviceName) const
Get the audio input name given its ID.
Known Issues
- Import asset but asset already exists: an error message is displayed when the same asset is imported twice.
- Play 3D sound but no listener is defined: a default listener is defined at coordinates (0,0,0).
- Play 3D sound but no 3D entity is attached: no error will be displayed but the sound will be considered as not spatialized.
- Play 3D sound but no 3D entity is attached to the listener: a default position is defined at coordinates (0,0,0) and an error message is displayed.
- Sound resource is not exported: an error message is displayed in the error log.