Download Interactive Multimedia Sound

Interactive Multimedia
Sound
Mikael Fernström
Data sources
• Microphones and transducers
– Sample acoustic reality
• Synthesis
– Simulate reality (and beyond ;-)
– Many different forms of synthesis, e.g. additive,
subtractive, FM, FOF, PM…
• Algorithms
– e.g. auralisation/sonification
Primary processing
• Recording
– Get clear recordings!
– Signal-to-noise ratio
– Acoustics
• Filtering
– Supressing or emphasizing parts of the spectrum
• Mixing
– Blending of several sounds sources, foreground,
background, speech, music, f/x
Sampling
• “So called” CD quality
– 16 bit, 44.1 kHz
– => 90 dB dynamic range
– Does not match normal human perception!
• Current professional standards
– 24 bit, 96 kHz
– => 138 dB dynamic range
Sampling…
• 8 bit sounds crackling, noisy
• Sample rates lower than 44.1 kHz, loss of
treble
• Example:
– Lowest standard encoding for human speech
approx 6 bits 5 kHz (sounds like bad CB radio)
Quantisation
Simple sine wave
Spectrum of sine
Noise spectrum
Aliasing
sound
FFT
Aliasing
Half of the FFT array gets reflected into
the other half of the array
Hearing threshold
Masking
Formats
• Common formats:
– WAV, AIFF, RAW…
– Stores data “as is”
• Masking is used in perceptual compression,
e.g. MP3, AAC, etc…
• MP3 is not MPEG!
– It’s Layer 3 of MPEG-1 standard
Streaming
• Download complete file before playing
• Start playing file when length of file is
sufficient to accommodate rest of download
• Start playing when length of buffer matches
data rate
Streaming
MIDI
• Musical Instrument Digital Interface
• Only contains control info, no sound!
– e.g. note-on, note-off, velocity/loudness
MIDI
• Sounds different depending on User synth,
mapping, etc.
• General MIDI “standard”, GM
• Lack of expression, only 7 bit
• Sometimes problems with latency, 32.5 kb/s
– Playing 10 notes simultaneously => 9.2 ms