multimedia context PDF

Preview

Summary

reviewer for multimedia...

Description

Audio in Multimedia - In a multimedia production, sound and music are crucial in helping to establish moods and create environments. Audio in PCs Many types of sounds are accessible with a PC. They include:  Music  Sound effects  Spoken narration  Video soundtracks  Real-time telephone conversations  Operating system alerts and prompts Digital Audio Recording - Digital recording devices capture sound by sampling the sound waves. Digitization of Sounds - Sound is a continuous wave that travels through the air - The wave is made up of pressure differences. Sound is detected by measuring the pressure level at a particular location. - Sound waves have normal wave properties (reflection, refraction, diffraction, etc.). Digital Audio Quality The quality and size of digital audio depends on:  The sampling rate  The sample size  The number of channels  The time span of the recording What is Streaming Audio? Streaming audio plays as it reaches your PC, making it unnecessary to wait until the entire file is downloaded to the computer. Audio File Formats An audio file’s format determines what files a PC can open and play, and how much space the file occupies on a disk. File formats include:  MP3  WAV  MIDI MP3 Format MP3 is a standard format for music files sent over the Internet. MP3s:  Use one of three MPEG standards for audio compression  Can compress an audio file to about one-twelfth of the space it occupies on a CD with no significant loss of sound quality WAV Format WAV is a standard for sound files on Windows and Macintosh PCs. WAVs: o Do not compress audio as much as MP3s o Are generally used for sound effects and other small files MIDI Format MIDI is a method and format for recording music from synthesizers and other electronic instruments. MIDIs:  Are created with a computer that has a sequencer  Do not contain actual musical notes

 Are small and load quickly on a Web site Audio Software for the PC Most new PCs come with some software and hardware for recording and managing audio files.  Audio editing software allows you to edit audio files and convert them from one format to another.  MIDI software includes programs for recording, storing, replaying, and editing MIDI files.  Composition software allows you to create sheet music for many voices or instruments. Audio Hardware Devices Audio hardware devices for the PC may include:  Audio or sound cards  Speakers  Microphones for voice input  MIDI input devices  CD/DVD burners Multimedia Data Representation  Digital Audio  Sampling/Digitisation  Compression (Details of Compression algorithms – following chapters)  Graphics/Image Formats  Digital Video (Basics of Video, Didgiatl Video, closely entwined with Compression) Digital Audio Application of Digital Audio - Selected Examples  Music Production – Hard Disk Recording – Sound Synthesis – Samplers – Effects Processing  Video - Audio Important Element: Music and Effects  Web - Many uses on Web – Spice up Web Pages – Listen to Cds – Listen to Web Radio Digitization of Sounds  Sound is a continuous wave that travels through the air  The wave is made up of pressure differences. Sound is detected by measuring the pressure level at a particular location.  Sound waves have normal wave properties (reflection, refraction, diffraction, etc.).  Source  Generates Sound  Air Pressure changes  Electrical - Loud Speaker  Acoustic - Direct Pressure Variations  Destination  Receives Sound  Electrical - Microphone produces electric signal  Ears - Responds to pressure hear sound



Do not contain sound waves or use sampling

–

Sound is required input into a computer: it needs to sampled or digitised:

–

Microphones, video cameras produce analogue signals (continuous-valued voltages).

– –

Continuous Analogue Waveform To get audio or video into a computer, we have to digitize it (convert it into a stream of numbers). Need to convert Analogue-to-Digital Specialised Hardware So, we have to understand discrete sampling (both time and voltage)

–

–

–





Sampling - divide the horizontal axis (the time dimension) into discrete pieces. Uniform sampling is ubiquitous. Quantization - divide the vertical axis (signal strength) into pieces. Sometimes, a non-linear function is applied. o 8 bit quantization divides the vertical axis into 256 levels; 16 bit gives you 65536 levels. That is the basic idea of digitizing a sound unfortunately things are (practically speaking) not so simple. Questions for producing digital audio (Analogue-to-Digital Conversion): 1. How often do you need to sample the signal? 2. How good is the signal? 3. How is audio data formatted?

Computer Manipulation of Sound  Once Digitised processing the digital sound is essentially straightforward although it depends on the processing you wish to do (e.g. volume is easier to code than accurate reverb) Sample Rates and Bit Size How do we store each sample value (Quantisation)?  8 Bit Value – (0-255)  16 Bit Value – (Integer) (0-65535) How many Samples to take? – 11.025 KHz Speech (Telephone 8KHz) – 22.05 KHz Low Grade Audio (WWW Audio, AM Radio) – 44.1 KHz CD Quality Implications of Sample Rate and Bit Size Affects Quality of Audio  Ears do not respond to sound in a linear fashion  Decibel (dB) a logarithmic measurement of sound  16-Bit has a signal-to-noise ratio of 98 dB virtually inaudible  8-bit has a signal-to-noise ratio of 50 dB  Therefore, 8-bit is roughly 8 times as noisy – 6 dB increment is twice as loud Typical Audio Formats  Popular audio file formats include .au (Unix workstations), .aiff (MAC, SGI), .wav (PC, DEC workstations)  A simple and widely used audio compression method is Adaptive Delta Pulse Code

predicts the next sample and encodes the difference between the actual value and the predicted value. Delivering Audio over a Network  Trade-off between desired fidelity and file size Bandwidth Considerations for Web and other media. Compress Files: Could affect live transmission on Web Streaming Audio Buffered Data:  Trick get data to destination before it's needed  Temporarily store in memory (Buffer)  Server keeps feeding the buffer  Client Application reads buffer Needs Reliable Connection, moderately fast too. Specialised client, Steaming Audio Protocol (PNM for real audio). Introduction to MIDI Definition of MIDI: a protocol that enables computer, synthesizers, keyboards, and other musical device to communicate with each other Components of a MIDI System: Synthesizer (a sound generator) Sequencer (a music editor) Track (to organize the recordings) Channel (to separate information in a MIDI system) Timbre (the quality of the sound) Pitch (musical note that the instrument plays) Voice (the portion of the synthesizer that produces sound) Patch (the control settings that define a particular timbre) Digital Audio and MIDI  There are many application of Digital Audio and MIDI being used together:  Modern Recording Studio - Hard Disk Recording and MIDI – Analogue Sounds (Live Vocals, Guitar, Sax etc) - DISK – Keyboards, Drums, Samples, Loops Effects - MIDI  Sound Generators: use a mix of – Synthesis – Samples  Samplers - Digitise (Sample) Sound then – Playback – Loop (beats) – Simulate Musical Instruments  Digital Audio, Synthesis, Midi and Compression MPEG 4 Structured Audio\We have seen the need for compression already in Digital Audio Large Data Files 



Basic Ideas of compression (see following Chapters) used as integral part of audio format MP3, real audio etc. Mpeg-4 audio - actually combines compression synthesis and MIDI to have a massive impact on compression.

Modulation (ADPCM). Based on past samples, it

MIDI, Synthesis encode what note to play and how to play it with a small number of parameters - Much greater reduction than simply having some encoded bits of audio.  Responsibility to create audio delegated to generation side MPEG 4 Structured Audio MPEG-4 covers the whole range of digital audio:  from very low bit rate speech  to full bandwidth high quality audio  built in anti-piracy measures  Structured Audio Structured Audio Tools MPEG-4 comprises of 6 Structured Audio tools 

Video Video Concept  Video places the highest performance demand on computer and its memory and storage.  Digital video has replaced analog video as the method of choice for making and delivering video for multimedia.  Digital video device produces excellent finished products at a fraction of the cost of analog.  Digital video eliminates the image-degrading analog-to-digital conversion.  Many digital video sources exist, but getting the rights can be difficult, time-consuming, and expensive. Analogue Video  Video information that is stored using television video signals, film, videotape or other noncomputer media  Each frame is represented by a fluctuating voltage signal known as an analogue wave form or composite video.  Composite analogue video has all the video components: o brightness, colour and synchronization  Then combined into one signal for delivery  Example : traditional television  Problems: colour blending, low clarity, high generation lost, difficult to edit. Digitizing Video  Digital video combines features of graphics and audio to create dynamic content for multimedia products.  Video is simply moving pictures.  Digitized video can be edited more easily.  Digitized video files can be extremely large.  Digital video is often used to capture content from movies and television to be used in multimedia.  A video source (video camera ,VCR, TV or videodisc) is connected to a video capture card in a computer.  As the video source is played, the analog signal is sent to the video card and converted into a digital file (including sound from the video).  Digital video is the digitisation of analogue video signals into numerical format

It creates the illusion of full motion by displaying a rapid sequence of changing images on a display device.  Conversion from analogue to digital format requires the use on an ADC (Analogue to Digital Converter)  A Digital to Analogue Converter (DAC) can be used to output digital video on analogue equipment  Video clip stored on any mass-storage device can be played back on a computer’s monitor without special hardware.  Setting up a production environment for making digital video, requires some hardware specifications.  Some specifications include computer with FireWire connection and cables, fast processor, plenty of RAM, fast and big hard disk. File Size and Formats  There is an important consideration: o file size in digitized video which included  frame rate  image size  Color depth.  Frame Rate o Animation is an illusion caused by the rapid display of still images. o Television and movies play at 30 fps but acceptable playback can be achieved with 15 fps.  Image Size o A standard full screen resolution is 640x480 pixels but to safe storing space a video with 320x240 for a computer display is still acceptable. o New high-definition televisions (HDTV) are capable of resolutions up to 1920×1080p60,  1920 pixels per scan line by 1080 scan lines, progressive, at 60 frames per second.  Color Depth o The quality of video is dependent on the color quality (related to the number of colors) for each bitmap in the frame sequence. o The color depth below 256 colors is poorer-quality image. o The frame rate to below 15 fps causes a noticeable and distracting jerkiness that unacceptable. Changing the image size and compressing the file therefore become primary ways of reducing file size.  Linear o It plays end to end in one direction, usually pertains to videotape editing specifically the editing of linear tape segments into one final master tape.  Non-linear o Refers to the editing of disk-based digital video. ...