Skip to content

audio.md

Latest commit

 

History

History
317 lines (207 loc) · 10.1 KB

audio.md

File metadata and controls

317 lines (207 loc) · 10.1 KB
Raw

Basic Understanding of Audio Formats and Codecs

Version: 1.0
Date: 01-2026

This section provides a practical overview of common audio formats and codecs, with emphasis on identification, compression efficiency, and relevance to embedded camera systems.

Audio Codecs vs Containers

It is important to distinguish between:

  • Audio codec: compression/decompression algorithm (e.g., G.711, AAC, MP3, Opus, FLAC)
  • Container / file format: packaging structure for audio data (e.g., WAV, AIFF, OGG, MP4)
  • Protocol / payload format: transport mapping (e.g., RTP payloads)

Many misunderstandings in embedded systems come from mixing these concepts.

File Type Identification (Magic Bytes)

Many audio formats can be identified by their initial byte signatures.

Format Magic Bytes (Hex) Notes
WAV 52 49 46 46 ("RIFF") RIFF container, commonly PCM audio
MP3 49 44 33 ("ID3") or FF FB / FF F3 / FF F2 ID3 tag is optional; MPEG frame sync may appear first
AAC (ADTS) FF F1 or FF F9 ADTS header for raw AAC streams
FLAC 66 4C 61 43 ("fLaC") Lossless, open-source
OGG Vorbis 4F 67 67 53 ("OggS") OGG container, Vorbis codec
AIFF 46 4F 52 4D ("FORM") Uncompressed PCM, Apple ecosystem
WMA ASF GUID-based header No simple short magic sequence

Notes

  • AAC inside MP4 does not use ADTS headers.
  • AAC over RTP typically uses LATM or MPEG-4 generic payload formats, not raw ADTS framing.
  • MP3 detection should consider both ID3 tags and MPEG frame sync bits.
  • WMA typically uses ASF container headers rather than short magic bytes.

Special Case: MIDI

MIDI (Musical Instrument Digital Interface) is not an audio codec.

  • MIDI files do not contain waveform audio
  • They store musical events (note on/off, velocity, control messages)
  • Sound generation depends on synthesizers or software instruments

As such, MIDI is not applicable to camera audio pipelines.

Compression Efficiency: AAC vs MP3

AAC (Advanced Audio Coding)

  • Higher compression efficiency than MP3
  • Better quality at the same bitrate
  • Widely used in streaming and mobile ecosystems
  • Common in MP4-based recording and cloud storage

MP3 (MPEG Audio Layer III)

  • Older generation codec
  • Broad device compatibility
  • Less efficient than AAC at low bitrates
  • Still used due to legacy support

Summary: AAC generally provides better quality-to-bitrate performance than MP3, especially at low and medium bitrates.

Codec Quality and Performance Evaluation

Codec Compression Efficiency Audio Quality Notes
Opus Excellent Excellent Designed for low-latency and packet loss resilience
AAC Very good Very good Popular for music and recording
MP3 Good Good Legacy codec with wide support
FLAC Lossless Excellent Large size, not suitable for streaming
PCM (WAV/AIFF) None Excellent Uncompressed, high bandwidth/storage
G.711 (μ-law / A-law) Low Narrowband Log-companded PCM, fixed 64 kbps

Important Clarification on G.711

G.711 is not uncompressed PCM.
It uses logarithmic companding to reduce dynamic range while maintaining simple decoding and minimal latency.

Standards and References for Audio Codecs and Formats

This section lists authoritative RFC and ISO/IEC references for each audio codec and format discussed in this document. Only primary specifications and widely accepted standards are included.

G.711 (μ-law / A-law)

Purpose: Narrowband speech codec for telephony, VoIP, RTSP cameras

Notes for camera systems

  • Fixed bitrate: 64 kbps
  • Very low latency
  • High interoperability with RTSP/NVR clients

PCM / WAV

Purpose: Uncompressed audio, diagnostics, local recording

Notes

  • WAV is a container, not a codec
  • Usually contains LPCM (Linear PCM)

AAC (Advanced Audio Coding)

Purpose: High-efficiency lossy audio for recording and streaming

Notes

  • AAC in MP4 ≠ AAC ADTS
  • Licensing may apply depending on distribution

MP3 (MPEG Audio Layer III)

Purpose: Legacy lossy audio format

Notes

  • Less efficient than AAC and Opus
  • Still widely supported for playback

Opus

Purpose: Low-latency, real-time speech and music (WebRTC, talk-back)

Notes

  • Variable bitrate, variable frame size
  • Designed for packet loss resilience
  • Strongly recommended for interactive camera audio

FLAC (Free Lossless Audio Codec)

Purpose: Lossless audio storage

Notes

  • Lossless compression
  • Not suitable for real-time camera streaming due to bandwidth

OGG Vorbis

Purpose: Open-source lossy audio format

Notes

  • OGG is a container
  • Vorbis is the codec

MIDI

Purpose: Musical control data (not audio)

Notes

  • MIDI is not an audio codec
  • Not applicable to camera audio pipelines

WMA (Windows Media Audio)

Purpose: Proprietary audio codec family

Notes

  • Proprietary
  • Rarely used in embedded camera systems

Summary Table (Quick Reference)

Codec / Format Type Main Standard
G.711 Codec ITU-T G.711
PCM / WAV Codec / Container RFC 2361, RIFF
AAC Codec ISO/IEC 14496-3
MP3 Codec ISO/IEC 11172-3
Opus Codec RFC 6716
FLAC Codec RFC 9639
OGG Vorbis Container + Codec RFC 3533
MIDI Control protocol MIDI 1.0 Spec
WMA Codec ASF Spec

Why These References Matter for Camera Systems

  • RFCs define interoperability (RTP, RTSP, WebRTC)

  • ISO/IEC defines codec bitstreams and decoding correctness

  • ITU-T ensures telecom compatibility

  • Using standards avoids:

    • Vendor lock-in
    • Interop bugs with NVR/VMS
    • Incorrect SDP / payload assumptions

Codec Ranking for Camera Systems (Practical View)

  1. Opus – Best overall for real-time, low-latency audio and talk-back
  2. AAC – Best for recording, storage, and cloud streaming
  3. MP3 – Legacy compatibility
  4. FLAC – Archival, non-real-time use
  5. PCM (WAV/AIFF) – Debugging, lab capture
  6. G.711 – Legacy RTSP, VoIP-style audio

Key Takeaways for Embedded Cameras

  • Codec choice must balance latency, CPU usage, bandwidth, and interoperability
  • For talk-back: Opus or G.711
  • For recording: AAC
  • For diagnostics: PCM/WAV
  • Containers and payload formats matter as much as the codec itself