MPEG Rawesak Tanawongsuwan
[email protected]
Video Compression • Spatial (intra-frame) compression – Compress each frame in isolation, treating it as a bitmapped image
• Temporal (inter-frame) compression – Compress sequences of frames by only storing differences between them
Spatial Compression • Image compression applied to each frame • Can be either lossless or lossy • Motion JPEG – Purely spatial compression – Apply JPEG to each frame – Used by most analogue capture cards – No standard, but MJPEG-A format widely supported
Temporal Compression • A video consists of a time-ordered sequence of frames • Contents do not change rapidly between frames • A solution to video compression would be predictive coding based on previous frames
Temporal Compression • Compression proceeds by subtracting images – subtract in time order and code the residual error
Temporal Compression • We can do better by searching for parts of the image to subtract from the previous frame.
Video Compression with Motion Compensation • Exploit temporal redundancy • Steps of Video compression based on Motion Compensation (MC) – Motion Estimation (motion vector search) – MC-based Prediction – Derivation of the prediction error, i.e., the difference.
Motion Compensation • Each image is divided into macroblocks of size NxN
Motion compensation • Target frame: the current image frame • Reference frame: either previous or future frame • Goal: – predict the target macroblock from the reference macroblocks – For each macroblock in the target frame, find most similar macroblock in the reference frame
• Motion vector (MV): displacement of the reference macroblock to the target macroblock • Prediction error: difference of the two corresponding macroblocks
Motion Compensation
Search for Motion Vectors • The difference between two macroblocks can be measured by several criteria, e.g. – Mean Absolute Difference (MAD)
• Goal is to find a closest match within the searching window
Search methods • Sequential (simple) • Logarithmic • Hierarchical
H.261 • An earlier digital video compression standard • Its principle of MC-based compression is retained in all later • Video compression standards designed for videophone, video conferencing, other audiovisual services over ISDN
H.261 Frame Sequence
• Intra-frames (I-frames) vs. Inter-frames (P-frames) • I-frames: independent images, spatial redundancy coding, coding method similar to JPEG, usually sent a couple of times in each second of video • P-frames: not independent, coded by a forward predictive coding method, temporal redundancy coding • Motion vectors in H.261 are always measured in units of full pixel and they have a limited range of about 15 pixels
I-frame coding
• Macroblocks are of size 16x16 pixels for the Y frame, and 8x8 for Cb and Cr frames, since 4:2:0 chroma subsampling is employed. A macroblock consists of four Y, one Cb, and one Cr 8x8 blocks.
P-frame (predictive) coding
MPEG • MPEG: Moving Pictures Experts Group, established in 1988 for the development of digital video • MPEG-1,-2,-4,-7,-21,A,B,C,D,E,…
MPEG-1 • MPEG-1 adopts the CCIR601 digital TV format also known as SIF (Source Input Format) • MPEG-1 supports only non-interlaced video. • Normal picture resolution is – 352x240 for NTSC video at 30 fps – 352x288 for PAL video at 25 fps • 4:2:0 chroma subsampling
Motion compensation • Forward prediction
Problem finding the good match
Motion compensation • Future frame may help find a good match
• MPEG introduces B-frame • Bi-directional motion compensation
Bi-directional motion compensation
average
MPEG Frame Sequence
Sizes of MPEG-1 Frames • The typical size of compressed P-frames is significantly smaller than that of I-frames – temporal redundancy is exploited in inter-frame compression
• B-frames are even smaller than P-frames – the advantage of bi-directional prediction
MPEG-2 • Higher quality video at a bit-rate of more than 4 Mbps • Support interlaced video because of the options for digital broadcast TV and HDTV • Support various picture resolutions as defined by DVD, ATV and HDTV
MPEG-4 • MPEG-4 departs from its predecessors in adopting an object-based coding • Higher compression ratio, also beneficial for digital video composition, manipulation, indexing, and retrieval • The bit-rate for MPEG-4 video now covers a large range between 5 kbps to 10 Mbps.
Block-based vs Object-based coding
MPEG-4 Video Principle • Retained from predecessor – Block based natural video coding – Motion estimation and compensation – Variable length coding
• MPEG-4 Specific – Object based coding – Arbitrary shape coding – 2D & 3D mesh and texture coding