Rtp Tutorial

  • May 2020
  • PDF

This document was uploaded by user and they confirmed that they have the permission to share it. If you are author or own the copyright of this book, please report to us by using this DMCA report form. Report DMCA


Overview

Download & View Rtp Tutorial as PDF for free.

More details

  • Words: 5,631
  • Pages: 85
RTP: Multimedia Streaming over IP

Colin Perkins USC Information Sciences Institute

Internet Multimedia • Internet Multimedia has long history: – RFC 741, "Network Voice Protocol", 1977 – First video experiments in the early 1980s

• Modern standards development began in 1992: – Developing from teleconferencing systems – Audiocast of IETF meetings • 20 sites on 3 continents • Precursors to RTP and the present standards

– Standardized RTP in 1996

• Widespread availability of suitable networks in the last couple of years • Unusual characteristics, worth exploring Copyright © 2002 Colin Perkins

Talk Outline • Use of IP for real-time traffic • Protocols for real-time multimedia over IP – Outline of signalling protocols – Outline of media transport protocols

• RTP: Real-time Transport Protocol – RTP data transfer protocol – RTP control protocol

• Robustness – – – –

Playout and timing correction Error correction Security Congestion control

• Security • Conclusions

Copyright © 2002 Colin Perkins

Talk Outline • Use of IP for real-time traffic • Protocols for real-time multimedia over IP – Outline of signalling protocols – Outline of media transport protocols

• RTP: Real-time Transport Protocol – RTP data transfer protocol – RTP control protocol

• Robustness – – – –

Playout and timing correction Error correction Security Congestion control

• Security • Conclusions

Copyright © 2002 Colin Perkins

The IP Protocol Stack Application programs HTML

MIME

Media codecs

HTTP SMTP RTP FTP SIP RTSP TCP

UDP

IP ADSL Ethernet PPP Twisted Pair Optical Fibre Wireless

• IP forms an abstraction layer – Applications and transport protocols above – Assorted link technologies below

Copyright © 2002 Colin Perkins

• Applications can't see the link layers – Just see the performance of the IP layer – Must assume lowest common denominator behaviour

• Link layer can't tell the needs of the application – Just see a series of packets – Optimisations for particular traffic classes are risky – Is the traffic really what you think?

• Decoupling applications from the network

IP Service Model The IP service model is limited • Best effort packet transport • Fragmentation • Routing and addressing

Copyright © 2002 Colin Perkins

Best Effort Packet Transport • Performance not guaranteed • Packets can be… – – – – –

lost delayed reordered duplicated corrupted

…and the transport protocol must compensate • Checksum to catch bit errors

• Many causes of problems: – – – – – – –

Congestion may cause loss Packet corruption may cause loss Route changeover may cause loss Queuing delay Multi-path IP routing may reorder Link-layer striping may reorder Spurious retransmission and router bugs cause duplicates

Assumption: Significant packet loss

Copyright © 2002 Colin Perkins

Packet Loss Patterns

Packet Loss Rate (percent)

45 40 35 30 25 20 15 10 5 0 8:00

10:00

12:00

14:00

Time of day

Copyright © 2002 Colin Perkins

16:00

18:00

Timing Disruption

45º line: clocks are synchronized

Reception time

Discontinuity due to route change.

Queuing jitter causes variation in inter-packet arrival time. Non-45º slope shows presence of clock skew Transmission time

Copyright © 2002 Colin Perkins

Best Effort Transport • Performance can be bad – Applications should be prepared to compensate – Doesn't have to be!

• Loss and jitter can be made arbitrarily low through careful engineering – Most backbone networks have very good performance – Interconnects and customer LANs are currently the main trouble spots – Explicit QoS doesn't appear necessary

Copyright © 2002 Colin Perkins

Fragmentation • IP fragments packets that exceed the MTU – Often >1500 octets

• This causes an undesirable loss multiplier effect – Loss of one fragment means the others must also be discarded – Better for the application to fragment, and generate small IP packets

Original Fragments

Packet loss Reconstructed

Copyright © 2002 Colin Perkins

Routing and Addressing • Most communication is unicast – Point-to-point

• Inefficient for many receivers – The server must generate n copies of the data, for n clients – Significant scaling bottleneck

• Multicast allows the server to scale – Adds network complexity

Copyright © 2002 Colin Perkins

• The use of IP with best effort transport implies heterogeneity • Multicast makes this an order of magnitude worse – Each receiver sees different loss characteristics – Hard to get timely feedback in a scalable manner

Properties of IP: Summary • Best effort, unreliable, packet delivery service – Much of the network performs well – Peering points and customer premises often lacking

• Provides a fragmentation service – Typically best avoided

• Provides a multicast service – In some parts of the network • 10% of broadband connections

• At best: simple, effective, service

Copyright © 2002 Colin Perkins

Transport Protocols • The IP service, by itself, is very limited – Just (tries to) deliver packets

• Always augmented by a transport protocol – TCP – UDP – (others in development)

Copyright © 2002 Colin Perkins

TCP • Enhances the raw IP service – – – –

Point-to-point and connection oriented Service selection through ports Reliable, in-order, delivery Rate adaptation to match network capacity

Copyright © 2002 Colin Perkins

TCP

– High link utilization – Fair share between flows

• Retransmission ensures that no data is lost

Congestion Window Size

• Rate adaptation matches throughput to capacity

– Reliable, in-order, delivery Time Slow start

Copyright © 2002 Colin Perkins

Slow start Congestion avoidance

Congestion avoidance

UDP • Uses the services of IP – – – –

best effort (unreliable) but timely fragmentation routing and addressing unicast and multicast

• Provides ports, in addition to IP addressing, but no other services

Copyright © 2002 Colin Perkins

Reliability/Timeliness Tradeoff

Reliable

Unreliable

Not timely

Timely

TCP

UDP RTP

• Protocols built on unreliable packet networks must make a fundamental trade-off: – Timely – Reliable

• TCP is at one extreme, UDP the other

Copyright © 2002 Colin Perkins

• Multimedia systems choose their transport carefully: – TCP for signalling – UDP for media data

• Application level protocols can blur the boundary – E.g. RTP for multimedia data – Essential for performance

Talk Outline • Use of IP for real-time traffic • Protocols for real-time multimedia over IP – Outline of signalling protocols – Outline of media transport protocols

• RTP: Real-time Transport Protocol – RTP data transfer protocol – RTP control protocol

• Robustness – – – –

Playout and timing correction Error correction Security Congestion control

• Security • Conclusions

Copyright © 2002 Colin Perkins

The Multimedia Protocol Framework Two fundamental components of the framework: • Signalling protocols • Media transport protocols

Copyright © 2002 Colin Perkins

Signalling Protocols The first stage of any multimedia session is signalling • User location • Session initiation, call setup and teardown • Media negotiation • Conference control Many signalling protocols exist • Teleconferencing: H.323 • Telephony: SIP • Streaming: RTSP, SAP

Copyright © 2002 Colin Perkins

Signalling Protocols: H.323 • Original signalling protocol for IP-based multimedia – Extension of H.320 ISDN conferencing to IP – Tightly coupled, small group, video conferencing

• Flexible media negotiation and call control • Reputation for complexity – ASN.1 encoding – Many RTT call setup • (mostly fixed in later versions)

Copyright © 2002 Colin Perkins

Signalling Protocols: SIP • RFC 2543 (recently updated) • SIP is used to invite someone to join a session – Media negotiation – User location – Call setup and teardown

• Considerable overlap with H.323 – More flexible integration with other Internet services • Email, Web, streaming media, recording, agents, etc.

– More limited media negotiation and call control • Extensions underway

– Very different style • Protocol operation is based on HTTP • Reuses much existing infrastructure Copyright © 2002 Colin Perkins

Signalling Protocols: RTSP • RFC 2326 • Designed for control of a media on demand server – Point-to-point VCR-style remote control – Record/play/rewind/fast-forward

• Widespread commercial use… – RealAudio, QuickTime

• May also be useful for controlling other devices – Voice mail – Interactive voice response

• Leverages HTTP and SIP infrastructure

Copyright © 2002 Colin Perkins

Signalling Protocols: SAP • RFC 2974 • A multicast announce-listen protocol for wide area announcement of multimedia sessions – Announcers periodically multicast SDP descriptions to a well known group – Inter-announcement interval is 10+ minutes – Listeners slowly build up a cache of sessions

• Suitable for announcing long-lived public sessions – E.g. radio/TV station, event coverage

• Mostly used with IP multicast – Talk of use with cable networks

Copyright © 2002 Colin Perkins

Media Transport Protocols Once the session has been setup, media flows Convergence on a single media transport protocol for: • Voice over IP • Teleconferencing • Streaming media Real-time Transport Protocol, RTP • Flexible, supports many codecs and media types • Extensible to new scenarios

Copyright © 2002 Colin Perkins

The Multimedia Protocol Framework

Call control Media negotiation RTSP

Light weight sessions

SIP

Media

Media

codecs

codecs

RTP

RTP

SAP

TCP

UDP IP

IETF Multimedia Protocol Stack

Copyright © 2002 Colin Perkins

Call control

RAS

Media negotiation H225.0

UDP

H.245 TCP

IP ITU Teleconferencing Protocols

Design Choices • Depending on the scenario, implement: – An appropriate signalling protocol • RTSP • SIP/H.323 • SAP

– Media transport using RTP • One or more codecs – MPEG – H.263

• Error correction and concealment • Congestion Control

Copyright © 2002 Colin Perkins

Talk Outline • Use of IP for real-time traffic • Protocols for real-time multimedia over IP – Outline of signalling protocols – Outline of media transport protocols

• RTP: Real-time Transport Protocol – RTP data transfer protocol – RTP control protocol

• Robustness – Playout and timing correction – Error correction – Congestion control

• Security • Conclusions

Copyright © 2002 Colin Perkins

RTP: Real-time Transport Protocol • The standard for real-time transport over IP networks – Streaming audio and video – Voice over IP

• Published as an IETF proposed standard – – – –

RFCs 1889 and 1890 in January 1986 Adopted by ITU as part of H.323 Adopted by 3GPP for next generation cellular telephony Widespread use in streaming: QuickTime, Real, Microsoft • (HTTP streaming still common)

• Recently revised for draft standard status – Work complete, awaiting publication – Changes include: • Clarifications and bug-fixes based on experience • Scalability improvements • Support for new codecs

– 100% backwards compatible Copyright © 2002 Colin Perkins

Philosophy of RTP • The challenge: – build a mechanism for robust, real-time media delivery above an unreliable and unpredictable transport layer – without changing the transport layer

Push responsibility for media

Make the system robust to

delivery onto the end-points

network problems; media

where possible

data should be loss tolerant

The end-to-end argument

Copyright © 2002 Colin Perkins

Application level framing

The End-to-end Argument • Two options for ensuring reliability – Pass responsibility hop-by-hop, along with the data • Email

– Responsibility remains with the end points, which ensure delivery even if the intermediate steps are unreliable

• Both TCP and RTP take the second approach • Consequences: – Intelligence tends to "bubble-up" the protocol stack to the end points – The intermediate systems can be simple, and need not be robust • They can simply discard data they cannot deliver, since it will be recovered end-to-end

• The network is dumb, but end-points are smart Copyright © 2002 Colin Perkins

Application Level Framing • Only the application has sufficient knowledge of its data to make an informed decision about how that data should be transported • Implications: – The transport protocol should accept data in application meaningful chunks ("ADUs") • The application must understand the data, • The application must be able to process ADUs independently, in arbitrary order, and in the presence of loss

– The transport protocol should expose details of delivery, allowing the applications to react intelligently if there are problems • • • •

Blind retransmission is not always appropriate Maybe the data is stable, and an updated version can be sent Maybe the data is obsolete, and doesn't need to be resent Maybe an alternate representation of the data can be sent

Copyright © 2002 Colin Perkins

Philosophy of RTP • The philosophy of RTP implies smart, network-aware, applications that are capable of reacting to problems end-to-end. – Both sender and receiver are intelligent – The network is dumb and can be unreliable

• Similar principles apply to the signalling protocols – Mostly end-to-end operation, limited support for network state

• Fits well with the IP service • Contrast with traditional applications: – Telephone network is smart, end-points are dumb – MPEG sender is smart, receiver relatively dumb

Copyright © 2002 Colin Perkins

Protocol Components Payload Payload Payload Payload Format Format Format Format

RTP Profile

RTP Data Transfer Protocol UDP IP

Copyright © 2002 Colin Perkins

RTP Control Protocol

Protocol Components

RTP Data Transfer Protocol

• Transports application data units – Audio – Video

• One RTP stream transports each media type Copyright © 2002 Colin Perkins

• Provides: – – – –

Source identification Payload identification Media sequencing Timing recovery

• Extensions allow for error correction

Protocol Components

RTP Control Protocol

• Reception quality feedback – Packet loss fraction – Average timing jitter

• Optional source description – Name, location, email address, phone number Copyright © 2002 Colin Perkins

• Mapping from media clock to external time-base – E.g. for lip synchronization

• Loosely coupled membership management

Protocol Components Payload Payload Payload Payload Format Format Format Format

• Provide the adaptation layer between a particular codec and RTP • Optimised for robustness to packet loss

Copyright © 2002 Colin Perkins

• Many payload formats exist, with more being developed: – H.261, H.263, M-JPEG, MPEG-2, MPEG-4, BT.656, SMPTE-292

Protocol Components

RTP Profile

• Define use of RTP in particular application scenarios – "Reasonable defaults" – Adaptation to unusual conditions • Single source multicast • Operation without back channel • Authenticated and secure operation Copyright © 2002 Colin Perkins

• Provides a namespace for payload formats

Combining the Pieces • A multimedia session comprises several RTP sessions – One for each media type

• Each RTP session: – Implements a particular RTP profile – Includes an RTP data flow • Transporting a single media type according to one or more payload formats – E.g. Audio switching between G.729 and Fax – E.g. Video using MPEG

– Includes an RTP control protocol flow • Providing reception quality feedback, user information, etc.

– Is defined by: • Source and destination IP addresses • A pair of UDP ports: one for RTP, one for RTCP

Copyright © 2002 Colin Perkins

RTP Data Transfer Protocol • The RTP Data Transfer Protocol delivers a single media stream from sender to one, or more, receivers – Few assumptions about the underlying transport – Usually runs over UDP/IP

• Typically implemented in an application or as a library – User level, not part of the kernel

Copyright © 2002 Colin Perkins

RTP provides: • Source identification • Media identification • Media transport – Padding, if necessary – Marking of significant events

• Sequencing • Timing recovery

Packet Format

V PX

CC

M

PT

Sequence Number

Timestamp Synchronization source (SSRC) identifier Contributing source (CSRC) identifiers

Payload data Padding

Copyright © 2002 Colin Perkins

Source Identification • Each packet carries a 32 bit synchronization source – Randomly chosen at startup, with collision detection

• Provides a transport layer independent identifier – Supports gateways – IPv4, IPv6, ATM

• Identifies a single timesynchronized media flow – Mapped to a persistent identifier using RTCP

Copyright © 2002 Colin Perkins

V

P X

CC

M

PT

Sequence Number Timestamp

Synchronization source (SSRC) identifier Contributing source (CSRC) identifiers

Payload data Padding

Source Identification • Each packet may include a list of contributing sources – Allows data from up to 16 sources to be identified – Each CSRC is the SSRC of a mixed participant

V

P X

CC

M

PT

Sequence Number Timestamp

Synchronization source (SSRC) identifier Contributing source (CSRC) identifiers

Payload data

• Allows RTP to support mixers and translators – Mixers combine several flows into one • E.g. Conferencing MCU

– Translators change the format of a flow, or gateway between different networks • Transcode to a lower bit-rate • Gateway between unicast and multicast

Copyright © 2002 Colin Perkins

Padding

Communication Models • Mixers and translators greatly expand the range of communications models available to RTP

RTP end point RTP Translator or mixer Multicast group, the network replicates data as necessary, with no translation or mixing.

Point-to-point communication via unicast

Four participants communicating via a multicast group

Replicated unicast: a group of three using an RTP translator/mixer to mediate communications. Copyright © 2002 Colin Perkins

Translated: multicast to unicast. Two participants communicating via a multicast group, with a third linked to the session by an RTP translator.

Media Identification • Each packet carries a 7 bit payload type field • Mapped to a payload format during session setup – Allows flexible signalling of codec type and parameters – Mapping can be static, if the profile allows

• Each flow carries only one type of media – Only audio, or only video

• The payload type allows the sender to switch between a set of payload formats – E.g. a flow carries only audio but may switch between fax and voice at any time

Copyright © 2002 Colin Perkins

V

P X

CC

M

PT

Sequence Number Timestamp

Synchronization source (SSRC) identifier Contributing source (CSRC) identifiers

Payload data Padding

Media Transport and Payload Formats • Packets contain a block of payload data, described by a payload format • Payload formats describe the mapping between codec output and RTP packets – Chosen so that each packet is independently decodable – Application level framing

• The payload data typically includes a payload header to ease parsing – E.g. The H.261 payload format copies some information from the GOB header so each set of macro-blocks can be decoded independently

Copyright © 2002 Colin Perkins

V

P X

CC

M

PT

Sequence Number Timestamp

Synchronization source (SSRC) identifier Contributing source (CSRC) identifiers

Payload data Padding

Media Transport: Marker • Each packet includes a bit to mark significant events – Start of talk spurt for audio – Last packet of frame for video

• A hint that special processing may be required

V

P X

CC

M

PT

Sequence Number Timestamp

Synchronization source (SSRC) identifier Contributing source (CSRC) identifiers

Payload data Padding

Copyright © 2002 Colin Perkins

Media Transport: Padding • Each packet may be padded beyond its natural size • Rarely used, but needed by some encryption algorithms – DES in CBC modes operates on 64 bit blocks

• The SRTP profile provides a better security solution

Copyright © 2002 Colin Perkins

V

P X

CC

M

PT

Sequence Number Timestamp

Synchronization source (SSRC) identifier Contributing source (CSRC) identifiers

Payload data Padding

Sequencing • Each packet contains a 16 bit sequence number – Random initial value – Increments monotonically with each packet sent – Wraps around to zero when the limit is reached

• Used to detect packet loss – Is not used to determine playout order

• Basic RTP does not provide error correction – The receiver is expected to conceal the error, and to continue processing – Extensions provide forward error correction and limited retransmission Copyright © 2002 Colin Perkins

V

P X

CC

M

PT

Sequence Number Timestamp

Synchronization source (SSRC) identifier Contributing source (CSRC) identifiers

Payload data Padding

Timing Recovery • Each packet contains a 32 bit timestamp • Indicates the sampling instant of the oldest payload data

V

P X

CC

M

PT

Sequence Number Timestamp

Synchronization source (SSRC) identifier Contributing source (CSRC) identifiers

– Determines playout order Payload data

• The clock rate is defined by the payload format: – Audio clock is sampling rate – Video clock is 90kHz, indicating the frame time – Mapping to codec time-base is also defined

• No requirements on stability or accuracy of clock – Implies receiver adaptation

Copyright © 2002 Colin Perkins

Padding

• Time code not carried directly, but mapping to wall clock time via RTCP sender reports

RTP Control Protocol (RTCP) • Each RTP data flow has an associated control flow • The control flow provides: – Time-base management – Quality of service feedback – Member identification and management

Copyright © 2002 Colin Perkins

Time-base Management • Timestamps map between the RTP timeline and NTP “wall-clock” time – If a common NTP clock is used for multiple streams, a receiver can synchronize them

• No explicit transport of SMPTE (or similar) time-codes – Can be derived from NTP timestamps • Accuracy limited by NTP resolution, unless external clock provided • RTSP provides a mapping function

• Also allows receivers to estimate data/packet rate and possibly clock skew

Copyright © 2002 Colin Perkins

Reception Quality Reporting • Quality of service feedback from each receiver: – – – – –

Loss fraction Cumulative number of packets lost Highest sequence number received Inter-arrival jitter Round-trip time

• Many uses: – Loss rate can be used to select amount of FEC to employ – Jitter gives estimate of playout buffer delay at receiver

Copyright © 2002 Colin Perkins

Membership Management • RTCP provides a canonical name, mapping SSRC to a persistent identifier

• Augments the membership management provided by the signalling protocol

– Used to associate streams for synchronisation

– Primarily using the explicit leave indication

• RTCP can optionally deliver source description data: – – – – –

Name Email address Phone number Location (extend with metadata)

• Provides loosely coupled presence information – Explicit leave message Copyright © 2002 Colin Perkins

RTCP reporting interval • RTCP is a low-rate reporting protocol – Not intended for uses that require instant feedback – Scalable to very large sessions • Statistical summary of group conditions

• Packets are sent periodically – The interval between packets is adjusted to limit RTCP to once per 5 seconds, or 5% of the data rate – Randomized to avoid synchronization

Copyright © 2002 Colin Perkins

RTP: Summary • Flexible and extensible media transfer protocol – Supports a range of codecs – Allows detection of network problems – Allows recovery of media timing

• Associated, low rate, reporting of reception quality, time-base, and presence information

Copyright © 2002 Colin Perkins

Talk Outline • Use of IP for real-time traffic • Protocols for real-time multimedia over IP – Outline of signalling protocols – Outline of media transport protocols

• RTP: Real-time Transport Protocol – RTP data transfer protocol – RTP control protocol

• Robustness – Playout and timing correction – Error correction – Congestion control

• Security • Conclusions

Copyright © 2002 Colin Perkins

Robustness • RTP operates over UDP/IP – Best effort delivery – Packets can be lost, delayed, reordered, duplicated, etc.

• Applications are responsible for correct playout – Timing recovery – Error concealment – Congestion control

Copyright © 2002 Colin Perkins

Timing Recovery • The network can seriously disrupt media timing • Receivers must include a jitter compensation buffer to reconstruct the media for playout

Original media stream

Sender

Router Constant inter-packet spacing

Internet

Network induces timing jitter into the media stream

Received media stream

Router

Receiver Variable inter-packet spacing

Copyright © 2002 Colin Perkins

Playout and Timing Correction

1st talk spurt

2nd talk spurt

Transmission Jitter affects inter-packet timing

Network Transit Reception Playout Buffer Network transit delay

Playout

Playout buffering delay added to compensate for jitter.

Copyright © 2002 Colin Perkins

Packet discarded due to late arrival

Playout and Timing Correction • RTP does not specify a standard playout buffer or timing reconstruction algorithm – Provides the necessary information; allowing product differentiation

• Compare with MPEG, where the buffer model is closely defined

Copyright © 2002 Colin Perkins

• Many trade-offs to consider: – latency versus quality – speed of reaction to change – buffering ability

• Typical design: – Streaming applications use large delay (10+ seconds) – Interactive applications try to keep delay low (tens of milliseconds)

Error Correction • Limited Retransmission – RTCP feedback profile

• Unequal error protection • Interleaving

• Forward Error Correction – Media specific – Media independent

Reliable

Unreliable

Not timely

Timely

TCP

UDP RTP

⇒ Add limited reliability to RTP Copyright © 2002 Colin Perkins

Retransmission in RTP • Why must retransmission be limited? – Timely versus reliable – We don't want to re-invent TCP

• How to implement retransmission? – RTCP provides a back channel – Modify the RTCP timing rules to allow early feedback • Keep the fundamental scaling rules to avoid potential for implosion

Copyright © 2002 Colin Perkins

RTCP Feedback Profile • RTCP reports sent as usual • Feedback can be sent early – Ignore 5 second rule – Borrow bandwidth from the next reporting interval – Delays next report – Send minimal report packet

Immediate FB mode

2

Report every relevant event immediately

Early RTCP mode

Report many of the events but not all

Send feedback + regular RTCP packets

Copyright © 2002 Colin Perkins

• Transport layer feedback – NACK, ACK sequence number

• Payload specific feedback – Reference picture selection – MPEG-4 NEWPRED

• Under development in IETF

Regular RTCP mode

Just regular RTCP packets

Group size

Forward Error Correction • Retransmission relies on feedback from receivers

• Alternative: – Sender adds redundant data to the media stream – Receivers use this to correct errors, without contacting sender

– Must request that lost packets are resent

• Works well in many cases • Two scenarios where it is inefficient:

• Forward Error Correction – Well known technique at the link layer – Also be applicable at the IP level and above

– Round-trip time is large – Many receivers, independent loss events

RTP

Copyright © 2002 Colin Perkins

RTP

RTP

FEC

RTP

RTP

RTP

FEC

RTP

Media Specific FEC • Some codecs may naturally be loss tolerant • Design payload format to take advantage of this

Copyright © 2002 Colin Perkins

Media Specific FEC • Some codecs may naturally be loss tolerant • Design payload format to take advantage of this – Audio/video redundancy – RFC 2198, AMR, H.263+

1

2

3

4

Original Stream

1

2

3

4

Redundant data added

1

2

4

Packet lost in transport

1

2

4

Reconstructed stream

Copyright © 2002 Colin Perkins

3

Media Independent FEC • Media specific FEC needs to be produced by the encoder

• Better if the FEC can be derived from pre-compressed media

– Part of the payload format – Either off-line or on-line

– Less load on sender – Perhaps less network efficient

⇒ Compression performed at time of transmission ⇒ To pick appropriate FEC ⇒ Undesirable ⇒ Too much load on sender to support many streams

Compress Source image

Packetize RTP

RTP

RTP

FEC Copyright © 2002 Colin Perkins

RTP

FEC

Parity FEC • Parity codes to protect serial communication well known • Can apply same technique to packet networks – Generate parity packet

Bit stream A

Parity code: A XOR B

Bit stream B Copyright © 2002 Colin Perkins

1

1

1

0

0

0

1

• Standard for parity FEC: – RFC 2733 – Flexible parity operation

• Standards for Reed-Solomon coding under development

1

1

1

0

0

0

1

0

0

1

1

0

1

0

1

0

1

0

1

1

1

1

0

1

0

1

1

1

0

0

1

1

0

1

0

Transmission loses B

Calculate parity to recover B = A XOR (A XOR B)

Unequal Error Protection • Not all data in the packets is equally important – Headers and codec state updates are vital – Media data is of variable importance • Data used for predication • Data used in a single frame

• Some links have high bit error rate – Causes packet corruption – Detected by UDP checksum – Packet discarded

• Seriously impacts wireless link performance – Cellular, especially

Partial Checksum at the UDP level

Copyright © 2002 Colin Perkins

Interleaving • Can interleave, to make bursts of loss appear as random loss

• Packet loss concealment and correction work best when loss is isolated – Single packet losses

– Adds considerable delay

• Packet loss on the Internet is bursty

1

2

3

4

5

6

7

8

9

1

5

9

13

2

6

10

14

3

1

5

9

13

2

6

10

14

1

2

4

5

6

Copyright © 2002 Colin Perkins

8

9

• Popular with streaming apps – Part of many audio payload formats

10 11 12 13 14

7

10

11 15

15 16

Original stream

4

8

12 16

Interleaved stream

4

8

12 16

Packet loss

12 13 14

16

Reconstructed stream

Error Correction: Summary • Extensive, and ongoing, research and standards work developing error correction for RTP – – – – –

Retransmission Media/codec specific FEC Media independent FEC Partial checksum Interleaving

⇒ Network doesn't have to be perfect • If it's okay for data traffic, it's probably okay for video • Unless you have very strict requirements

⇒ Acceptable to overprovision for QoS • No real need for RSVP or differentiated services

⇒ Well designed applications can tolerate significant loss • Often, 5% loss acceptable

Copyright © 2002 Colin Perkins

Congestion Control But…

… the preceding assumed traffic is well behaved … assumed flows respond to congestion in the network … or, QoS and flow admission control employed

Copyright © 2002 Colin Perkins

Congestion Control

– No admission control – The network accepts all packets and tries to deliver them.

• However, no guarantee of delivery provided – Excess packets discarded if links become congested.

• The transport protocol must detect loss, and reduce its rate to allow the congestion to clear – TCP does this automatically – RTP does not Copyright © 2002 Colin Perkins

Normal operation

Congestion Collapse

Packets delivered

• An IP network provides a best-effort packet-switched service.

Packets sent

Congestion Control

– Possible rate changes depend on the codec – Complex feedback loop between codec and network

• For RTP, implies senders should observe receiver feedback – If loss fraction is non-zero, consider sending less – As loss decreases, consider sending faster

Copyright © 2002 Colin Perkins

Normal operation

Congestion Collapse

Packets delivered

• Adaptation must be done by the application

Packets sent

TCP Friendly Rate Control • Possible to predict the longterm average throughput of TCP s

T= R

3p 2p + 3 p(1 + 32 p 2 ) ⋅ Trto 8 3

• Derive throughput in terms of observed loss rate, RTT and packet size – Measurable qualities in RTP

• Adapt sending rate to match – Driven by reception of RTCP – Drives codec operation

Copyright © 2002 Colin Perkins

• The "best current practice" congestion control scheme for multimedia flows • With appropriate parameters: – Fair to TCP on average – Slowly changing rate

Limitations of Congestion Control • TCP friendly algorithms are new, and evolving – Very limited deployment – Not clear that they have reached their final form

• Interactions between codec and network are not well defined – Unclear how slow response, or limited adaptability, will impact fairness

Copyright © 2002 Colin Perkins

• Human factors aspects play a key role – Congestion control implies variable quality – Subjectively very annoying, unless the rate of change is slow – Can have a significant impact on congestion control

Talk Outline • Use of IP for real-time traffic • Protocols for real-time multimedia over IP – Outline of signalling protocols – Outline of media transport protocols

• RTP: Real-time Transport Protocol – RTP data transfer protocol – RTP control protocol

• Robustness – Playout and timing correction – Error correction – Congestion control

• Security • Conclusions

Copyright © 2002 Colin Perkins

Security Several aspects to multimedia security • Confidentiality of the media RTP can help here • Authentication of the sender • Watermarking • Storage

Copyright © 2002 Colin Perkins

Security in RTP • Basic RTP provides limited security – Packets may be encrypted • DES is specified; algorithm may be negotiated during session setup

– Does not support sender authentication

• A secure RTP profile is under development – Encryption for confidentiality • Encrypts only the payload data, not the headers • AES in counter or F8 mode • Robust to bit errors, allows header compression, suitable for cellular wireless

– Sender authentication • Adds a trailer to each packet, containing authentication code • HMAC-SHA1

Copyright © 2002 Colin Perkins

Watermarking, Storage and DRM • RTP is concerned only with the transmission of media • Does not consider: – The contents of the video image – How the image is captured, generated and stored

Copyright © 2002 Colin Perkins

• Implications: – Receivers may store the payload • RTP cannot influence the process • The copy may not be perfect

– Watermarks or other embedded data can be inserted at source • But must be robust to packet loss

Talk Outline • Use of IP for real-time traffic • Protocols for real-time multimedia over IP – Outline of signalling protocols – Outline of media transport protocols

• RTP: Real-time Transport Protocol – RTP data transfer protocol – RTP control protocol

• Robustness – Playout and timing correction – Error correction – Congestion control

• Security • Conclusions

Copyright © 2002 Colin Perkins

Conclusions • IP provides a non-optimal service for video transport – Careful network engineering can solve many problems

• Multimedia protocol framework comprises: – Signalling: H.323, SIP, RTSP, SAP – Media Transport: RTP + codecs

• RTP provides: – Robust, flexible and extensible media transport – Range of error correction schemes – Range of security solutions

• Limitations: – Congestion control

Copyright © 2002 Colin Perkins

For More Information IETF Audio/Video Transport Working Group http://www.ietf.org/html.charters/avt-charter.html

Colin Perkins http://www.east.isi.edu/~csp/

Copyright © 2002 Colin Perkins

Related Documents

Rtp Tutorial
May 2020 7
Rtp
May 2020 7
Rtp
May 2020 9
Rtp
May 2020 7
Comunicado Rtp
November 2019 14
Rtp Slides
May 2020 8