Download Multimedia

Chapter 6
Multimedia Networking
A note on the use of these ppt slides:
We’re making these slides freely available to all (faculty, students, readers).
They’re in powerpoint form so you can add, modify, and delete slides
(including this one) and slide content to suit your needs. They obviously
represent a lot of work on our part. In return for use, we only ask the
following:
 If you use these slides (e.g., in a class) in substantially unaltered form,
that you mention their source (after all, we’d like people to use our book!)
 If you post any slides in substantially unaltered form on a www site, that
you note that they are adapted from (or perhaps identical to) our slides, and
note our copyright of this material.
Computer Networking: A Top
Down Approach Featuring the
Internet,
2nd edition.
Jim Kurose, Keith Ross
Addison-Wesley, July 2002.
Thanks and enjoy! JFK / KWR
All material copyright 1996-2002
J.F Kurose and K.W. Ross, All Rights Reserved
1
Multimedia, Quality of Service: What is it?
Multimedia applications:
network audio and video
(“continuous media”)
QoS
network provides
application with level of
performance needed for
application to function.
2
Chapter 6: Goals
Principles
 Classify multimedia applications
 Identify the network services the apps need
 Making the best of best effort service
 Mechanisms for providing QoS
Protocols and Architectures
 Specific protocols for best-effort
 Architectures for QoS
3
Chapter 6 outline
 6.1 Multimedia Networking Applications
 6.2 Streaming stored audio and video

RTSP
 6.3 Real-time Multimedia: Internet Phone
Case Study
 6.4 Protocols for Real-Time Interactive
Applications

SIP
4
MM Networking Applications
Classes of MM applications:
1) Streaming stored audio
and video
2) Streaming live audio and
video
3) Real-time interactive
audio and video
Jitter is the variability
of packet delays within
the same packet stream
Fundamental
characteristics:
 Typically delay sensitive


end-to-end delay
delay jitter
 But loss tolerant:
infrequent losses cause
minor glitches
 Antithesis of data,
which are loss intolerant
but delay tolerant.
5
Streaming Stored Multimedia
Streaming:
 media stored at source
 transmitted to client
 streaming: client playout begins
before all data has arrived
 timing constraint for still-to-be
transmitted data: in time for playout
6
Streaming Stored Multimedia:
What is it?
1. video
recorded
2. video
sent
network
delay
3. video received,
played out at client
time
streaming: at this time, client
playing out early part of video,
while server still sending later
part of video
7
Delay Jitter
variable
network
delay
(jitter)
client
reception
constant bit
rate playout
at client
buffered
data
constant bit
rate
transmission
client playout
delay
time
8
Jitter: Definitions
 Input: t0, …, tn
 Delay Jitter:
 Delay jitter J
 For every k: |t0 + kX - tk |  J
 X = (tn - t0) / n
 Rate Jitter
 Rate Jitter A
 Ik = tk - tk-1
 For every k and j: |Ij - Ik|  A
 Jitter and buffering
 delay versus jitter
9
Streaming Stored Multimedia: Interactivity
 VCR-like functionality: client can
pause, rewind, FF, push slider bar
 10 sec initial delay OK
 1-2 sec until command effect OK
 RTSP often used (more later)
 timing constraint for still-to-be
transmitted data: in time for playout
10
Streaming Live Multimedia
Examples:
 Internet radio talk show
 Live sporting event
Streaming
 playback buffer
 playback can lag tens of seconds after
transmission
 still have timing constraint
Interactivity
 fast forward impossible
 rewind, pause possible!
11
Interactive, Real-Time Multimedia
 applications: IP telephony,
video conference, distributed
interactive worlds
 end-end delay requirements:
 audio: < 150 msec good, < 400 msec OK
• includes application-level (packetization) and network
delays
• higher delays noticeable, impair interactivity
 session initialization

how does callee advertise its IP address, port
number, encoding algorithms?
12
Internet Multicasting
13
Multicast: Connecting multiple users
 Unicast: connecting
two users
 Multicast: connecting
multiple users



Conference call
Event broadcast
Chat room
 Supporting multicast
 Multiple unicast
• Inefficient

Build a multicast tree
• Assume cost on links
14
The Multicast Problem
?
 GIVEN: Network
 GIVEN: User Requests (and quality of service requirements)
 ESTABLISH: Efficient distribution (trees)
15
What is The Need for Multicasting?
 Use bandwidth efficiently


3 times the
traffic
Replace many unicast connections
with one multicast connection
Duplicate packets only when
necessary
 Theoretical model


Links have cost
Find the minimum cost multicast
tree
16
Steiner Tree Problem
 Input:
Given a graph G(V,E)
 Costs on edges: Cost(e)
 Set of terminal nodes S

 Output:
 Tree T,
 Includes all the nodes in S
 Optimization: Minimum cost
 Complexity: NP-complete
17
Steiner Tree: MST Heuristic
 MST Heuristic :

define GS = complete graph on terminals
• edge cost = shortest path cost

find TMST = MST of GS

replace each edge of TMST with the path in G

output TMST
18
MST Approximation ratio
 MST is at least 2-approximation
 TMST = output of MST algo (on GS)
 T* = optimal Steiner Tree in G
 Cost(TMST,G) ≤
Cost(TMST,GS)
 Cost(T*,GS) ≤ 2 Cost(T*,G)
 Cost(TMST,GS) ≤
Cost(T*,GS)
 Conclusion: Cost(TMST,G) ≤ 2C ost(T*,GS)
 MST is at most 2-approximation
 Edges cost black = 2 red =1
 MST cost = 2k-2, OPT cost = k
 Ratio 2-(1/k)
1
2
3
4
k
5
19
Multicast: Online setting
 Online Steiner problem
 Terminal arrive one at a time
 Need to allocate a tree to current terminals
 Online Greedy :
 At time t:
•
•
•
•
Current tree Tt-1
arriving terminal vt
Find shortest path Pv from vt to Tt-1
Let Tt be the union of Pv and Tt-1
20
Performance of Online Greedy
 Sort terminal by their cost
cost(u1) ≥ cost(u2) ≥ … ≥ cost(un)
 CLAIM: cost(uk) ≤ 4 OPT/k (need to prove it …)
 Assuming the claim:
cost(T) = cost(u1) + … + cost(un)
≤ 4 OPT/1 + … + 4OPT/n
≤ 4 OPT(1 + ½ + ⅓+¼+⅛… + 1/n)
≈ 4 ln(n) OPT
21
Proving the claim
 Consider Uk= {u1 , … , uk}
 Suppose cost(uk) > 4 OPT /k,
 Then cost(ui) > 4 OPT/k, for i<k (they where sorted)
 Every ui and uj are distance at least 4 OPT/k
 Let GS be the graph on Uk
 with cost = shortest distance
 Let TU be the MST of GS
 Cost (Tu) > 4(k-1) OPT/k ≥ 2 OPT
 Contradiction to the MST heuristic
• which is at 2-approximation!
 QED
22
Chapter 6 outline
 6.1 Multimedia Networking Applications
 6.2 Streaming stored audio and video

RTSP
 6.3 Real-time Multimedia: Internet Phone
Case Study
 6.4 Protocols for Real-Time Interactive
Applications

SIP
23
Streaming Stored Multimedia
Application-level streaming
techniques for making the
best out of best effort
service:
 client side buffering
 use of UDP versus TCP
 multiple encodings of
multimedia
Media Player
 jitter removal
 decompression
 error concealment
 graphical user interface
w/ controls for
interactivity
24
Streaming Multimedia: Client Buffering
variable
network
delay
client video
reception
constant bit
rate video
playout at client
buffered
video
constant bit
rate video
transmission
client playout
delay
time
 Client-side buffering, playout delay compensate
for network-added delay, delay jitter
28
Streaming Multimedia: Client Buffering
constant
drain
rate, d
variable fill
rate, x(t)
buffered
video
 Client-side buffering, playout delay compensate
for network-added delay, delay jitter
29
Streaming Multimedia: UDP or TCP?
UDP
 server sends at rate appropriate for client (oblivious to
network congestion !)
 often send rate = encoding rate = constant rate
 then, fill rate = constant rate - packet loss
 short playout delay (2-5 seconds) to compensate for network
delay jitter
 error recover: time permitting
TCP
 send at maximum possible rate under TCP
 fill rate fluctuates due to TCP congestion control
 larger playout delay: smooth TCP delivery rate
 HTTP/TCP passes more easily through firewalls
30
Streaming Multimedia: client rate(s)
1.5 Mbps encoding
28.8 Kbps encoding
Q: how to handle different client receive rate
capabilities?
 0.5+ Kbps ADSL
 100Mbps Ethernet
A: server stores, transmits multiple copies
of video, encoded at different rates
31
User Control of Streaming Media: RTSP
HTTP
 Does not target multimedia
content
 No commands for fast
forward, etc.
RTSP: RFC 2326
 Client-server application
layer protocol.
 For user to control display:
rewind, fast forward,
pause, resume,
repositioning, etc…
What it doesn’t do:
 does not define how
audio/video is encapsulated
for streaming over network
 does not restrict how
streamed media is
transported; it can be
transported over UDP or
TCP
 does not specify how the
media player buffers
audio/video
32
Chapter 6 outline
 6.1 Multimedia Networking Applications
 6.2 Streaming stored audio and video

RTSP
 6.3 Real-time, Interactive Multimedia:
Internet Phone Case Study
 6.4 Protocols for Real-Time Interactive
Applications

SIP
33
Real-time interactive applications
 PC-2-PC phone
 instant messaging
services are providing
this
 PC-2-phone
Going to now look at
a PC-2-PC Internet
phone example in
detail
 videoconference with
Webcams
34
Interactive Multimedia: Internet Phone
Introduce Internet Phone by way of an example
 speaker’s audio: alternating talk spurts, silent
periods.

64 kbps during talk spurt
 pkts generated only during talk spurts

20 msec chunks at 8 Kbytes/sec: 160 bytes data
 application-layer header added to each chunk.
 Chunk+header encapsulated into UDP segment.
 application sends UDP segment into socket every
20 msec during talkspurt.
35
Basic IP Telephony
Audio Packet Transfer
…
 Digitization (e.g., sampling at 8kHz, 16 bits per
sample, i.e, 128 kb/s or 320 bytes per 20 ms)
 Real-time compression/encoding (e.g., G.729A at 8
kb/s)
 Transport to remote IP address and port number
over UDP (Why not TCP?)
 Processing on receiver side is the reverse
36
Basic IP Telephony:
Sampling, Quantization, Encoding
+127
10101111…01101101
+0
-127
Sample at twice the
highest voice frequency
2 x 4000=8000
Round off samples to
one of 256 levels
(introduces noise)
37
Internet Phone: Packet Loss and Delay
 network loss: IP datagram lost due to network
congestion (router buffer overflow)
 delay loss: IP datagram arrives too late for
playout at receiver


delays: processing, queueing in network; end-system
(sender, receiver) delays
typical maximum tolerable delay: 400 ms
 loss tolerance: depending on voice encoding, losses
concealed, packet loss rates between 1% and 10%
can be tolerated.
38
Basic IP Telephony: Problems with UDP
Sender
1
2
3
4
5
6
7
(a)
(b)
1
2
Receiver
3
5
7
timeline
Unreliable UDP
a) Packet loss
b) Out-of-order (very rarely)
c) Jitter (delay variation)
39
6
Basic IP Telephony: Receive buffer
 Sequence number: to detect packet loss; Just
ignore the loss!
 Receive buffer: to absorb jitter
Sender
1
2
3
1
4
5
2
6
3
8
7
5
9
7
0
1
8
6
2
9
3
0
4
2
3
2
3
Receiver
1
2
1
7
2
1
5
2
3
5
7
8
9
0
6
7
8
9
0
40
2
4
Basic IP Telephony:
Timestamp Vs sequence number
•
•
Sender
t1
t2
t3
1
2
3
1
2
t4
t5
Silence …
3
t6
Silence suppression
Variable length packets
t7
4
5
t8
t9
6
7
4
5
6
7
Receiver
Playout time vs packet loss detection
41
Basic IP Telephony :
Real-time Transport Protocol - RTP
IP header
V PX
UDP header
CC
M Payload type
Sequence number
Timestamp (proportional to sampling time)
RTP Header
Source identifier (SSrc)
msg
Encoded
Audio
Optional contributors’ list (CSrc)
sendto(…, msg, …)
recvfrom(…, msg, …)
42
Recovery from packet loss (1)
forward error correction
(FEC): simple scheme
 for every group of n
chunks create a
redundant chunk by
exclusive OR-ing the n
original chunks
 send out n+1 chunks,
increasing the bandwidth
by factor 1/n.
 can reconstruct the
original n chunks if there
is at most one lost chunk
from the n+1 chunks
 Playout delay needs to
be fixed to the time to
receive all n+1 packets
 Tradeoff:
 increase n, less
bandwidth waste
 increase n, longer
playout delay
 increase n, higher
probability that 2 or
more chunks will be
lost
48
Recovery from packet loss (2)
2nd FEC scheme
• “piggyback lower
quality stream”
• send lower resolution
audio stream as the
redundant information
• for example, nominal
stream PCM at 64 kbps
and redundant stream
GSM at 13 kbps.
• Whenever there is non-consecutive loss, the
receiver can conceal the loss.
• Can also append (n-1)st and (n-2)nd low-bit rate
chunk
49
Recovery from packet loss (3)
Interleaving
 chunks are broken
up into smaller units
 for example, 4 -5 units per
chunk
 Packet contains small units
from different chunks
 if packet is lost, still have
most of every chunk
 has no redundancy overhead
 but adds to playout delay
50
Summary: Internet Multimedia: bag of tricks
 use UDP to avoid TCP congestion control (delays)
for time-sensitive traffic
 client-side adaptive playout delay: to compensate
for delay
 server side matches stream bandwidth to available
client-to-server path bandwidth


chose among pre-encoded stream rates
dynamic server encoding rate
 error recovery (on top of UDP)
 FEC, interleaving
 retransmissions, time permitting
 conceal errors: repeat nearby data
51
Chapter 6 outline
 6.1 Multimedia Networking Applications
 6.2 Streaming stored audio and video

RTSP
 6.3 Real-time, Interactive Multimedia:
Internet Phone Case Study
 6.4 Protocols for Real-Time Interactive
Applications

SIP
52
SIP
 Session Initiation Protocol
 Comes from IETF
SIP long-term vision
 All telephone calls and video conference calls take
place over the Internet
 People are identified by names or e-mail
addresses, rather than by phone numbers.
 You can reach the callee, no matter where the
callee roams, no matter what IP device the callee
is currently using.
53
SIP Services
 Setting up a call
 Provides mechanisms for
caller to let callee know
she wants to establish a
call
 Provides mechanisms so
that caller and callee can
agree on media type and
encoding.
 Provides mechanisms to
end call.
 Determine current IP
address of callee.

Maps mnemonic
identifier to current IP
address
 Call management
 Add new media streams
during call
 Change encoding during
call
 Invite others
 Transfer and hold calls
54
Setting up a call to a known IP address
Bob
Alice
167.180.112.24
INVITE bob
@193.64.2
10.89
c=IN IP4 16
7.180.112.2
4
m=audio 38
060 RTP/A
VP 0
193.64.210.89
port 5060
port 5060
Bob's
terminal rings
200 OK
.210.89
c=IN IP4 193.64
RTP/AVP 3
3
75
m=audio 48
ACK
port 5060
• Alice’s SIP invite
message indicates her
port number & IP address.
Indicates encoding that
Alice prefers to receive
(PCM ulaw)
• Bob’s 200 OK message
indicates his port number,
IP address & preferred
encoding (GSM)
m Law audio
port 38060
GSM
time
port 48753
time
• SIP messages can be
sent over TCP or UDP;
here sent over RTP/UDP.
•Default SIP port number
is 5060.
55
Setting up a call (more)
 Codec negotiation:



Suppose Bob doesn’t have
PCM ulaw encoder.
Bob will instead reply with
606 Not Acceptable
Reply and list encoders he
can use.
Alice can then send a new
INVITE message,
advertising an appropriate
encoder.
 Rejecting the call
Bob can reject with
replies “busy,” “gone,”
“payment required,”
“forbidden”.
 Media can be sent over RTP
or some other protocol.

56
Example of SIP message
INVITE sip:bob@domain.com SIP/2.0
Via: SIP/2.0/UDP 167.180.112.24
From: sip:alice@hereway.com
To: sip:bob@domain.com
Call-ID: a2e3a@pigeon.hereway.com
Content-Type: application/sdp
Content-Length: 885
c=IN IP4 167.180.112.24
m=audio 38060 RTP/AVP 0
Notes:
 HTTP message syntax
 sdp = session description protocol
 Call-ID is unique for every call.
• Here we don’t know
Bob’s IP address.
Intermediate SIP
servers will be
necessary.
• Alice sends and
receives SIP messages
using the SIP default
port number 5060.
• Alice specifies in Via:
header that SIP client
sends and receives
SIP messages over UDP
57
Name translation and user location
 Caller wants to call
callee, but only has
callee’s name or e-mail
address.
 Need to get IP
address of callee’s
current host:



user moves around
DHCP protocol
user has different IP
devices (PC, PDA, car
device)
 Result can be based on:
 time of day (work, home)
 caller (don’t want boss to
call you at home)
 status of callee (calls sent
to voicemail when callee is
already talking to
someone)
Service provided by SIP
servers:
 SIP registrar server
 SIP proxy server
58
SIP Registrar
 When Bob starts SIP client, client sends SIP
REGISTER message to Bob’s registrar server
(similar function needed by Instant Messaging)
Register Message:
REGISTER sip:domain.com SIP/2.0
Via: SIP/2.0/UDP 193.64.210.89
From: sip:bob@domain.com
To: sip:bob@domain.com
Expires: 3600
59
SIP Proxy
 Alice send’s invite message to her proxy server
 contains address sip:bob@domain.com
 Proxy responsible for routing SIP messages to
callee

possibly through multiple proxies.
 Callee sends response back through the same set
of proxies.
 Proxy returns SIP response message to Alice

contains Bob’s IP address
 Note: proxy is analogous to local DNS server
60
Example
Caller jim@umass.edu
with places a
call to keith@upenn.edu
SIP registrar
upenn.edu
SIP
registrar
eurecom.fr
2
(1) Jim sends INVITE
message to umass SIP
proxy. (2) Proxy forwards
request to upenn
registrar server.
(3) upenn server returns
redirect response,
indicating that it should
try keith@eurecom.fr
SIP proxy
umass.edu
1
3
4
5
7
8
6
9
SIP client
217.123.56.89
SIP client
197.87.54.21
(4) umass proxy sends INVITE to eurecom registrar. (5) eurecom
registrar forwards INVITE to 197.87.54.21, which is running keith’s SIP
client. (6-8) SIP response sent back (9) media sent directly
between clients.
Note: also a SIP ack message, which is not shown.
61
Comparison with H.323
 H.323 is another signaling
protocol for real-time,
interactive
 H.323 is a complete,
vertically integrated suite
of protocols for multimedia
conferencing: signaling,
registration, admission
control, transport and
codecs.
 SIP is a single component.
Works with RTP, but does
not mandate it. Can be
combined with other
protocols and services.
 H.323 comes from the ITU
(telephony).
 SIP comes from IETF:
Borrows much of its
concepts from HTTP. SIP
has a Web flavor, whereas
H.323 has a telephony
flavor.
 SIP uses the KISS
principle: Keep it simple
stupid.
62