Download Introduction to the PPNCG - University of Manchester

Introduction to the PPNCG
Networking for the PPARC Community
Introduction to the PPNCG
UK Network Topologies
External Connectivity – Europe & US
Astronomy & Astrophysics Sites
Grid Network Monitoring
PingER – World wide Monitoring
QoS – a micro Introduction
Astronomy Sysman Meeting 29/30 April 02
R. Hughes-Jones Manchester
1
Introduction to the PPNCG
Membership includes HEP and Astronomy users
Dave Terrett , Bob Bentley, Ralph Spencer
Remit
Ensure the community has the required networking facilities
Monitor end-to-end performance
Investigate new network applications / technologies
Provide advice on kit / facilities
Active Network Monitoring
PPNCG ping, ftp and traceping
ICFA monitoring
Report problems to UKERNA
Regular meetings with UKERNA invited
Recognised as a subject group in JNUG and JISC
Links to several Grid Projects
Astronomy Sysman Meeting 29/30 April 02
R. Hughes-Jones Manchester
2
SuperJANET4: Backbone and
Access links
Worldcom supplied the transmission
UKERNA layer the IP service
Core PoP IP router at Worldcom
Backbone Access Router at MANs
Access Links:
Large MAN 2.5 Gbit -> 10-20 Gbit
Medium MAN 622 Mbit -> 2.5 Gbit
4 node DWDM development net
Deployment Status:
Backbone Oct 00 Routers Nov 00
All sites Mar 01
Proved to be Stable
Constant growth of traffic
Upgrade Backbone to 10Gbit Jun 02
Astronomy Sysman Meeting 29/30 April 02
R. Hughes-Jones Manchester
3
SuperJANET4: ping rtt Core routers
Jun 01
Astronomy Sysman Meeting 29/30 April 02
R. Hughes-Jones Manchester
4
SuperJANET4: ping rtt Site nodes
Lancaster
Glasgow
MAN / LAN Issues
Bristol
Cambridge
Astronomy Sysman Meeting 29/30 April 02
R. Hughes-Jones Manchester
5
London MAN Upgrade
UDP Packet loss
%
UDPmon Tests
Manchester – London
MAN was 155 Mbit ATM
UDP Throughput
Mbit/s
1st Oct
Time interval in Weeks
Astronomy Sysman Meeting 29/30 April 02
R. Hughes-Jones Manchester
Richard HJ
6
Previous External Connectivity
Europe:
TEN-155
155Mbit Access link
US:
6 * 155 Mbit links
Peer in Hudson St.
622 Mbit to Esnet
622 Mbit to Abilene.
Astronomy Sysman Meeting 29/30 April 02
R. Hughes-Jones Manchester
7
Europe – Access links (1)
ICFAMON Plot from RAL to CERN for 19th Oct to 1st Nov 2001
UK Access link 155 Mbit ATM
Sustained rate 130 Mbit
Contract to end of Nov 01
Bad news for users !
Astronomy Sysman Meeting 29/30 April 02
R. Hughes-Jones Manchester
8
Europe – Access links (2)
Traceping Oxford to CERN for 31st October 2001
loss around
ten155-gw.ja.net router
Astronomy Sysman Meeting 29/30 April 02
R. Hughes-Jones Manchester
J. Macallister
9
New External Connectivity
6 * 155 Mbit links
2.5Gbit line installed
IP commodity peer in London
Research traffic over 2.5G bit
Peer in Hudson St.
622 Mbit to Esnet
622 Mbit to Abilene.
Astronomy Sysman Meeting 29/30 April 02
R. Hughes-Jones Manchester
10
Connectivity to Europe : Geant
Start mid November 2001
UKERNA switched off TEN-155 3 Dec 2001
Astronomy Sysman Meeting 29/30 April 02
R. Hughes-Jones Manchester
11
Connectivity to Europe
ICFAMON Plot from DL to CERN for 18th Feb to 3rd Mar 2002
UK Dante Access link 2.5 Gbit POS
Remember 19th Oct to 1st Nov 2001
Access link over loaded
Astronomy Sysman Meeting 29/30 April 02
R. Hughes-Jones Manchester
12
Monitoring: US Traffic
UKERNA Traffic data Kbit/s. Blue Traffic from US; Maroon Traffic to US
7 day periods 1 hour averages
14 Jan 2002 (800Mbit/s)
peak 86% of total 930 Mbit
17 Jan 2002
Peering altered 22 Jan
Weekend-Before
Weekday-After
Weekday-Before
22 Jan 2002
Weed day peak 175 Mbit/s
Astronomy Sysman Meeting 29/30 April 02
R. Hughes-Jones Manchester
13
Monitoring: US Traffic
UKERNA Traffic data Kbit/s. Blue Traffic from US; Maroon Traffic to US
7 Dec 2001 (900kbit/s)
29 Jan 2002 (175kbit/s)
peak is 88% of total BW 930 Mbit
10 minute averages
10 minute averages
Last 7 days 1 hour averages
Astronomy Sysman Meeting 29/30 April 02
R. Hughes-Jones Manchester
14
Astronomy & Astrophysics Sites
Astronomy Sysman Meeting 29/30 April 02
R. Hughes-Jones Manchester
15
Connectivity to Australia
ICFAMON Plot from DL to Anglo-Australian Observatory
for 11th Apr to 24th Apr 2002
Packet loss reasonable
rtt improves ~420 ms to ~300 ms
Variations ~100ms
Astronomy Sysman Meeting 29/30 April 02
R. Hughes-Jones Manchester
16
Connectivity to US
ICFAMON Plots for 11th Apr to 24th Apr 2002
DL to NOAO, Arizona
DL to Goddard GSFC NASA
Astronomy Sysman Meeting 29/30 April 02
R. Hughes-Jones Manchester
17
Connectivity to Hawaii
ICFAMON Plot from DL to The Joint Astronomy Centre
for 11th Apr to 24th Apr 2002
Packet loss good
rtt ~210 ms
Variations – queuing
traceroute:
Cross SuperJANET4 to NY OK
Cross Abilene to Seattle OK
Enters uhnet
Stops after 2-3 routers
No connectivity to La Palma
traceroute ends in iac.es network
Tenerife ?
Astronomy Sysman Meeting 29/30 April 02
R. Hughes-Jones Manchester
18
Grid Network Monitoring
 Several tools in test – plugged into a coherent structure:
 PingER, RIPE one way times, iperf, UDPmon, rTPL, GridFTP, and
NWS prediction engine
 continuous tests for last few months to selected sites:

DL Man RL UCL CERN Lyon Bologna SARA NBI SLAC …
 The aims of monitoring for the Grid:



to inform Grid applications, via the middleware, of the current status of the
network – input for resource broker and scheduling
to identify fault conditions in the operation of the Grid
to understand the instantaneous, day-to-day, and month-by-month
behaviour of the network – provide advice on configuration etc.
 Network information published in LDAP schema
 Will be used by UK GridPP and e-science centres
 AstroGrid ?
Astronomy Sysman Meeting 29/30 April 02
R. Hughes-Jones Manchester
19
Network Monitoring Architecture
LDAP
Schema
Grid Apps
GridFTP
PingER
(RIPE TTB)
IperfER
UDPmon
rTPL
NWS
etc
Local Network
Monitoring
Store & Analysis
of Data (Access)
Backend LDAP script to fetch metrics
Monitor process to push metrics
local
LDAP
Server
Grid Application access via
LDAP Schema to
- monitoring metrics;
- location of monitoring data.
Access to current and historic data
and metrics via the Web, i.e. WP7
NM Pages, access to metric forecasts
Robin Tasker
Astronomy Sysman Meeting 29/30 April 02
R. Hughes-Jones Manchester
20
Network Monitoring Components
Clients WEB Display
Predictions
LDAP
Web I/f
LDAP
Table
plot
Grid Broker
LDAP
LDAP
Table
raw
raw
plot
Analysis
LDAP
Table
raw
plot
raw
raw
Scheduler
Cron
script
control
Cron
script
Cron
script
control
Tool
Ping
Netmon
UDPmon
Astronomy Sysman Meeting 29/30 April 02
R. Hughes-Jones Manchester
iPerf
Ripe
21
Ping & UDP throughput MAN-RAL
From 20 Oct 01
PingER rtt (ms)
dl – RAL
1000 byte packet
Forecast
UDPmon Zero packet loss!
UDPmon throughput Mbit/s
man – RAL
300 * 1400 byte frames
Astronomy Sysman Meeting 29/30 April 02
R. Hughes-Jones Manchester
22
Ping & UDP throughput MAN-CERN
From 20 Oct 01
PingER rtt (ms)
dl – cern
1000 byte packet
Forecast
UDPmon throughput Mbit/s
man – cern
300 * 1400 byte frames
Astronomy Sysman Meeting 29/30 April 02
R. Hughes-Jones Manchester
23
iperf TCP & UDP throughput MAN-SARA
From 20 Oct 01
Iperf TCP throughput Mbit/s
ucl – sara
262144 byte buffer
Forecast
UDPmon throughput Mbit/s
man – sara
300 * 1400 byte frames
Astronomy Sysman Meeting 29/30 April 02
R. Hughes-Jones Manchester
24
iperf & Pinger UK-Bologna
From 20 Oct 01
Iperf throughput
ucl – Bologna
262144 byte buffer
Forecast in green
PingER rtt (ms)
dl – Bologna
1000 byte packet
Forecast
Astronomy Sysman Meeting 29/30 April 02
R. Hughes-Jones Manchester
25
UDPmon Loss
iperf throughput UCL-SARA
From 1 Nov 01 – Geant Operational
Throughput
Mbit/s
MAN – SARA
Iperf Throughput
Mbit/s
UCL – SARA
262144 byte
buffer
Geant Enabled
Astronomy Sysman Meeting 29/30 April 02
R. Hughes-Jones Manchester
Routing Stable
26
PingER deployment
Les Cottrell
 Measurements from
 34 monitors in 14 countries





Over 600 remote hosts
Over 72 countries
Over 3300 monitor-remote site pairs
Measurements go back to Jan-95
Reports on RTT, loss, reachability, jitter, reorders, duplicates …
 Countries monitored


Contain 78% of world population
99% of online users of Internet
 Lightweight (100bps/host pair)

Very useful for inter-regional and poor links, need more intensive for high
performance & Grid sites
Astronomy Sysman Meeting 29/30 April 02
R. Hughes-Jones Manchester
27
Losses: World by region, Jan ‘02
 Packet loss <1%=good, <2.5%=acceptable, < 5%=poor, >5%=bad
 Russia,
S America bad
 Balkans,
M East,
Africa,
S Asia,
Caucasus poor
Monitored
Region \
Monitor
BR CA DK DE HU IT JP
RU CH
(1) (2) (1) (1) (1) (3) (2)
(2) (1)
Country
COM
0.2
0.3
Canada
1.8 1.6 0.3 0.5 9.0 0.3 1.4 21.7 0.7
US
0.4 2.6 0.2 0.3 8.0 0.1 1.4 13.8 0.3
C America
Australasia
E Asia
1.2 3.5 1.0 1.1 9.0 0.9 2.0 5.2 1.5
Europe
0.4 5.6 0.3 0.5 5.4 0.4 1.3 15.5 1.1
NET
1.7 6.2 1.0 1.3 8.0 1.6 3.6 21.9 0.7
FSU4.5
0.5 9.8 0.5 1.6 11.2 4.3
Balkans
Mid East
4.6 1.4 3.0 8.5 2.8 3.2 11.8 2.0
Africa
5.8
1.5 12.0 1.2 4.2 11.9 2.0
Baltics
5.3 0.8 2.3 7.7 2.2 3.5 10.8 4.8
S Asia
1.6 7.3 0.1 3.1 9.2 3.0 3.9 17.9 1.5
Caucasus
S America 24.1 11.3 0.6 0.9 6.7 12.9 7.7 23.0 9.3
Russia
35.9 24.1 22.2 13.4 23.8 21.7 13.6 0.7 8.7
Astronomy Sysman Meeting 29/30 April 02
Avg
7.5 6.9 2.8 2.4 9.8 3.7 3.9 13.8 3.1
R. Hughes-Jones Manchester
Pairs
64 144 54 67 70 203 190 114 209
UK US
(3) (16) Avg
0.3 0.2
0.7
0.5 3.5
1.3
0.9 2.7
0.9 0.9
0.8
1.8 1.3
1.4
1.5 2.6
1.0
1.0 2.9
0.8
0.9 4.3
1.2
2.0 4.0
3.8 3.8
2.5
2.1 4.2
1.9
2.5 4.8
2.1
3.9 4.3
3.1
3.0 4.9
3.2 3.2
1.1
6.6 9.5
24.1 12.7 18.3
3.2
2.8 4.4
192 1990
A
v
gAvg
-NA +
(WEU
H+ JP Pairs
Region
COM
0.27
23
Canada
0.74 126
US
0.88 2149
C America
0.89
19
Australasia 1.30
18
E Asia
1.61 215
Europe
1.38 852
NET
2.00
85
FSU2.09
48
Balkans
3.83 109
Mid East
2.70
57
Africa
2.72
45
Baltics
3.12
67
S Asia
3.12
97
Caucasus
3.22
19
S America
6.30 203
Russia
17.57
91
28
Avg
3.16
Pairs
Quality
improvement seen
from SLAC &
NASA
 NASA results courtesy of
Andy Germain, NASA,
Astronomy Sysman Meeting 29/30 April 02
GSFC
R. Hughes-Jones Manchester
29
100
Iperf mem-mem vs file copy disk to disk
Fast Ethernet
OC3 Over 60Mbits/s iperf >> file copy
Disk
limited
0
Iperf TCP Mbits/s
Astronomy Sysman Meeting 29/30 April 02
R. Hughes-Jones Manchester
400
30
QoS: Terms and Concepts
Configurable Queues
Discard
Test
Dequeue
Identify & Classify
Police
Sort
Fail
 Identifying frames – marking / setting IP precedence bits
 Sorting frames into queues
 Selecting which frame to send
 Action taken when a queue is full
Astronomy Sysman Meeting 29/30 April 02
R. Hughes-Jones Manchester
31
QoS: What Next?
 Dante propose the following services:



IP Premium (EF)
(AF) difficult to define in a way that suits most NRNs
Best Efforts
Scavenger “Less than best efforts”
 UKERNA ran a Think Tank to Study QoS requirements in the UK


MB-NG Network development project to test MPLS and QoS
SuperJANET is expected to offer similar services to Dante
 Applications need end to end QoS – so we need to cross:





LAN
SuperJANET4
Dante
Remote NRN
Remote LAN
Astronomy Sysman Meeting 29/30 April 02
R. Hughes-Jones Manchester
32
More Information Some URLs
PPNCG Home page with Stop Press:
http://ppncg.rl.ac.uk/
PPNCG Page for monitoring Astronomy & Astrophysics Sites
http://icfamon.dl.ac.uk/ppncg/astronomy.html
and e-mail:
hepgripe@rl.ac.uk
DataGrid WP7 Networking:
http://www.gridpp.ac.uk/wp7/index.html
IEPM PingER home site:
http://www-iepm.slac.stanford.edu/
IEPM-BW site:
http://www-iepm.slac.stanford.edu/bw
Astronomy Sysman Meeting 29/30 April 02
R. Hughes-Jones Manchester
33