Download Campus Network Design

Brittany Cunningham
Victor Antonov
Trevor Marsh
8 December 2009
1.
2.
3.
4.
5.
6.
7.
Design Decisions
Population & Needs
Wide-Area Network
Routing Protocol
Main Campus
Satellite Campuses
Remote Campuses
2009.12.08
Campus Network Design
Remote Access
VoIP
Wireless
Security and
Authentication
11. Network Management
12. Costs Evaluation
7.
8.
9.
10.
2
Brittany Cunningham






Route summarization
Distributed routing and switching
Simplified implementation and management
Broadcast domain control
Infrastructure changes
Quality of Service
2009.12.08
Campus Network Design
4
2009.12.08
Campus Network Design
5
Victor Antonov
2009.12.08
Campus Network Design
6

Students
 WWW, e-mail, multimedia access

Staff
 E-mail, VoIP, WWW

Faculty
 E-mail, VoIP, multimedia/WWW

Research
 VoIP, e-mail, multimedia



Most student access will come from the
dorms but some will be from academic access
points
Student needs will be mostly in download
bandwidth
Upload (disregarding video upload) is not
expected to be great. Illegal upload needs to
be discouraged.
Type of Object
Size in
Kb
# objects DL
# objects UL
traffic DL (MB)
traffic UL (MB)
inte
intra
r total
intra
inter
total
intra
inter
total
intra
inter
total
E-mail message
10
5
30
35
2
10
12
732
4,395
5,127
293
1,465
1,758
Web page
50
10
190
200
1
2
3
7,324
139,160
146,484
732
1,465
2,197
Spreadsheet
100
2
1
3
1
1
2
2,930
1,465
4,395
1,465
1,465
2,930
Word processing
document
200
2
2
4
2
1
3
5,859
5,859
11,719
5,859
2,930
8,789
Image view/upload
500
5
50
55
3
20
23
36,621
366,211
402,832
21,973
146,484
168,457
Presentation document
2,000
1
1
2
1
0
1
29,297
29,297
58,594
29,297
0
29,297
5 min songs @ 96 kbps
3,600
0
100
100
0
20
20
0
5,273,438
5,273,438
0
1,054,688
1,054,688
230,400
0
1
1
0
1
1
0
3,375,000
3,375,000
0
1,687,500
1,687,500
80.8
8979.3
9060.1
58.2
2828.1
2 hrs of movie @ 256 kbps
2886.3 GB
859.0359
total Mbps
273.6681 (24 hrs)
1288.553
9
total Mbps
410.5021 (16 hrs)
* Estimated 15,000 students
2009.12.08
Campus Network Design
9
Type of Object
Size in Kb
# people
# objects DL
intra
inter
# objects UL
total
intra
inter
traffic DL (MB)
total
intra
inter
traffic UL (MB)
total
intra
inter
total
Terminal screen
4
2,000
30
0
30
20
0
20
234
0
234
156
0
156
E-mail message
10
15,000
5
15
20
2
10
12
732
2,197
2,930
293
1,465
1,758
Web page (including
simple GIF and
JPEG graphics)
50
15,000
15
30
45
1
2
3
10,986
21,973
32,959
732
1,465
2,197
Spreadsheet
100
15,000
2
1
3
1
1
2
2,930
1,465
4,395
1,465
1,465
2,930
Word processing
document
200
15,000
2
2
4
2
1
3
5,859
5,859
11,719
5,859
2,930
8,789
Graphical computer
screen
500
5,000
3
0
3
0
0
0
7,324
0
7,324
0
0
0
2,000
15,000
1
1
2
1
0
1
29,297
29,297
58,594
29,297
0
29,297
50,000
10,000
1
1
2
0
0
0
488,281 488,281
976,563
0
0
0
115,200
5,000
0
1
1
0
0
0
0 562,500
562,500
0
0
0
2,400
5,000
5
3
8
5
3
8
93,750
Presentation document
High-resolution (printquality) image
1 hrs of video stream @
256 kbps
VoIP
58,594
35,156
93,750
58,594
35,156
532.9
1085.5
1618.4
36.9
7.2
44.1 GB
153.4460
total Mbps
4.1784 (24 hrs)
230.1690
total Mbps
6.2676 (16 hrs)



Least amount of traffic generated
VoIP telephony important
Higher UL rate because of audio and video
links
Type of Object
Size in Kb
# people
# objects DL
# objects UL
traffic DL (MB)
intra
inter
total
intra
inter
total
intra
inter
traffic UL (MB)
total
intra
inter
total
E-mail message
10
200
7
8
15
7
8
15
14
16
29
14
16
29
Web page (including simple
GIF and JPEG
graphics)
50
200
10
25
35
0
0
0
98
244
342
0
0
0
Spreadsheet
100
200
1
1
2
1
1
2
20
20
39
20
20
39
Word processing document
200
200
3
2
5
2
3
5
117
78
195
78
117
195
Graphical computer screen
500
100
1
0
1
0
0
0
49
0
49
0
0
0
2,000
100
1
1
2
1
1
2
195
195
391
195
195
391
50,000
100
0
1
1
0
1
1
0
4,883
4,883
0
2,400
200
10
20
30
5
10
15
4,688
9,375
5.1
14.5
Presentation document
High-resolution (print-quality)
image
4,88
14,06
VoIP
2,34
3
19.5
4,883
8
7,031
4,68
4
2.6
3
9.7
12.3
1.850
GB
9
1.1637
total Mbps
(24 hrs)
8
3.4912
total Mbps
(8 hrs)
5.552




Most research organizations and universities are connected
via Internet2 – a research network
Internet2 is developing and deploying advanced network
applications and technologies for research and higher
education
Internet2 recreates the partnerships of academia, industry,
and government that helped foster today’s Internet in its
infancy.
Research partnership gives access to (anonymized) traffic
data unavailable from commercial networks



Some areas of research can generate huge
amounts of data
A separate line will be dedicated to the
research needs and access to Internet2
Needs for some areas of research are
described in the next slides


Dependant on the area of physics but usually produces large
amounts of data
Russian example on High Energy Physics research
 In 2003 produced ~30 TB
 Predicted needed connectivity for 2006 was 1-2.5 Gbps
 While a university might not produce all this data and exchange it with
the world, it is safe to assume that in 2009-2010 all educational
physics research might need ~2 Gbps connection

Some examples of physics research applications:
 Large, high-quality images of the sky (astrophysics)
 Complex 3D models (fluid/air dynamics)




Audio and visual information on species,
habitats, conditions
DNA models, genetic sequences
Neuroinformatics - neuroimaging resources,
including multi-scale imaging
Protein identification, characterization,
quantification

Other areas of research that will produce a lot
of traffic over the network:
 Weather science
 High-performance computing
 Chemistry
 Geography
Victor Antonov


Main Campus
4 Secondary Campuses
 In the same metro area as main campus

50+ satellite campuses
 Nationwide

Connections to the Internet and Internet2
 Serving main and secondary campuses

Redundancy of the WAN





Metro Ethernet technology to connect smaller campuses
EVPL (Ethernet Virtual Private Line) topology with point-topoint Ethernet virtual connections
Multiple EVCs to enable hub and spoke configuration
Bandwidth of 1Gb (which can be later scaled up for growing
bandwidth needs)
Two providers for redundancy: COX and Verizon





Cost-effectiveness
Scalable bandwidth (1Gb and higher)
Low operating, maintenance, administration
costs
Simplicity of native Ethernet format over
traditional WAN technologies
Customer controls IP addressing and routing


Layer 2/3 switches and/or routers
Highly redundant network
 Full mesh topology

MPLS backbone
 Costly
 Highly reliable and scalable

Benefits of MPLS (basic)
 Node-to-node connections (virtual links)
 Highly scalable
 Independent of any Data Link layer technology
 Less overhead (no segmentation and reassembly)
 Highly compatible with IP

Benefits of MPLS
 Connections are unidirectional
▪ A bi-directional traffic will use two connections which
allows a link failure to ideally affect only one of the
traffic directions
 Multi-level tunneling
 Fast recovery time – MPLS Fast Reroute offers
recovery time of <50 ms
▪ Geared towards real-time application (VoIP) support


Ethernet interface on fiber (100BASE-FX)
Ethernet over MPLS over Ethernet
 Customers’ Ethernet packets are transported over
MPLS and the service provider network uses
Ethernet again as the underlying technology to
transport MPLS

Fast Reroute Implemented

Scalability
 pure Ethernet MAN are limited to a maximum of 4,096 VLANs for the
whole network, when using MPLS, Ethernet VLANs have local meaning
only

Resiliency
 30 to 1 sec convergence for pure Ethernet vs 50 msec for MPLS-based
MAN (Fast Reroute)

Multiprotocol convergence
 an MPLS-based Metro Ethernet can backhaul not only IP/Ethernet traffic
but virtually any type of traffic coming from customer networks or other
access networks

End to End administration and maintenance
 MPLS-based MAN offers a wider set of troubleshooting and OAM MPLS-
based tools which can effectively troubleshoot and diagnose network
problems
 MAC ping, MAC traceroute, LSP ping etc.

University is the provider itself
 It will receive internet access and provide it to main and secondary
campuses
 Can provide access for closely related organizations – research
foundation , R&D sites, high schools
 Operates and administers its own network
▪ Can freely implement policies


Main campus is closely connected with the core network
Customers are secondary campuses and an related
organizations (see above)

Two providers of the metro-ethernet services
 COX and Verizon

Ethernet solutions: EVPL (Ethernet Virtual Private
Lines) topology with point-to-point Ethernet virtual
connections (EVCs)
 Multiple EVCs will be used to enable hub-and-spoke
configuration to interconnect campuses.


Separate internet access
OC-1 lines offering ~50Mbps transmission
speeds
 Main BW consumer is distance learning video links
▪ Assuming roughly 120 students per remote campus, this
is 30 Mbps traffic at peak times

Access to university resources achieved
through VPN
MetroEther
net Area
Network
(main and
secondary
campuses)
Cox
Verizon
Satellite
Campuses
Brittany Cunningham
What determines convergence time?
 Time to detect path loss
 Time to detect new best path
 Time to update routes and tables
2009.12.08
Campus Network Design
33




Stubby areas
Hierarchical design limits queries
Fast convergence
Cisco hardware is optimized for EIGRP
2009.12.08
Campus Network Design
34





Fewer queries to core
Allows traffic filtering
Control multicast traffic
Smaller routing tables
Naturally synergizes with hierarchical design
2009.12.08
Campus Network Design
35




Rendezvous point near multicast source
Auto-rendezvous on all other L3 switches
IGMP snooping
No cross-campus VLANs
2009.12.08
Campus Network Design
36
Brittany Cunningham







15 buildings
Approximately 750 faculty and staff
Approximately 15,000 students
Electronic records
VoIP phone system
Complete wireless coverage
Research
2009.12.08
Campus Network Design
38
2009.12.08
Campus Network Design
39
2009.12.08
Campus Network Design
40


WAN links to partnered universities
High-performance computing clusters
2009.12.08
Campus Network Design
41
Brittany Cunningham







1-4 buildings each
Approximately 250 faculty and staff
Approximately 8,000 students
VoIP phone system
Complete wireless coverage
Backups from main server farm
WAN links to main campus
2009.12.08
Campus Network Design
43
Brittany Cunningham



50+ remote sites
Approximately 2,000 students
Local staff with access to university resources
2009.12.08
Campus Network Design
45


Faculty and Staff must have secure access to
files and other resources
Access must be available anywhere with an
internet connection
Solution: VPNs
2009.12.08
Campus Network Design
46

Consider:
 What resources should require a VPN?
 What resources could be supported by web VPNs?
 How can we make connecting as easy as possible?

Adaptive Security Appliance
2009.12.08
Campus Network Design
47
Brittany Cunningham



Main and satellite campuses only
Traffic is in separate traffic VLAN
802.1Q VLAN tagging to ensure QoS
2009.12.08
Campus Network Design
49
Trevor Marsh

Cisco’s Unified Wireless Network
 Quality name
 Guaranteed support won’t end in a year because company
bankrupts
 Provides easy and proven configurations
 Offers:
▪ Context Aware: Track assets, perform condition monitoring,
improve process flow, and use location and other contextual
information
▪ Wireless Network Security: Proactive threat protection, RF
visibility, and wired network security help ensure that data remains
private and secure and that the network is protected from
unauthorized access.
▪ Radio Frequency (RF) Solutions: Spectrum analysis can help detect
and eliminate sources of RF interference in wireless networks.

Cisco Catalyst 6500 or 7600 series switch
 After placement of a Cisco Wireless Service
Module(CiSM) you can have up to 2100 access
points

Use Cisco Aironet 1250 series access point
 Allows for upgrade to 802.11n

Management of all of the access points is
easier due to Cisco’s use of LWAPP
(Lightweight Access Point Protocol)
 Handles all of the access points at once
 Can assign each access point with a primary and
secondary controller

Each wireless controller will be bundled with
the switch which will allow access to the
distribution layer

This allows for the usage of the same DCHP
server and access to anything else in the
Distribution Layer, provided properly
accessed, such as storage and others.


802.11n is not yet popular enough
802.11a 5.2Ghz band will be primarily used
while 802.11b/g (2.4Ghz) will be sparingly
used for legacy devices

Potentially less interference
 Provides at least eight, and potentially up to 22,
non-overlapping channels, compared with three
for 802.11b/g
 Allows for auto-configuration of channels and
power to access points

There will be one controller per switch, which
means two controllers per building
 Placement in all buildings will allow for enough
coverage for all of ODU
 If one fails the other will automatically cover the
slack
 Automatic reboot after 3 minutes

If any access point fails, the CiSM will increase
the power to the others
Brittany Cunningham




Located in Distribution Layers
Additional ACLs may be on Access Layer
No ACLs in Core-Why?
Careful planning is necessary during design
and implementation
2009.12.08
Campus Network Design
59



DHCP snooping
Intrusion Detection Systems (IDS)
Port security
2009.12.08
Campus Network Design
60





Resnet gateway
Server gateway
Between core and exterior gateways
Remote site gateways
VPN connection gateway
2009.12.08
Campus Network Design
61
Brittany Cunningham

TACACS+ for networked devices
 Authentication
 Authorization
 Accounting


Locally-configured credentials as backup
Solarwinds Network Monitoring System
2009.12.08
Campus Network Design
63
Brittany Cunningham
Item
Quantit
y
Cost per Unit
Total Cost
Catalyst 4500 Series Switch
75
$8,000
$600,000
Catalyst 6500 Series Switch
16
$20,000
$320,000
ASA 5500 Series
2
$3,000
Wireless Access Points
1,200
$800
$960,000
Cisco 6500 Wireless Services Module
8
$30,000
$254,000
Cabling Estimate *
1
$1,000,000
$6,000
Hardware Overhead (40%)
$1,000,000
$1,256,000
TOTAL
$4,396,000
* University will hire a contractor for all cabling.
2009.12.08
Campus Network Design
65
Item
Cost
Orion Network Performance Monitor (500 devices)
$8,475
Orion Netflow Traffic Analyzer (500 devices)
$5,995
Orion IP SLA Manager 1 (25 IP SLA source devices)
$3,995
Orion Network Configuration Manager (1000 nodes)
$10,495
LANsurveyor
$1995
IPv4 Allocation and Assignment (ARIN; /20)
$2,250
IPv6 Allocation and Assignment (ARIN; /40)
Free w/ IPv4
AS Number Assignment (ARIN)
$500
ARIN Maintenance Fee (Per Year)
$100
Non-Hardware Overhead (40%)
$13,522
TOTAL
2009.12.08
Campus Network Design
$47,327
66












http://www.uwec.edu/hiltonts/101/CBAsample/projectsam
ple.htm
http://cisco.com
http://www.ciscopress.com
http://www.netcraftsmen.net/resources/archivedarticles/431.html
http://etutorials.org/Networking/Lan+switching+first-step
http://www.engr.wisc.edu/computing/security.html
http://www.solarwinds.com
http://www.arin.net
Rizwan Bhutta, Network Systems Senior Engineer
Sheila Brink, Network Systems Senior Engineer
Jeff Spyker, Network Systems Senior Engineer
Robert Perry, Network Systems Senior Engineer
2009.12.08
Campus Network Design
67