Untitled 3
SEA2 Network Infrastructure & Architecture Summary
|
Our SEA2 advanced Network Operations Centre (NOC) in Seattle,
Washington is an essential part of our core network. The SEA2 NOC is
staffed by highly experienced Systems & Network Administrators 24 hours
a day, 365 days a year, and features three (3) parallel N+1 500KVA UPS
systems with maximum battery life, a 1.5MW diesel generator, and 100s of
tons of AC capacity.
The local network is arranged in four logical segments:
-
Core - redundant Cisco 12008/12012 GSR routers connected to
the Internet and the rest of our national coast-to-coast
network via multiple Gigabit Ethernet lines; running the
full BGP4 protocol, thus ensuring route optimization and
redundancy.
|
-
Distribution A - redundant Catalyst 4912G master switches
that ensure a fully redundant switching fabric between the
core routers and the DistB distribution switches.
|
-
Distribution B - redundant high-end Cisco Catalyst 6509 MSFC2
/ Supervisor 2 master routing switches that distribute
access to the physical and virtual LANs and route internal
traffic.
|
-
Access - redundantly 2 Gb/s uplink connected Catalyst
2950G-48-EI switches to which all servers are connected.
Each and every server gets a full 100Mb/s full duplex port
to the access layer. 1 Gb/s access ports are also available.
|
Our network does not use any 10Mb/s switches, or any hubs - at all.
It is completely Gigabit Ethernet (multiple, redundant 1000 Mb/s for all
network layers) based. Both internal and external network provide
multiple, redundant, paths. Each and every piece of the network is
fully redundant: there is no single point of failure. The network
also includes a number of multi-terabyte tape and RAID-based backup
systems.
Physical Network Infrastructure
OnFiber, q.West, ELI, Integra Telecom, and Sabey, some of the most
financially healthy fibre carriers, provide our local private,
redundant, SONET-ring based fibre transport circuits.
The multiple, different carrier, OC-192 SONET rings each
consist of two redundant OC-192s traversing two separate paths and
having two diverse routes to our facilities, thus providing the ultimate
in reliability and performance. Even if both of the lines are cut at any
point, the OC-192 ring is still operational, with less than 50 ms (0.05
second) self-healing restoration time.
The multiple private SONET rings provide us the ultimate
in reliability: no downtime even in the case of a major fibre cut in the
area (due to separate paths and multiple fibre transport providers).
A number
of other fibre carriers are on site, too: ELI, Integra Telecom, OnFiber,
q.West, AboveNet and Level 3 just to name a few, and more are nearby.
Furthermore, we do not only connect within the Seattle metro area. The
Seattle portion of our network is fully tied in with our national
coast-to-coast IP backbone, and connects to Palo Alto, CA (PAO1)
directly and Palo Alto, CA (PAO1), Chicago, IL (CHI1) and Ashburn, VA
(IAD1) via our Seattle, WA site (SEA1).
|
Traffic from SEA2
considers all the outgoing routes - be it at the same site or at one of the many
other sites - weighted for physical distance, reliability, performance and
latency, and picks the most direct, lowest latency route available to the
destination. That is, all the packets take the most direct, fastest, way out and
have a full BGP selection of all the transit providers and peers at all the
sites, not just the transit/peering lines present at that particular site. The
network is fully inter-connected and meshed.
For more details
on our core network, such as a full backbone connectivity and peering diagram
and network maps of our transit Tier 1 backbone connectivity providers, please
review details on our core network.
Seattle is one of
the best connected high-tech cities in the US (all major networks peer locally
in Seattle, and it is under 30ms to California, under 75ms to the East Coast),
and has some of the most reliable power anywhere. Furthermore, we are located in
a stable and level area (ground level of a well-built single story facility),
providing the best resistance to earthquakes, and have multiple diverse-path
fibre transport circuits with OnFiber, q.West, ELI, Integra Telecom and Sabey.
In Seattle we
peer both at the SIX (Seattle Internet Exchange) and PAIX Seattle, thus reaching
almost all the major networks locally. This is further built upon by our
redundant connectivity to multiple Tier 1 providers via separate circuits,
terminating at diverse locations in Tukwila (south of Seattle) and Seattle
downtown. We are redundantly set-up Internet connectivity wise (multiple
backbone connectivity providers), as well as physically. In terms of fiber
optics, we have multiple private SONET rings with multiple financially-healthy
fibre providers, taking different physical paths.
We have built our
facilities in the best possible area for Internet connectivity, and then further
reinforced it with redundancy in all possible forms and factors. The big 1.5MW
generator can keep well over 5000 servers up and running indefinitely (though
with the very reliable and abundant electricity in Seattle, power there is not
typically an issue). The three (3) N+1 redundant 500KVA UPS systems (with
maximum battery life each) ensure that even if one UPS fails, there is no
noticeable impact whatsoever. There is currently 180 tons of Air Conditioning
within the data centre, with easy ability to triple it to 540 tons using the
existing dry coolers. The entire data centre has been built out with no expense
being spared, and with further future expansion in mind. As it stands now, it is
already able to accommodate 5,000 - 8,000 total servers.
To ensure the
best routing of our customers' data, we pay for commercial top-priority transit
network capacity to our tier 1 backbone providers and have full burst capacity
available at all times on all circuits. Thus we receive the highest level of
service, including 99.99% or 100% uptime and no packet-loss guarantees, from all
our connectivity providers. 90%+ of traffic from us to end users only passes
over one or two backbones on its way to reaching the end destination. In
addition, we are committed to under 50% network capacity utilization at all
times; we ensure that our network always has double the capacity it needs, even
at peak hours.
Our network
design and operations ensure the very best connectivity, speed, and reliability
possible: 100% network uptime, no packet loss, and under 3 ms low latency
connection to Tier 1 backbones .
Our SEA2 network & site in summary: