As enterprises push out more video and complex applications, wide area network (WAN) managers are tasked with satisfying...
an insatiable hunger for more bandwidth. But endlessly bonding multiple T1 lines is neither cost-effective nor scalable, and WAN pros who support increasingly distributed environments say managed MPLS networks are becoming too expensive to deploy everywhere.
While advanced services like MPLS still enjoy high adoption rates among enterprises, some IT pros prefer Ethernet-based WAN services. Over the last five years, WAN managers have begun turning to Carrier Ethernet WAN and Metro Ethernet services, once considered fringe technology with scarce availability. These services are attractive because they offer higher capacity links at a much lower cost per bit, but WAN pros say they're also more comfortable using a networking standard they've become intimately familiar with on the local area network (LAN).
"[Metro Ethernet] is like the network we have in our data center—just bigger," said Randy Wert, supervisor of networks and communications at Fort Wayne Community Schools. Wert connects 4,200 employees and more than 31,000 students across 56 schools in Fort Wayne, Ind., via Metro Ethernet. "It's just like any other pipe to me ... [so] it meant one less thing for us to learn and maintain a knowledge base on."
We would've never dreamed [we were capable of] delivering 10 megs to a branch.
Vice President and Manager of Network Engineering, Eastern Bank Corp.
Before migrating to the Metro Ethernet service from Comcast Corp. two years ago, the district had migrated from multiple ISDN lines to bonded T1s and finally to frame relay. With frame relay, Wert had to purchase and maintain routers with frame relay line cards. The transition to Metro Ethernet simplified his hardware requirements. He replaced those legacy routers with inexpensive and easier-to-manage 12-port Ethernet switches.
"At this point, all I really need as far as the equipment edge is an Ethernet switch that can do routing, which is almost any Ethernet switch," Wert said. "I could replace all of these routers."
Ethernet WAN services have seen steep rates of adoption since 2007, according to annual surveys of enterprises and small- to medium-sized businesses (SMBs), conducted by Nemertes Research.
Last year, 71% of the 203 businesses surveyed reported using an Ethernet WAN service, up from 46% in 2007, according to Nemertes' 2010/11 Communications and Computing Benchmark. Adoption is projected to reach 75% by the end of this year and 77% by the end of 2012.
Eric Mermelstein, enterprise infrastructure architect at Columbian Chemicals—a producer of carbon black additives for rubber, plastic, liquid and other industrial products—tunnels replication traffic through a virtual private network (VPN) on a 10 Mbps Ethernet-based Internet circuit provided by Cogent Communications between Columbian's headquarters in Marietta, Ga., and a nearby disaster recovery site.
"Some of the reasons I think people choose Ethernet are that it's easier to manage. It's not as complex as ATM or frame relay," Mermelstein said. "It's a regular network patch cable, versus something like a T1 or an E1."
Bob Primavera, vice president and manager of network engineering at Boston-based Eastern Bank Corp., will migrate five regional offices and 17 branch offices to Comcast's Metro Ethernet service by the end of 2011—replacing point-to-point T1 lines and a 75-mile OC-48 fiber ring from Verizon Business, as well as frame relay and MPLS services from AT&T.
Primavera felt he lost nothing by migrating some sites from MPLS, a Layer 3 technology, to Metro Ethernet services, a Layer 2 technology. On the contrary, he said he has gained more flexibility and scalability.
"We have the ability [to create] any point-to-point or multipoint topology on demand—whenever is required—without having to contact the service provider at all," which would ordinarily be required in an MPLS service, he said. "That is indeed a very powerful asset that we gained by going to a Layer 2 technology."
However, MPLS and Metro Ethernet services also aren't always mutually exclusive, according to Kevin O'Toole, vice president of commercial services at Comcast, which recently announced commercial availability of its Ethernet WAN services aimed at the midmarket. While SMBs view Metro Ethernet as a replacement for MPLS and T1s, large enterprises more often use Metro Ethernet for last-mile access to their MPLS clouds, he said.
In general, MPLS continues to enjoy slightly higher adoption rates than Ethernet WAN services, reaching 80% this year, according to Nemertes. The two WAN services can also run side-by-side, as many carriers offer Ethernet over MPLS clouds. MPLS adoption is expected to hit 82% by the end of 2012, according to Nemertes.
Ethernet WAN services are relatively simple, and networking pros have deep experience with managing the technology inside their own networks, making them an attractive WAN option, especially Ethernet-based Internet connections, according to Brian Washburn, research director at Current Analysis.
"Every [network engineer] knows what an Ethernet interface is, and the interface cost is relatively cheap," Washburn said. "There's also the nice marriage of Ethernet [WAN services] with your own IP services, so you just use Ethernet as a plug to get Internet access today or IP VPNs tomorrow. Whatever you want to do at the IP layer, you can do it with Ethernet [WAN] access."
Ethernet WAN services: Bigger, cheaper pipes
Those warm and fuzzy feelings about Ethernet only go so far in selling WAN managers on Ethernet WAN and Metro Ethernet services. The true headliner is the sheer size of the pipes, which they say are orders of magnitude cheaper than other connectivity options.
When evaluating Comcast's Metro Ethernet service for the Fort Wayne schools, Wert measured it against MPLS services from Verizon and other fiber network operators. MPLS and other fiber services cost anywhere from seven to 20 times more than Metro Ethernet for 1 Gigabit Ethernet (GbE) to the schools and 10 GbE to the data center, Wert said.
"Our monthly fee [for Ethernet] at a gigabit [cost] three times less than [what] the next lowest vendor we were talking to at the time [wanted for] 100 megabits," he said. "I could honestly care less what kind of transport they were going to give us because it was such a huge upgrade."
Mermelstein would like to expand his use of Ethernet WAN services at Columbian Chemicals in the remote corners of the world where the company operates its plants, because they would be a "cheaper and faster" alternative to the existing T1 and 3G access at those plants today. Unfortunately, availability of the technology is limited in those regions.
"Because of where we're located, it's very expensive to bring in a T1 or E1 because they're priced based on distance [and how far] away you are from the central office. It can get really expensive," he said. "[Ethernet] is simpler; it's usually cheap and you usually get higher speeds. We have a 10 meg [Ethernet] circuit [at our headquarters], and with the flip of the switch, we could go up to 100 meg."
Adding capacity to Ethernet WAN and Metro Ethernet services can be done quickly and easily because the service provider does not need to add new copper pairs, as is necessary to bond multiple T1 circuits, said Comcast's O'Toole.
"If a customer buys 20 Mbps, the device we're going to put at that location ... is capable of a gigabit," he said. "We can just provision that electronically. We don't have to go out and literally change the facility."
At Eastern Bank, regional offices on the legacy fiber ring had received 300 Mbps. They now connect via Metro Ethernet at 2 Gbps. Branch offices that had previously been on T1 lines receive 10 Mbps on Metro Ethernet. Although WAN operating expenses haven't dropped significantly, Primavera said he gets triple the bandwidth for the same price.
The Metro Ethernet service has also been more reliable, he said. Although the legacy circuits at the branches used a variety of technologies, they shared a common point of failure: the last mile. No matter the carrier or the connectivity, last-mile access in the suburbs of Boston is delivered over Verizon's copper lines, Primavera said. By contrast, Comcast has built out an independent fiber-to-the-premises network.
"With [our legacy connection], we were always having issues between the central office and where our branches are located," Primavera said. "The circuit was not running clean and we were experiencing high rates of [cyclic redundancy checking] CRC errors on our lines, and with the sensitivity of the applications we support, we demand that those services be clean."
Ethernet WAN, Metro Ethernet capacity enables advanced services
Bigger pipes aren't just for bragging rights. Being able to afford these leaps in capacity has enabled WAN managers to support advanced services that were once unthinkable.
Affordable access to 1 GbE and 10 GbE metro area network (MAN) links has enabled Wert to centralize his servers and support bandwidth-hungry services on Ethernet such as Voice over IP (VoIP), video conferencing and multicast IPTV. Those would have been unrealistic—if not impossible—goals three years ago, when the 37 elementary schools in the district had 256 Kbps pipes, he said.
Now, the connections "don't even need QoS" because there is so much excess capacity, Wert said. Most buildings use 3 to5 Mbps of their 1 GbE links, and the most he has seen one school use is half of that total capacity.
The bandwidth gains have also enabled Wert to support nightly backups over the MAN for the first time in nearly a decade. In the past, before Fort Wayne schools centralized their servers, backups required a staffer in each building to stay after hours to back up the local servers via tape. Because backups were so labor-intensive, they rarely occurred nightly and usually dropped off entirely in the summer, after faculty and staff took off at the end of the school year, Wert said.
"[Metro Ethernet services] provided us with a network that allowed us to ignore the network," he said. "It's hard to quantify [the ROI] in either dollar figures or to some extent the number of projects people have been able to do now because [bandwidth] is just not a problem for us anymore."
Eastern Bank is using its newfound wealth of capacity to support prerecorded videos on digital signage at its banks. Primavera will also use the additional bandwidth to support new banking applications that enable customers to scan checks for deposit and then transport that data over the WAN to the regional offices.
"We would've never dreamed of delivering 10 megs to a branch," Primavera said. "The improvements in bandwidth will only make [those services] turn around even quicker."
Let us know what you think about the story; email: Jessica Scarpati, News Writer.