Wide area network (WAN) performance and application delivery rarely troubles an IT organization that maintains local servers in every branch. When an organization decides to consolidate those servers in a central data center, it's often difficult to predict the impact such a project will have on the WAN.
Most WAN managers can make educated guesses based on branch size and user behavior, but this foresight was fuzzier for one IT pro supporting branch environments with a large floating user population and unreliable network connections. The combination of WAN optimization and WAN monitoring tools hasn't especially improved application performance over his WAN, but those technologies have ensured that users are none the wiser to the fact that once-local resources are now hundreds or thousands of miles away.
"We thought we were going to have to upgrade our data network once we started moving all this traffic onto it, and what we found was we've never had to upgrade our networks for bandwidth [at] any of the offices," said Gerry Holmes, IT director at the Canadian Cancer Society, a Toronto-based charity that provides support for cancer patients and research across Canada. "[Users] never noticed that we'd taken anything out of the local office."
We're providing enhanced services over the same network.
IT Director, Canadian Cancer Society
Although the Canadian Cancer Society's Ontario division appears midsized by employee count—about 475 staff spread across 40 locations in the province—there are another 65,000 volunteers who support the organization. Of course not all use the WAN, but the impact of those who do can be unpredictable. A branch could support a handful of volunteers one week, and perhaps 20 the next week, Holmes said.
A typical branch office connects to the WAN via a DSL circuit that provides 2 Mbps download and 600 Kbps upload speeds. Typically, DSL service-level agreements (SLAs) offer fewer availability and mean time to repair (MTTR) guarantees than SLAs for other WAN connectivity options, such as point-to-point T1 circuits and MPLS services.
Until five years ago, IT services for each of the nonprofit's 35 Ontario branch offices were supported by PCs configured to function as makeshift servers. The setup was ripe with security and storage vulnerabilities, prompting Holmes to evaluate the cost of replacing them with business-class servers—about $5,000 for each office.
At the same time, he deployed virtualization in his data center using VMware; his reseller suggested centralizing the branch servers and looking into WAN optimization products from Riverbed Technology. It was an attractive alternative for ensuring WAN performance, especially given the fact that the branches had no local IT presence, Holmes said.
"We looked at this and said, 'If we could use this device, we would be able to get rid of all of those servers out in the offices and all of the work associated with maintaining 35 remote offices," he said. "When you're talking about sending someone from Toronto to Thunder Bay or Timmins, these places aren't drivable—you have to fly to them, which is quite expensive—and even some of our offices you could drive to, it's a four- to six-hour drive."
Improved WAN performance inspires confidence in new services
Holmes piloted Riverbed's WAN optimization appliance, the Steelhead, and discovered that users suffered no degradation in WAN performance or application delivery from the consolidation of servers from the branch to his data center.
"They didn't notice anything," he said. "We never had any complaints about the centralization."
But when he momentarily shut off the Steelheads during the pilot, users almost immediately started complaining that email was lagging. The email server had always been centralized, but users had rarely noticed, Holmes said. Its delivery model only became painfully obvious to users once optimization was turned off and it competed with other centralized services on a modest WAN link.
"Within half an hour, we had a call from one local office asking what we had 'done to' the email," he said.
Holmes also deployed Riverbed's monitoring platform, Cascade, to catch and analyze any WAN performance hiccups and improve the MTTR. Cascade immediately helped him resolve an ongoing WAN performance problem at one of his branches. Cascade revealed that a user had been chewing up much of the available bandwidth by streaming Internet radio.
"Up until that point [when we had deployed Cascade], we couldn't see it. You knew something was wrong, but you couldn't really tell what it was," Holmes said. "Everything is optimized now, but there are still things that can go wrong. People can change their usage patterns and use applications they weren't using before ... and when users start complaining about slowdowns, it's easy to suspect the network."
WAN optimization enabled Holmes to squeeze more capacity out of his pipes and support Quality of Service (QoS) for the first time at locations with DSL circuits, which didn't natively support QoS. Confident in his ability to ensure WAN performance for a centralized application delivery model, Holmes ripped out his legacy Nortel TDM switches from the branches and deployed Mitel's Voice over IP (VoIP) products in his data center to support VoIP, centralized voicemail and warm transfers with direct inward dialing (DID) over the WAN.
Holmes has also deployed the Steelheads between his primary data center in Toronto and a nearby disaster recovery (DR) site, enabling him to support nightly WAN backups for the first time. Prior to the Riverbed deployment, branch office DSL lines usually choked on any backup attempts.
"We're providing enhanced services over the same network," Holmes said. "I don't think we could operate today without it."
Let us know what you think about the story; email: Jessica Scarpati, News Writer.
Dig deeper on WAN optimization and application acceleration