Who doesn't have a horror story about their wide area network (WAN) service provider? Swapping carriers isn't a...
casual decision, given the expense and inconvenience of installing new circuits at 75 branch offices. But there are some things WAN managers shouldn't put up with -- namely, frequent outages and consistently dismal customer service.
Bob Andreini hit the breaking point with his previous WAN service provider about three years ago. Andreini, global director of ISIT at Measurement Specialties, a global designer and manufacturer of sensors and sensor-based systems, had been using one operator, which he declined to identify, for connectivity and managed services at manufacturing plants throughout the United States, Europe and China.
More on WAN service providers
"It just wasn't going well. We measured [WAN performance] and we were having failures that we could actually measure in days per quarter, which is really horrible. You usually want to run 99.999, which [translates to] minutes [of downtime] -- not days," Andreini said. "And, of course, it would always happen at the month's end or quarter's end."
Andreini gave the WAN service provider many chances to redeem itself, but the carrier consistently failed to deliver for the 11 manufacturing plants it supported at the time.
"We didn't [change WAN service providers] right away," he said. "We really tried to work with the provider for a long time. We tried to give these guys every opportunity [to improve]."
The WAN service provider had been bought out and seemed to have undergone a turnover in engineers, which affected WAN uptime and customer service, Andreini said. After the merger, the operator regularly failed to give his team any advance notice about scheduled downtime or maintenance, which could result in day-long WAN outages at any of the sites.
"The provider, quite frankly, couldn't manage the network," he said. "It happened more than once, so we decided that we couldn't run our business this way."
Andreini switched to Virtela in 2007. The changeover required a hectic few weeks of ordering and installing new circuits, plus reconfiguring and testing all his network hardware, but he has no regrets about making the change.
Since dumping the previous provider, Measurement Specialties has added three new plants to the WAN and experienced "virtually no problems" in terms of network performance and support, Andreini said. If any problems arise, he and his global infrastructure team have "very easy access" to Virtela's global network operations centers (NOCs).
"Most of the time, we [learn of] an issue when Virtela tells us," Andreini said. "Basically, I call one number anytime -- 24 hours a day -- and somebody answers the phone. Usually, it's not just somebody who's going to take your information and say, 'You'll get a call back in two hours.' I get somebody who can start helping me right away."
Dumping the previous WAN service provider has enabled Andreini and his team to spend less time responding to network crises and more time developing and implementing bigger IT projects.
One of the hardest things to do is change a telecom vendor, and you don't take these [decisions] lightly. But I wasn't sleeping well at night, and I'm hoping I sleep a little better now.
Mike Wald, Vice President of IT Operations, Flowserve Corp.
"We were spending almost all of our time keeping the network running, and now we're able to focus a lot more … on value-added services for the business -- global ERP systems, global CRM applications. We're doing a lot with virtualization," he said. "We just didn't have time for this stuff before."
WAN service provider offers enterprise more sales pitches than support
After giving his previous WAN service provider 10 years of second chances, Mike Wald, vice president of IT operations at Flowserve Corp., a machinery supplier for energy companies, said he could no longer stomach the terrible customer support he had been receiving.
The WAN provider, which he declined to identify, was slow to respond to questions or problems and failed to deliver timely reporting, Wald said. But they were quick to push sales pitches whenever they had the chance, he added.
"It was very difficult to push money across the table to them without it becoming a long contracting process or a long discussion about services we weren't interested in," Wald said. "[Customer service] had actually gotten better, but you reach a point where it isn't getting better fast enough."
Wald was confident he could negotiate a lower price with another WAN service provider, but he was more interested to hear how his next carrier would improve service and support.
Flowserve wanted a provider with a "footprint large enough for what we need," which could deliver customer support from "actual badged employees, versus a subcontractor," Wald said. He also wanted his next WAN service provider to have a local presence near Flowserve's 18 sites around the world to simplify support and billing, as they could be handled locally.
After inviting four WAN service providers to bid -- including his previous one -- Wald chose Orange Business Services for WAN connectivity and administration, managed WAN optimization and managed wireless network services.
"We asked each of the vendors to talk us through a monthly report," Wald said. "Orange sat down with a complete top-down review on a monthly basis of the hot points, the cold points, over-utilized, under-utilized -- the parameters you would expect of a highly customer-focused organization."
Even after the honeymoon period, he said, service and support has been consistently good.
"One of the hardest things to do is change a telecom vendor, and you don't take these [decisions] lightly," Wald said. "But I wasn't sleeping well at night, and I'm hoping I sleep a little better now."
Network services get a bad rap owing to WAN service provider problems
When Rick Drescher was hired as a consultant seven years ago at Studley Inc., a New York City-based real estate services firm, his first task was to straighten out the firm's network performance problems with its WAN service provider at the time, Qwest Communications.
At least one of Studley's 14 branches at the time -- they now have 19 -- would suffer a network outage every week, Drescher said. Downtime lasted anywhere from 15 minutes to an entire business day.
"The performance was so bad and our reliability was so bad that it ended up impacting what the end users thought about other IT services when the other IT services were fine," said Drescher, now director of technical services at Studley. "It was the network that was down."
Although WAN performance was the biggest problem, it was compounded by the lackluster support Drescher said he received from overseas call centers. At one point, he had to use a voicemail transcription application to understand the agent's message because of his thickly accented English and the jitter from the call being an international voice over IP (VoIP) call.
"[Although] I'd rather have the network be up most of the time than have to deal with a billing problem … it got to the point where I [was] paying [our provider] $40,000 a month for my WAN and data center [connectivity] and I [couldn't] even open up a ticket," Drescher said.
He has used AT&T as his primary WAN for the past seven years with fewer problems. He said having a Master Service Agreement in place with a new WAN service provider before he started ordering services means there are no more surprises.
"When we order new circuits now," he said, "it's brain-dead easy."
Let us know what you think about the story; email: Jessica Scarpati, News Writer