Summary:
- Customer Since 2012
- Located in Las Vegas, NV
- Industry: Business Process Outsourcing and Contact Center Solutions
- Company Size: Midsize Enterprise
- Founded and Status: Knoah Solutions was founded in 2006 and is now part of 24-7 Intouch
Services Provided:
- Technical Consultation – Recommend solutions based on business requirements.
- System Engineering – Built custom solutions where no commercial option existed.
- Systems Integration – Created custom integrations between the phone system and CRM products.
- Maintenance and Support – Providing ongoing training, maintenance, and support.
Challenge:
Knoah Solutions needed to replace an unstable call center telephony system with a high availability system and integrate it with various CRM systems.
Benefits:
- Made the telephony system much more reliable
- Reduced infrastructure costs
- Provided skills not found in-house
About Knoah Solutions
Knoah Solutions, Inc. has provided management solutions for customers worldwide since 2006. The company offers business process outsourcing and contact center solutions. Knoah Solutions is now part of 24-7 Intouch.
The Knoah Solutions call center in Las Vegas, Nevada, serves a wide swath of different industries, including medical, real estate, housing services, and more. Knoah employees are custom trained to represent the interests and act on the behalf of each specific client when their customers call for help.
Background
The company’s telephony system is absolutely mission-critical to the business. Customer service representatives in the Las Vegas center alone take thousands of calls per day on behalf of Knoah’s clients. When Knoah changed phone system providers from Cisco to Asterisk, Cloud Brigade was called in to create a high availability system with automated failover and custom integrations with customer relationship management systems.
Business Challenges
- Business Bottlenecks – Knoah’s failover process for its telephony system was manual and kludgy. Calls and data could be lost in the transition from one system to another.
- Irresolvable Complexity – Cloud Brigade had to invent a high availability phone system and figure out why calls were dropping intermittently.
- Inefficient Systems and Processes – Cloud Brigade created custom integrations with CRM systems to pre-populate caller information so that customer service reps know a caller’s history.
- Skills and Staffing Gaps – Cloud Brigade provides unique expertise that Knoah’s IT team doesn’t have.
Why Cloud Brigade
Knoah’s IT team had just completed a migration from a Cisco telephony system to one based on the open source software Asterisk. Cloud Brigade was brought in to set up high availability failover in case of a massive failure of the primary system. High availability means that there is a replicated, stand-by system that can immediately and automatically take over operations if the main system fails for any reason.
Mike Barletta, IT Manager at Knoah Solutions, explains the reason Cloud Brigade was hired. “It’s an obscure bit of technology to work with telephony systems. Our internal team is good but we needed someone to deal with the minutiae. Finding someone with that knowledge is hard. Chris Miller and his team at Cloud Brigade have the tenacity to really dig in on these systems to help us get exactly what we need.”
“We have a pretty good internal IT team here. We’re certainly not slackers. If we go to Cloud Brigade for a fix, it’s not going to be for a run of the mill problem. We need someone at Chris Miller’s level or close to it.”
– Mike Barletta, IT Manager, Knoah Solutions
Ensuring Availability of the Call Center Phone System
Cloud Brigade’s first assignment was to bring more reliability to the new PBX system based on Asterisk open source software. Knoah’s previous PBX system had a long history of failure, so setting up this mirror image failover capability was critical. At the time, there was no commercial high availability solution on the market, so Cloud Brigade architected and designed a system that replicated the primary PBX (“Side A”) to a secondary stand-by system (“Side B”). Both sides must stay in sync in terms of the capabilities and data they support. However, only one system is in use at a time.
Setting up a high availability system from scratch is a complicated process, notes Chris Miller, Founder and CEO of Cloud Brigade. “It’s not for the faint of heart,” says Miller. The Linux-HA system is very complex, and at that time the documentation was problematic. The Cloud Brigade team really had to dig to find the right information to get the system set up properly. Miller says there were challenges in figuring out how to do the storage replication and database replication to keep the A and B sides in total sync. Also, they had to come up with workarounds for the networking layer where the two sides have to communicate with each other. “We did a lot of testing until we were satisfied that the system would work well.”
As Knoah’s business grew, they needed to scale this phone system to accommodate a greater volume of calls. Cloud Brigade addressed the growth requirement by setting up clusters of high availability machines for the call center. Knoah’s various clients were assigned to different sets of computers that all had their Side A and Side B replication.
Cloud Brigade also set up a call center monitoring system based on QueueMetrics software. This lets Knoah track all sorts of metrics pertaining to the calls that come in and the call center agents’ performance. Cloud Brigade set it up to run outside of the PBX phone system, but also in high availability mode.
Cloud Brigade continues to maintain and support the telephony and metrics systems to keep them running smoothly. Every now and then a replication hiccup occurs and those issues get ironed out. The system that Cloud Brigade engineered has been remarkably stable over the years, providing Knoah with comfort knowing that calls will not be lost if there ever is a failure such as a server crash anywhere in the system.
Custom Integration Puts Caller Information on Agents’ Screens
The Knoah call center agents need to have as much information as possible about the people on the other end of a call—name, location, previous interactions with the call center, etc. This information comes from the customer relationship management systems like Salesforce belonging to each Knoah client. Having this information is made possible by the custom integrations Cloud Brigade developed between the call flow software and the clients’ different CRM applications.
“When a call comes in, the system resolves the inbound number and does a database lookup. It populates the screen with data and pushes the link to the agent to view on their screen,” says Barletta. “This enables our agents to pick up the phone and say, ‘How are you doing today, Mike? I see that you have been a customer for 6 years.’ All of this takes place before the caller even says anything. We also have the person’s complete history of calling our support center. It’s a great way to put the callers at ease by showing that we know them and stand ready to help them. All of this happens because of the integrations Cloud Brigade set up for us.”
Cloud Brigade Tackles the Vexing Problems
The Cloud Brigade team has had to tackle a few really vexing issues that no one else could solve. “We had an oddball problem with calls occasionally getting disconnected,” says Barletta. “We didn’t know if the customer was upset and hung up on us, or if there was a technical issue that caused the call to drop. The problem was intermittent and we had no idea what caused it. We tossed it over the fence to Cloud Brigade. They had to be above our knowledge level to figure it out.”
There were a lot of links in the chain to look at as potential culprits—from the phone company, to the firewall, to Knoah’s network and the PBX. Digging into all the potential factors, Cloud Brigade discovered a feature in the firewall that played havoc with the phone system.
“It turns out that our SonicWall firewall has a security feature that shifts traffic from one port to another every few minutes to prevent a sort of straight line through the firewall. It’s like a port redirection, making traffic do a zigzag through the firewall,” says Barletta. “It’s actually a security feature but it was terribly disruptive for our telephony traffic because our phone system is expecting everything to be coming from one port. Now, by design, the firewall was shifting ports. When a shift happened, calls would drop. And it would happen every 15 minutes or such, so when calls were short, they were fine. If a call ran as long as 15 minutes, or happened during a ‘shift episode,’ it would get cut off.”
“We wouldn’t have figured that out ourselves, but Chris at Cloud Brigade dug into the minutiae of the firewall to discover that feature. We reconfigured the firewall and eliminated the problem,” says Barletta.
There have been other instances of baffling issues that Cloud Brigade solved for Knoah Solutions. For example, at times the A/B replication of the phone system has had problems. “We’ve asked ourselves, ‘Shouldn’t it do that? Why didn’t it do that?’ We turn it over to Chris and feel confident he’ll get it resolved.”
“We have an IT team in Las Vegas and one in India and our time zones are 12 hours apart. No matter what time of day or night we call Cloud Brigade, they are responsive.”
– Mike Barletta, IT Manager, Knoah Solutions
Technology
- Software: Asterisk, FreePBX, QueueMetrics, CentOS Linux, Corosync, Pacemaker
- Hardware: Supermicro servers
- Development Tools: Perl, PHP, Python
Services
- Technical Consultation – Recommend solutions based on business requirements.
- System Engineering – Built custom solutions where no commercial option existed.
- Systems Integration – Created custom integrations between the phone system and CRM products.
- Maintenance and Support – Providing ongoing training, maintenance, and support.