• General call quality (not including “echo”) • Signaling and connection issues • “Echo” • Computer Telephony Integration (CTI) Issues
Source: JDSU ( www.jdsu.com), from an internal, two-year survey of its VoIP-analysis customers. Results reflect “the type of VoIP problems experienced” from 720 respondents in 80 countries (including enterprise users, service providers, and system integrators).
Figure 2: VoIP Problems
what the caller at either end hears. Indeed, it is the endpoint’s access to, and egress from, the backbone network where most congestion–resulting in lost packets and added delay–occurs. Users should ask whether the passive monitor uses real-time transport control protocol (RTCP) reports in its quality assessments, which we encourage. RTCP reports, issued by VoIP endpoints, generally provide accurate, end-to-end packet loss and latency statistics.
Active testing is not without its faults.
Indeed, the MOS ratings of different vendors’ equipment are based on several different PESQ algorithms, or they may not be based on a PESQ “standard” at all. MierConsulting has seen MOS ratings between different vendors’ active test equipment vary by as much as half a MOS point over the same VoIP connections.
One more note on PESQ: Some vendors claim that their PESQ measurements now take into account characteristics of network delay, which is called “conversational” PESQ. Some PESQs, including PESQ-LQ (for listener quality), measure before and after voice files but don’t factor in temporal impairments like latency. We look forward to assessing these newer enhancements in the field, and users may want to be on the lookout for products touting conversational PESQ.
What’s Hot?
Beyond the voice-quality issues already discussed, there’s a lot more happening in the industry. The respondents underscore a few other new capabilities in their latest and greatest VoIP-analysis tools. Here are a few of them:
• RTCP XR support. Several vendors tout the fact that their products’ can now also process the “extended reports” extension of RTCP. XR adds considerable data of use in assessing link and connection quality. It is clear that vendors of IP phones and other endpoints are adding RTCP XR support, so looking for this support in analysis tools should yield tangible benefits soon.
• Toward a video quality standard. Video over IP is coming.
When and to what extent remains to be seen, but some vendors are focusing more on video quality monitoring. Brix, ClearSight, and others, for example, have developed their own video-stream quality-measurement metrics in the absence of an accepted industry standard.
• Greater capacities. A key in scalable VoIP monitoring is the ability to monitor more and more concurrent calls. PC or laptop-based software programs are limited in this regard, and so some vendors–including JDSU and Agilent–are touting the increased capacities their appliances can handle. Both vendors’ units can monitor thousands of concurrent calls and accurately track all VoIP traffic traversing bi-directional gigabit backbone links.
• Prices are still fairly high. Prices are, however, trending downward. A starting configuration price for many of the products on the table is $20,000 and could easily reach or exceed $100,000 in a typical configuration. Shopping for the right product and the best value, then, becomes paramount.
Survey Says…
All things considered, what is the most reliable VoIP call-quality assessment metric? That’s one of the questions we asked two dozen VoIP measurement and monitoring vendors in the survey. We qualified “most reliable” to mean the metric, algorithm, or technique “most likely to align with human” assessment. Here’s what they said:
• R factor: 35 percent.
• Human MOS panels: 22 percent.
• PESQ: 13 percent.
• VQmon: 13 percent. (VQmon is a proprietary extension to R factor from Telchemy, which is licensed by many vendors, including several of the respondents.)
• MOS, unqualified: 9 percent.
• Other, including a “combination” of techniques: 9 percent.
How much can different vendors’ products vary in their VoIP quality measurement ratings, given the same VoIP calls and connections? Most respondents indicated that there is variability, but only a few actually quantified it.
“We have seen ratings for the exact same VoIP call rated as much as 35 to 40 percent apart,” says J. Scott Haugdahl, Chief Technology Officer of WildPackets, Inc. “The ITU E-model (R factor) standard has many options that may or may not be implemented by every vendor. And the variables that are used–echo, for instance–may not be consistently measured.”
Echo, it turns out, is a major bugaboo affecting VoIP implementers. Figure 2 shows a breakdown of actual VoIP problems encountered, based on responses from 720 worldwide users of JDSU’s VoIP test and monitoring gear, assembled over two years. These include enterprise network managers, system integrators, and service providers.
Fully one quarter of problems reported by VoIP implementers are collectively summarized as “echo.” This is likely the result of a phenomenon called “aggravated echo.” Echo for VoIP calls only occurs on calls that pass through gateways to and from the PSTN. A little bit of echo, where the echo reflection is, say, 20 or 30 milliseconds delayed, is not uncommon with PSTN calls. But this is below the threshold that people can detect. However, the
References:
Archives