Keynote Systems released the results of its latest VoIP Service Provider Study, which evaluates how end users experience the reliability and audio clarity of leading voice service providers. The study compares the relative performance of AT&T landline service or POTS (plain old telephone service) with six broadband VoIP providers (AT&T CallVantage, EarthLink trueVoice, Lingo, Packet8, Verizon VoiceWing and Vonage) and two cable voice service providers (Time Warner Digital Phone and Comcast Digital Voice) for the month of October, 2008 by collecting data from calls made from residential locations in New York and San Francisco. Overall, reliability is improving slightly, however audio quality is significantly different from the previous study.

Results Overview:

Reliability 

Provider                     Reliability Points     Rank  
AT&T Landline                      996                1  
Time Warner Digital Phone          925                2  
Verizon VoiceWing                  872                3  

Audio Quality 

Provider                     Reliability Points     Rank  
Comcast Digital Voice              901                1  
Verizon VoiceWing                  609                2  
AT&T Landline                      506                3  

Read an abstract of Keynote’s VoIP study online:
 http://www.keynote.com/docs/kcr/Voice_W6_CIStudy.pdf 

With analysts predicting strong growth in the market, it is imperative for VoIP companies to perfect their offering now to capture new users and satisfy existing customers. Consumers and commercial customers will be looking to VoIP service as an opportunity to save money in this economy. The VoIP services with the best end user experience will be positioned to take advantage of current economic pressures.

VoIP Service Provider Study, Wave 6 Competitive Results:

The Wave 6 VoIP Service Provider study evaluated traditional landline service provider AT&T VoIP providers AT&T, CallVantage, Lingo, Packet8, EarthLink, trueVoice, Verizon, VoiceWing and Vonage; and cable voice services Comcast Digital Voice and Time Warner Digital Phone. Keynote analyzed the audio quality and reliability, key performance indicators that affect the end user experience across these VoIP providers, as well as traditional phone service for comparison.

Overall Best VoIP Service Providers

Keynote ranked VoIP Service Providers in two categories: Reliability and Audio Quality. The Service Availability, Call Completion, Average Answer Time, and Dropped Audio performance factors all contribute to the Reliability ranking. This high-level ranking identifies which voice service providers are providing the most reliable end user experience. The Audio Quality ranking is computed as an aggregate of the performance factors related to audio clarity and audio delay. This high-level ranking identifies which service providers are providing the most reliable and best sounding VoIP service. Each ranking is computed based on 1000 total points available as the best case, or winning result.

Reliability Overview

In Wave 6 of Keynote’s VoIP Service Provider Study, Time Warner’s Digital Phone maintains its second place ranking behind landline service, and the remaining providers have generally increased their scores as compared to the Wave 5 study.

Audio Quality Overview

In Wave 4 and again in Wave 5, Keynote reported that the landline service outperformed the competition when it comes to delivering great sounding call audio with minimal delay. In Wave 6, we see digital return to the top with Comcast and Verizon providing better audio quality than landline.

General Observations

• The best providers always deliver dial tone and connect the call to the number dialed in a timely fashion.
• Only one of the providers in the study failed to provide dial tone 99.9% of the time or better.
• All providers had very small percentages of calls with dropped audio, but only two providers had zero calls with dropped audio.
• One VoIP provider required two seconds more than any other voice provider to connect calls after dialing.
• Most providers had slightly more audio delay and slightly lower MOS in Wave 6 as compared to Wave 5.
• Eight of the nine providers in the study had a better call completion rate in Wave 6 than was evidenced in the Wave 5 results.