Abstract

Subjective listening tests were performed in order to evaluate the impact of transmission over an IP network on the quality of two “well-known” audio codecs. Those two codecs were WM9 and Helix 9 (Real), tested at two different modes and bit-rates. A dedicated network simulator was used in collaboration with the IRISA institute (Rennes 1 university), who developed this tool. This network simulator is accurately described in the paper. The network perturbations considered were packet losses (the parameters used were the loss rate and the mean burst size), delay and jitter. All these four parameters were adjusted independently allowing for different combinations of network parameter values. Many configurations were generated and a large amount of time was spent in order to listen carefully to the different impact on the overall quality of the codecs. Eventually, six configurations that yielded six distinctly different quality levels were kept to run the quality tests. The kept parameters values are commonly found in actual IP networks. As the goal of the audio tests was to assess the quality, the subjective audio test methodology used was the one known as MUSHRA. That stands for MUlti Stimuli with Hidden Reference and Anchor points. This is a method dedicated to the assessment of intermediate quality. It has been recommended at the ITU-R under the name BS.1534 [1]. An important feature of this method is the inclusion of the hidden reference and specific signals as anchor points along the quality scale. The objective of the test was to observe the “quality behaviour” of the two mentioned codecs at two different bit rates in a simulated network environment. Consequently, the anchor points were the two codecs at a given bit rate without any simulated network, and the full band reference. Results presented in the paper are of interest not only because they show the specific weakness of the two codecs when faced with network perturbations but also because they show a specific use of the MUSHRA method, which makes these kinds of tests possible.