English
中文
Display size and quality as well as camera features and performance have for a long time been—and still are—major criteria for consumer smartphone buying. However, with users recording and consuming more and more video and audio content on their mobile devices, the importance of audio quality has greatly increased in recent years.
People use smartphones to record videos of family and friends, to shoot selfie videos, and to film concerts and other events. On the consumption side of things, they listen to music and watch their own videos as well as purchased content on their devices. The variation in audio quality between devices for both recording and output is huge, however, and there has been little guidance and information available for consumers who care about audio quality.
This is why we are introducing DXOMARK Audio—to provide comprehensive, neutral, and reliable test data about smartphone audio recording and output quality to consumers and other interested parties. We have developed a set of protocols for mirroring the ways consumers use their smartphones to record and consume audio content, along with a testing and benchmarking methodology that measures results in a reliable and repeatable way.
(Please note that for now, we at DXOMARK are testing the playback performance only of a smartphone’s internal speakers, not its performance when coupled to headphones or external speakers; nor are we testing its performance when coupled to an external microphone or when using non-native recording applications.)
DXOMARK Audio testing and scoring
DXOMARK uses both objective and perceptual tests to evaluate playback and multimedia recording capabilities. Our objective tests rely on equipment such as spectrograms and sound-level meters to provide results. Our perceptual tests rely on the skills and experience that our sound experts have acquired after years of working in the fields of audio engineering and audio industrial design in a wide array of industries. These perceptual tests are no less scientific than our objective measurements, as our specific protocols ensure that any perceptual measurement is consistent over time, meaning that the same test carried out months later on the same device will achieve identical results.
We have broken down the different audio quality attributes into sub-attributes. We calculate the scores for all sub-attributes and combine them into attribute scores that feed into the overall score. Just as for our image quality tests, DXOMARK uses an open-scale scoring system, which means as the quality of both speakers and microphones improves, the scores will also increase. This not only allows us to rank contemporaneous devices, but also allows us to rank and compare future devices with older smartphones.
The DXOMARK Audio team and lab
DXOMARK has a dedicated team of audio engineers and experts whose sole responsibility is to test the audio capabilities of the latest smartphones. We have recruited these highly-trained specialists from the fields of electronic engineering, software engineering, and acoustic industrial design, and their experience and skills are reflected in DXOMARK’s thorough and wide-ranging audio tests.
These tests, which our engineers carry out for up to a week for a single smartphone, take place in our dedicated state-of-the-art Audio lab which includes two major test setups: the anechoic box and the listening room.
Our specially-equipped anechoic box completely absorbs sound waves and is isolated from outside noise. We place the device under test inside the box along with microphones and subsequently seal it. We then record the results using microphones and sensors. In addition to microphones and sensors, our box also has speakers that simulate various use case scenarios when measuring the device’s recording capabilities.
The listening room houses an array of speakers that are placed around a human subject or the device under test. Using microphones arranged in an identical layout, we recreate specific environments as well as replicate use cases typical for a smartphone. We record these environments, such as a busy street in the real world, and then faithfully recreate them in the listening room. We also use the room to compare the performance of two different smartphones to ensure the impartiality of our scoring system.
The DXOMARK Audio ranking
For our initial ranking, we have tested seven devices from five different brands—the Apple iPhones 11 Pro Max and XS Max, the Samsung Galaxy Note 10+ and S10+, the Honor 20 Pro, the Huawei Mate 20 X and the Sony Xperia 1. In the table below, you can see that the Huawei Mate 20 X’s 75 points put it into the top spot in the ranking by a margin of one point. The Mate 20 X features the largest dimensions, and unlike other tested smartphones, it has stereo microphones, leading to a score of 75 points and putting it into the top spot in the ranking by one point. At 74 and 71 points respectively the iPhone XS Max and 11 Pro Max occupy the podium positions – and interestingly enough, the newer device has a lower score than the older model. The iPhones are followed by the Samsung duo Note10+ and S10+ which are only separated by one point. The Honor 20 Pro at 53 points and the Sony Xperia X1 at 45 lag somewhat behind the competition.
Looking more closely at the Playback and Recording sub-scores, we can see that the Huawei beats its competitors in both categories. Most devices deliver a performance that is fairly balanced between Playback and Recording. The Sony is the only device that is clearly much better in one category than the other, coming in last for Playback by quite a margin, but doing comparatively well for Recording, where it comes fifth but isn’t that far off the best devices.
We’ll add new devices to the ranking soon, so keep an eye on DXOMARK.com for updates.
We hope you found this brief overview of our new Audio protocol useful and that it has given you some understanding of how we test and score smartphone microphones and speakers. For more details about audio testing and scoring, please read the following articles:
Also click through to see our DXOMARK Audio reviews and test results.
DXOMARK encourages its readers to share comments on the articles. To read or post comments, Disqus cookies are required. Change your Cookies Preferences and read more about our Comment Policy.