Introducing the new DxOMark Mobile test protocol

When we launched our original DxOMark Mobile test protocol in 2012, among the first smartphone cameras we tested were the Apple iPhone 4 and 5, the Samsung Galaxy S2 and S3, the Nokia 808, and the HTC 8X. Back in the day, those were all high-end devices with very capable cameras.

Apple’s iPhone 4 was the first smartphone we tested on DxOMark Mobile. Its 5 Megapixel camera featured a 1/3.2-inch sensor with a 1.75 µm pixel size and a f/2.8 aperture.

A lot has happened in the five years since then. Smartphone photography has pretty much eradicated the compact camera market segment, and the mobile sector has become the number one driver of innovation in imaging. The latest smartphone models come with camera technology and features that have slowly but surely brought our original test protocol to its limits.

One of the last devices that we tested using the old DxOMark protocol, the HTC U11, combines a large 12 Megapixel 1/2.55-inch sensor with a fast f/1.75 aperture, optical image stabilization and phase detection AF.

Advanced CMOS image sensor technology and more powerful chipsets allow for real-time multi-frame processing, resulting in substantial improvements in low-light performance.
On-sensor phase detection and laser-based time-of-flight sensors have drastically improved autofocus speed and precision, especially in low-light conditions.
Dual-camera setups have brought optical zooming capabilities and bokeh-like features that simulate on smartphone cameras the shallow depth-of-field of a fast lens on a DSLR. Other dual-cams combine the image data from an RGB sensor with the information captured by a secondary monochrome chip for improved detail, reduced noise levels, and increased dynamic range.

Apple’s iPhone 7 Plus was one of the first smartphones to offer optical zoom in a dual-cam setup:

To take these developments into account and to be prepared for the future, we have redesigned the DxOMark Mobile test protocol to include new and expanded outdoor and lab test scenes and evaluation methodologies. With the new protocol, we capture and analyze more than 1500 images and two hours of video for each device. Compared to the previous protocol, it provides the following new elements:

A new zoom sub-score that is based on extensive testing at multiple focal lengths
A new bokeh sub-score based on lab and outdoor testing
Low-light testing down to 1 Lux
Motion-based test scenes for more accurate evaluation of camera performance and processing techniques in real-world situations

Many smartphone components, such as displays and chipsets, have become commoditized in recent years, but camera quality is more than ever a key differentiator for manufacturers and a major decision criterion for smartphone buyers. The new DxOMark Mobile test protocol enables consumers to make informed buying decisions by evaluating the true camera performance of the latest mobile devices and by challenging OEM marketing claims.

You can find a detailed overview of the new DxOMark Mobile test protocol here. Click here for a comparison of the original and new protocols, looking at the scores of some of the best performing smartphone cameras. The smartphone reviews that followed the previous DxOMark Mobile protocol are still valid for evaluation purposes, but please keep in mind that you cannot compare their test results with those obtained with the new protocol.