A closer look at DXOMARK’s laptop protocol

Timbre describes a device’s ability to render the correct frequency response according to the use case and users’ expectations, taking into account bass, midrange, and treble frequencies, as well as the balance among them. Good tonal balance typically consists of an even distribution of these frequencies according to the reference audio track or original material. We evaluate tonal balance at different volumes depending on the use case. In addition, we look for unwanted resonances and notches in each of the frequency regions as well as for extensions at low- and high-end frequencies.

Dynamics covers a device’s ability to render loudness variations and to convey punch as well as clear attack and bass precision. Sharp musical notes and voice plosives sound blurry and imprecise with loose dynamics rendering, which can hinder the listening experience and voice intelligibility. This is also the case with movies and games, where action segments can easily feel sloppy with improper dynamics rendering. As dynamics information is mostly carried by the envelope of the signal, not only does the attack for a given sound need to be clearly defined for notes to be distinct from each other, but sustain also needs to be rendered accurately to convey the original musical feeling.

In addition, we also assess the signal-to-noise ratio (SNR) in capture evaluation, as it is of the highest importance for good voice intelligibility.

Spatial describes a device’s ability to render a virtual sound scene as realistically as possible. It includes perceived wideness and depth of the sound scene, left/right balance, and localizability of individual sources in a virtual sound field and their perceived distance. Good spatial conveys the feeling of immersion and makes for a better experience whether listening to music or watching movies.

We also evaluate capture directivity to assess the device’s ability to adapt the capture pattern to the test situation.

The volume attribute covers the loudness of both capture and playback (measured objectively), as well as the ability to render both quiet and loud sonic material without defects (evaluated both objectively and perceptually).

An artifact is any accidental or unwanted sound resulting from a device’s design or its tuning, although an artifact can also be caused by user interaction with the device, such as changing the volume level, play/pausing, typing on the keyboard, or simply handling it. Artifacts can also result from a device struggling to handle environmental constraints, such as wind noise during recording use cases.

We group artifacts into two main categories: temporal (e.g., pumping, clicks) and spectral (e.g., distortion, continuous noise, phasing).

From the end-user’s point of view, color rendering refers to how the device manages the hues of each particular color, either by exactly reproducing what’s coded in the file or by tweaking the results to achieve a given signature. For videos, we expect that devices will reproduce the artistic intent of the filmmaker as provided in the metadata. We evaluate the color performance for both SDR (Rec 709 color space) and HDR (BT-2020 color space) video content.

We evaluate minimum and maximum brightness to help us ascertain if a laptop can be used in low light and in bright, backlit environments.

We also evaluate a device’s brightness range, which gives crucial information about its readability under various kinds and levels of ambient lighting. A high maximum brightness allows a user to use the laptop in bright environments (outdoors, for example), and a low minimum brightness will ensure that user can set the brightness according to their preference in a dark environment.

We evaluate maximum contrast using a checkerboard pattern, which also lets us see blooming impacts display performance.

We evaluate the electro-optical transfer function (EOTF), which represents the rendering of details in dark tones, midtones, and highlights. It should be as close as possible to that of the target reference screen but should adapt to bright lighting conditions to ensure that the content is still enjoyable.

We evaluate the uniformity of the laptop display both at maximum and minimum brightness to assess any uniformity defects that would be noticeable to the end-user.

We use a spectrophotometer to evaluate spectral reflectance level on laptop displays when turned off. Additionally, we use a glossmeter to measure the reflectance profile — for example, how diffuse reflectance is. These two measurements are important indicators of laptop readability in bright lighting environments.

Click for more information about these measurements.

Exposure measures how well the camera adjusts to and captures the brightness of the subject and the background. It relates as much to the correct lighting level of the picture as to the resulting contrast. For this attribute, we also pay special attention to high dynamic range conditions, in which we check the ability of the camera to capture detail from the brightest to the darkest portions of a scene.

The color attribute is a measure of how faithfully the camera reproduces color under a variety of lighting conditions and how pleasing its color rendering is to viewers. As with exposure, good color is important to nearly everyone. Pictures of people benefit greatly from natural and pleasant skin-tone representation.

The texture attribute focuses on how well the camera can preserve small details. This has become especially important because camera vendors have introduced noise reduction techniques that sometimes lower the amount of detail or add motion blur. For some applications, such as videoconferencing in low-bandwidth network conditions, the preservation of tiny details is not essential. But users using their webcam in high-end videoconferencing applications with decent bandwidth will appreciate a good texture performance score.

Texture and noise are two sides of the same coin: improving one often leads to degrading the other. The noise attribute indicates the amount of noise in the overall camera experience. Noise comes from the light of the scene itself, but also from the sensor and the electronics of the camera. In low light, the amount of noise in an image increases rapidly. Some cameras increase the integration time, but poor stability or post-processing can produce images with blurred rendering or loss of texture. Image overprocessing for noise reduction also tends to decrease detail and smooth out the texture of the image.

The artifacts attribute quantifies image defects not covered by the other attributes, caused either by a camera’s lens, sensor, or in-camera processing. These can range from straight lines looking curved or strange multi-colored areas indicating failed demosaicing. In addition, lenses tend to be sharper at the center and less sharp at the edges, which we also measure as part of this sub-score. Other artifacts such as ghosts or halo effects can be a consequence of computational photography.

The focus attribute evaluates how well the camera keeps the subject in focus in varying light conditions and at multiple distances. Most laptops use a fixed-focus lens, though we expect to see autofocus cameras in laptops in the future. Our testing methodology applies to both fixed-focus and autofocus in all tested situations. When several people are at different distances from the camera, a lens design with a shallow depth of field implies that not all people will be in focus. We evaluate the camera’s ability to keep all faces sharp in such situations.

Read more about this measurement on this scientific paper: https://corp.dxomark.com/wp-content/uploads/2017/11/Dead_Leaves_Model_EI2010.pdf