A) GBD generation

GBD are critical for accurate CGM. Morovic et al. [Reference Morovic and Luo17] propose an universal Segment Maxima algorithm to compute the GBD for a given gamut. However, benefiting from the linear gamut boundaries in the CIE-xyY space, we can simplify the algorithm as locating the most saturated colors in different luminance as the boundary points. Let $w(x_w, y_w)$ be the white point under a given illuminant condition, to a given color $c(x, y)$, we approximate the hue and the saturation of c, denoted $h_c$ and $\rho _c$, respectively, as

(1)\begin{equation} \begin{cases}h_c = {\rm atan}((y - y_w) / (x - x_w) ) \\ \rho_c = \sqrt{(x - x_w)^2 + (y - y_w)^2}. \end{cases}\end{equation}

Note that a color gamut is a 3D solid, and $c(x, y)$ may represent many colors located in $(x, y)$ with different luminance. Gamut boundary points are then obtained by,

(2)\begin{equation} b(h, l) = \arg \displaystyle \max_{\forall h, l; c \in \mathcal{C}} \rho_c, \end{equation}

where l is a given luminance, and $\mathcal {C}$ is the set of all possible colors of the gamut. Figure 2 shows the obtained Rec.2020 gamut boundaries sampled at several luminance with the peak luminance 4000 nits (normalized to 0.4). As shown in Fig. 2, only few GBD can well define the linear boundaries at a luminance layer l in the CIE-xyY space.

Fig. 2. An example of the sampled Rec.2020 gamut solid under the peak luminance 4000 nits (normalized to 0.4).

With (1) and (2), we can obtain the GBD of arbitrary RGB gamuts in the CIE-xyY space. Since the hue and the luminance are continuous, hue quantization and luminance sampling are necessary in GBD computations. The hue quantization level can be experimentally set to 1 to 2 degrees. The sampling rate of luminance is application-depended. The bit-depth of colors also greatly affects the quality of the GBD. Our simulations show that 12 or higher bit colors can generate satisfied GBD.

B) Computations of CMC

The Munsell color system adopts Value (lightness), Hue, and Chroma to present colors [Reference Cleland16]. The colors having the same Value and Hue but different Chroma form the constant hue loci at a given luminance layer. From the Munsell Renotations [Reference Newhall, Nickerson and Judd15], where 14 sets of Munsell colors are defined in different luminance layers, we interpolate the constant hue loci of different hues in arbitrary luminance layers. Figure 3 shows the examples of the obtained constant hue loci overlapped on the Rec.2020 gamut as $\Omega _S$, and the DCI-P3 gamut as $\Omega _T$ at different luminance layers.

Fig. 3. Examples of the constant hue loci at different luminance layers (green: Rec.2020; red: DCI-P3; cyan: constant hue loci). (a) l=0.06. (b) l=0.29. (c) l=0.75.

Although we can directly adopt the constant hue loci segments between $\Omega _S$ and $\Omega _T$ as the CMC, the segments are non-linear, and not convenient to either color-moving operators or CMC storage. We note that, as shown in Fig. 3, the constant hue loci near the green vertex have relative big curvatures, which may lead to relatively big errors if we linearize those loci. However, the MacAdam Ellipses [Reference MacAdam18] show that human vision system has relatively big tolerance of color differences in the region. The loci segments near the blue, the purple, and the red vertices have relatively small curvatures, and their length are also small. Although human vision system has relative small tolerance of color differences in those regions [Reference MacAdam18], linearizing those loci will lead to relatively small errors too.

After testing the perceptual color accuracy with many real-world images and test patterns, we conclude that the constant hue loci segments between $\Omega _S$ and $\Omega _T$ can be linearized without greatly degrading the final visual quality. This greatly decreases the LUT size, and simplifies the color-moving computations. In this paper, we adopt the linearized loci segments as the CMC. Figure 4(a) shows the examples of the CMC between the Rec.2020 and the DCI-P3 gamut at layer l=0.06. Note that the paths connecting the vertices of $\Omega _S$ and $\Omega _T$ do not follow the constant hue loci. A general rule of color moving is to move a vertex color of $\Omega _S$ to the corresponding vertex color of $\Omega _T$.

Fig. 4. Examples of the linearized constant hue loci and CMC redundancy removal (Rec.2020 to DCI-P3, l=0.06). (a) Linearized segments of the constant hue loci. (b) CMC prediction.

Analysis of the CMC shows that there are great redundancy existing among the CMC in the same luminance layer. For example, as shown in Fig. 4(b), the CMC of hue 5G can be well represented by its neighbors hue 2.5G and 7.5G with line interpolation, thus the CMC of hue 5G is redundant, and can be removed from the CMC list. By removing the redundant CMC, we further decrease the LUT size. Also, our simulations show that it is not necessary to save many layers' CMC. Few key luminance layers are enough to interpolate visually pleased mapped colors. Our simulations show that 6–15 key luminance layers are good enough for most real-world contents. This further decreases the total number of the CMC saved in the LUT size, thus decreases the hardware costs.

A) Definition of $\Omega _Z$

Most serious artifacts in GC-based CGM are caused by directly mapping the out-of-gamut colors to the boundaries of $\Omega _T$, thus details and smooth transitions of those colors may be lost. An effective solution of the issue is to move the out-of-gamut colors proportionally inside $\Omega _T$ according to their original positions in $\Omega _S$. In such a way, we can preserve parts of the original color variations in the mapped colors. Furthermore, we also need to appropriately move small parts of the colors that are inside $\Omega _T$ but near its boundaries, thus we can maintain the smooth transition between the moved colors and unchanged colors. The TPZ $\Omega _Z$ is designed for this propose. The size of $\Omega _Z$ determines the limits that the out-of-$\Omega _Z$ colors can be moved inside $\Omega _T$. The bigger the $\Omega _Z$ is, the smaller the space between the boundaries of $\Omega _T$ and $\Omega _Z$, and the less the details in out-of-$\Omega _Z$ colors are protected, but the more saturation are preserved. The smaller the $\Omega _Z$ is, the bigger the space between the boundaries of $\Omega _T$ and $\Omega _Z$ is, the more details and color variations are protected, but the more saturation is sacrificed. Therefore, the size of $\Omega _Z$ should achieve a reasonable compromise between detail protection and saturation preserving.

In this paper, we define the $\Omega _Z$ along the extensions of the vertex paths between $\Omega _S$ and $\Omega _T$. Let $v_s$ and $v_t$ be a pair of corresponding vertices of $\Omega _S$ and $\Omega _T$, respectively. The corresponding vertex of $\Omega _Z$, denoted $v_z$, is such a point that satisfies (3),

(3)\begin{equation} \vert v_t v_z\vert = \alpha \vert v_s v_t\vert, \end{equation}

where α is a size control factor, and can be experimentally set as $\alpha \in [0, 0.4]$. When $\alpha = 0$, $v_z$ becomes $v_t$, namely, $\Omega _Z$ achieves its maxima and is equal to $\Omega _T$. In such a case, the proposed algorithm performs similarly to the GC-based CGM. As α increases, $\Omega _Z$ becomes smaller, and more details are preserved, but more saturation is sacrificed. Depending on concrete applications, α can be either a constant for all gamut vertices, or a victor where each element corresponds a gamut vertex pair. We recommend to set α as a vector adaptively to the contents statisticsFootnote ¹ for preserving more saturation during CGM. Figure 5 shows the examples of the $\Omega _Z$ defined at different luminance and overlapped by the CMC we obtained in Section III.

Fig. 5. Examples of the $\Omega _Z$ overlapped by CMC at different luminance layers (green: $\Omega _S$, red: $\Omega _T$, cyan: $\Omega _Z$, and “VTX” means vertex). (a) l=0.06. (b) l=0.415. (c) $l = 0.74$.

Once $\Omega _Z$ is determined, we extend the CMC, which are between the boundaries of $\Omega _S$ and $\Omega _T$, to the boundaries of $\Omega _Z$. Such a CMC path consists of three points: (1) the start point on a boundary of $\Omega _S$, denoted s; (2) the inner point on the corresponding boundary of $\Omega _T$, denoted t; and (3) the end point on the corresponding boundary of $\Omega _Z$, denoted z, and can be written as $\overline {stz}$. Note that the size of $\Omega _Z$ should not be too small. First, a very small size $\Omega _Z$ may sacrifice too much saturation. Second, a very small size $\Omega _Z$ will lead to bigger spaces between the boundaries of $\Omega _T$ and $\Omega _Z$. Since the real constant hue loci are non-linear, the big spaces may lead to relatively high linearization errors between the linearized CMC and the real constant hue loci. Our simulations show that controlling α of (3) in $[0.1, 0.4]$ can achieve good compromise between details protection and saturation preserving.

In practice, we need not update $\Omega _Z$ frequently. It can be either computed only one time at the stage of device initialization, or updated under the controls from the received ST.2094-40 metadata. For example, we may only update $\Omega _Z$ when the statistics of contents has great changes. Since (3) only contains linear and simple operators, updating $\Omega _Z$ is real-time and hardware friendly.

B) Constant hue loci estimation

Constant hue loci estimation for arbitrary out-of-$\Omega _Z$ colors is crucial to correctly move the colors in the perceptual non-uniform CIE-xyY space without introducing visible hue distortions. In this section, we firstly introduce the conception of hue sector. In a given luminance layer, we define the polygon formed by two neighboring CMC paths and two corresponding boundaries of $\Omega _S$ and $\Omega _Z$ as a hue sector. Figure 6 shows an example of the hue sector consisting of the CMC of hue 2.5G and 10G, and the boundary segments of $\Omega _S$ and $\Omega _Z$ between the two CMC paths in layer l = 0.06, i.e., $[s_1, z_1, z_2, s_2]$.

Fig. 6. An example of the hue sectors and the estimated constant hue loci of an arbitrary color c at layer l=0.06.

To a given color c that is outside $\Omega _Z$, we firstly determine its hue sector. With the example shown in Fig. 6, assume that c is in the hue sector consisting of two neighboring CMC paths $\overline { s_1 t_1 z_1}$ and $\overline {s_2 t_2 z_2}$. We propose to estimate the constant hue locus of c from $\overline {s_1 z_1}$ and $\overline {s_2 z_2}$ with following two rules: (1) if c is on one of $\overline {s_1 z_1}$ and $\overline {s_2 z_2}$, its constant hue locus is the CMC path; and (2) when c is between $\overline {s_1 z_1}$ and $\overline {s_2 z_2}$, its constant hue locus should gradually change from $\overline {s_1 z_1}$ to $\overline {s_2 z_2}$ depending on its position. According to the two rules, we propose an efficient but reliable anchor point-based strategy to estimate the constant hue locus that passes an arbitrary out-of-$\Omega _Z$ color c. Let A be the intersection of $\overline {s_1 z_1}$ and $\overline {s_2 z_2}$, the vector from c to A, namely $\overrightarrow {cA}$, indicates the direction of the constant hue locus of c, and point A is thus called the anchor. The intersections of line $\overline {cA}$ and the corresponding boundaries of $\Omega _S$, $\Omega _T$, and $\Omega _Z$ form the constant hue locus of c, denoted $\overline {s_r t_r z_r}$. Figure 6 shows the details of this strategy.

C) Color moving along constant hue loci

The essential of the color-moving operator in the proposed algorithm is to compute an appropriate position, denoted $c'(x', y')$, inside $\Omega _T$ for a given out-of-$\Omega _Z$ color c on its constant hue locus $\overline {s_r t_r z_r}$ (see Fig. 6). We propose to compute $c'$ by mapping $\overline {s_r z_r}$ to $\overline {t_r z_r}$, and keep the relative position of $c'$ on $\overline {t_r z_r}$ unchanged as c is on $\overline {s_r z_r}$. The relative position of c on $\overline {s_r z_r}$, denoted γ, is defined as

(4)\begin{equation} \gamma = \frac{\vert \overline{c z_r}\vert }{\vert \overline{s_r z_r}\vert }. \end{equation}

To keep the γ unchanged after moving c to $c'$ along its constant hue locus $\overline {s_r z_r}$, we have

(5)\begin{equation} \begin{cases} x' = x_{z_r} + \gamma ( x_{t_r} - x_{z_r} ) \\ y' = y_{z_r} + \gamma ( y_{t_r} - y_{z_r} ), \end{cases}\end{equation}

where $(x_{t_r}, y_{t_r})$ and $(x_{z_r}, y_{z_r})$ are the coordinates of $t_r$ and $z_r$, respectively (see the demonstration in Fig. 6).

In practice, due to the cost limits, the LUT only contains the CMC of few key luminance layers. An input color c may have no corresponding luminance layer in the LUT for directly carrying out color moving with (5). We propose a luminance-adaptive interpolation strategy to solve the problem. Let $Y_c$ be the luminance of c. We firstly locate two available neighboring luminance layers, denoted $Y_L$ and $Y_H$, respectively, in the LUT, and $Y_L \le Y_c < Y_H$. Thus the target colors of c in layers $Y_L$ and $Y_H$, denoted $c'_L$ and $c'_H$, respectively, can be determined from (5). The output $c'$ is then interpolated as

(6)\begin{equation} \begin{cases} x' = x'_L + \beta ( x'_H - x'_L ) \\ y' = y'_L + \beta ( y'_H - y'_L ), \end{cases}\end{equation}

where $(x'_L, y'_L)$ and $(x'_H, y'_H)$ are the coordinates of $c'_L$ and $c'_H$ in layers $Y_L$ and $Y_H$, respectively; and

(7)\begin{equation} \beta = \frac{Y_c - Y_L}{Y_H - Y_L}. \end{equation}

As can be seen from (6) and (7), when $Y_c \approx Y_L$, $\beta \approx 0$, and $c'$ is approximately equal to $c'_L$; if $Y_c \approx Y_H$, $\beta \approx 1$, and $c'$ is approximately equal to $c'_H$. Otherwise, $c'$ smoothly varies along the cross-layer path $\overline {c'_L c'_H}$. This guarantees the naturalness and smoothness of the obtained colors in $\Omega _T$. Also, as shown in (3)–(7), all computations in the proposed algorithm do not have complex computation such as exponential or Trigonometric operators. This is suitable for hardware implementations.

A) Test images and simulation environment

Considering the popularity of the Rec.2020 and the DCI-P3 gamut in HDR contents distribution eco-systems, we focus our tests on the CGM between the two gamuts. HDR contents creators may only adopt Rec.2020 as a color container, e.g., the DCI-P3 contents encapsulated in the Rec.2020 gamut. The contents are not suitable for testing CGM algorithms due to lack of the out-of-gamut colors. In this paper, we use the famous LUMA HDRV HDR contents database [Reference Froehlich, Grandinetti, Eberhardt, Walter, Schilling, Brendel, Sampat, Tezaur, Battiato and Fowler19] as the test images. The contents are mastered in real Rec.2020 gamut for the purposes of HDR technique developing and testing, including tone-mapping, CGM, etc. Especially, the contents contain abundant details in the colors near the Rec.2020 gamut boundaries. This is very challenging to CGM algorithms. All the 14 videos containing over 14300 real Rec.2020 frames are tested in our simulations to comprehensively evaluate the proposed algorithm. Note that the test images we used in algorithm developing are independent of the test images of performance evaluations in this section.

All the test CGM algorithms are implemented with ${\rm C}/{\rm C}++$, and run on a 8-core 3.2M Hz CPU workstation with the Linux operation system. Experimental results are reviewed on a Samsung 9800 UHDTV under ${\rm HDR}+$ mode, which supports the PQ electronic-optical transform function (EOTF) [20].Footnote ²

B) Parameters used in simulations

We adopt the Munsell Renotations published by The Munsell Science Laboratory of Rochester Institute of TechnologyFootnote ³ to compute the CMC, and experimentally set α in (3) to $[0.3, 0.35, 0.3]$ for the red, the green, and the blue vertices. Note that as luminance increases, the number of the gamut vertices changes too. The control factors of the new vertices can be interpolated from the three basic factors. Also, instead of directly saving all the coordinates of the CMC into the LUT, we pre-compute the direction vectors of the CMC (see Section III-B) at the quantized positions in the CIE-xyY space. Thus, we can estimate the constant hue loci of arbitrary out-of-$\Omega _Z$ colors from the direction vectors at quantized positions, and avoid going over all hue sectors for locating the hue sector of a given color. This further decreases the hardware costs and increases the efficiency. Our researches show that a $32 \times 32 \times 6$ LUT performs well in all simulations, where 32 is the number of the quantized positions along the x- and the y-axes, and 6 is the number of the sampled luminance layers.

Following the parameters given in [Reference Azimi, Bronner and Pourazad10], we set the inner gamut determining factor of the Azimi CGM to 0.5. The GC-CGM only need the Rec.2020 to DCI-P3 color-space conversion matrix.