Perceptual Image Quality Assessment Technique for Color Images based on HSI Colour Space

Received Feb 18, 2020 Revised Dec 10, 2020 Accepted Dec 11, 2020 A new full-reference Perceptual Image Quality Assessment (PIQA) technique based on Hue-Saturation-Intensity (HSI) transformation method for colour images is offered in this work. Fundamentally, it is combined with three approaches (colour transformation, histogram processing and Human Visual System (HVS) based weighting) and it uses the Discrete Cosine Transform (DCT) domain. The PIQA is composed of four phases. At the first phase, a colour image is transformed using the HSI because an important characteristic of eyes reaction to intensity of light and colour knowledge is used for quality assessment processes. All channels' DCT coefficients have been calculated at the second phase, because many specifications cannot be assessed in the spatial domain. At the third phase, histogram based quality assessment results are obtained by using histogram of each channel (Hue, Saturation and Intensity). These results are weighted for obtaining perceptual quality result taking into account the HVS specifications at the last phase because the human eye has different sensitivities to the intensity and the others (i.e., hue and saturation). Experimental outcomes about feasibility of the PIQA on test images under different deteriorations, both sensible by the HVS and with the same Peak Signal to Noise Ratio (PSNR) result are offered. The PIQA shows better performance in comparison to state-of-the-art techniques.


INTRODUCTION
Today, digital images are commonly used with application areas ranging from the entertainment industry to the daily life. The Internet, via World Wide Web (www) browsers, thanks to capabilities in computer power and network bandwidth has brought all around the world into offices and homes. In addition, digital images have been frequently used on social media platforms (i.e., instagram, twitter, facebook, etc.) [1,2].
After digital images are acquired, they are generally deteriorated for some signal processing reasons (encoding, compression, acquisition, transmission, steganalysis, manipulations, etc.). From this perspective, quality measurement of distorted or deteriorated images is very important issue at this point. Moreover, it is important that the image quality assessment (IQA) result used for which purpose (forensic or medical issues, human vision system (HVS) based evaluation, etc.) [3]. The amount of assessed deterioration in an image can have dissimilar comments with respect to various practices and defines whether the given image has asked the quality amount to be valuable for implementation [4]. This means that it is very crucial to present implementation oriented IQA techniques.
Basically, there are two kinds of IQA techniques in the literature: No-Reference Image Quality Assessment (NRIQA) techniques and Full-Reference Image Quality Assessment (FRIQA) techniques. The NRIQA techniques are known as blind-IQA techniques as there is no way for comparison in between the original visual material and other processed version because original image is not available. Many of the NRIQA algorithms in the literature don't utilize natural image modelling but these algorithms assume that noise type effecting to the image quality is known. Some of them consider image blurring or JPEG compression artefacts by exploring the features of the artefact in different domains [5][6]. These NRIQA techniques are designed to identify special image degradation styles and calculate their occurrence from detailed properties of the categorized artefacts. It can be easily said that the most them can be categorized as distortion-free techniques since they can handle only once or some special distortions [7,8].
In FRIQA techniques, direct assessment in between the original image and its deteriorated model is performed. There is a restriction to the feasibility of FRIQA techniques as the original image is necessary to perform the computation. The basic and well-known FRIQA metrics are Mean Square Error (MSE) and Peak Signal to Noise Ratio (PSNR). These widely-used approaches are generally used in signal processing applications; however the outcomes of these approaches do not relevant enough with the perception of human vision system (HVS) [9]. On the other hand, some other algorithms are used for measuring images, such as PSNR-HVS-M [10], SSIM (Structural SIMilarity) [11], MS-SSIM (MultiScale Structural SIMilarity) [12], UQI (Universal Quality Index) [13], VIF (Visual Information Fidelity) [14] and so on. All of these methods have some problems in terms of reliable HVS based quality assessment. These problems are detailed following sections.
In addition to the fundamental IQA techniques presented above, there is one more inter-type of IQA technique: Reduced-Reference (RRIQA). In fact, it is kind of FRIQA technique. In RRIQA philosophy, only limited information about the original image is accessible while measuring of a digital image under assessment [15][16][17]. Hence, RRIQA techniques lie in between the NRIQA and FRIQA techniques in terms of accessible information about the original image. All of these just extract some of the specifications from both of them (original one and its deteriorated version) and measures quality of the image corresponding to these specifications, which are the features of all the specifications in the images. Obtained features generally define the image content or deterioration based specifications [18][19][20].
The presented perceptual IQA technique (PIQA) based on HSI colour model offers a different approach from well-known FRIQA techniques. Basically, the presented technique is combined with three layers (colour transformation, histogram processing and human visual system (HVS) based weighting) and it uses the Discrete Cosine Transform (DCT) domain. Thanks to DCT coefficients, high and low frequency values can be easily decomposed and then low frequency values are used for measurement processes because high frequency values can be ignored while image quality calculation in terms of the HVS.
The paper is arranged as follows. HSI colour model, the DCT and histograms of images are introduced in Section 2. The phases of the presented technique are detailed in Section 3. Section 4 presents experimental IQA results and comparisons. Conclusions are provided in Section 5.

SOME FUNDAMENTAL THEMES
HSI colour space, the DCT and image histogram have been used for the presented HVS based PIQA technique. Their details and usage fundamentals have been presented below.

HSI colour transformation
In the RGB (red, green, blue) colour space, each channel appears in its main spectral parts of colour image. It is fundamentally based on a Cartesian coordinate system. The term full-colour image is used as a 24bits (8 bits for each colour channel) RGB colour image (Figure 1a). This model is suitable for colour presentation (as in image captured by any camera or mobile device in a display system) [21].
On the other hand, when human observes a colour object, it is described by its hue, saturation and brightness. Hue (H) is a colour feature that defines a pure colour. Saturation (S) presents a measurement of the grades to which a pure color is subtilized by white light. Brightness is a subjective descriptor and it objectifies the achromatic concept of intensity (I) and it is a key feature in defining colour perception [1,21]. This is called as HSI colour model (Figure 1b). The HSI decouples the intensity part from the color carrying information (Hue, Saturation) in a digital color image [21]. From this point of view, it can be said that the HSI model is an important transformation method for improving an IQA technique based on the HVS. The HSI colour space model has been used for the presented PIQA technique because of this important feature.
Given an image in RGB colour type, H, S and I channels can be calculated using the Eq. (1), Eq. (3) and Eq. (4), respectively.
Well-known Lena images (in RGB form), its distorted (blurring) version and their Hue, Saturation and Intensity channels can be seen in Figure 2. In addition, deteriorations or differences can be easily sensed by the HVS on all channels, too.
It is assumed that the RGB values are normalized to the range between 0 and 1. Angle Θ is calculated with respect to the red axis of the HSI. Hue can be normalized to the range between 0 and 1 dividing by 360° all values resulting from Eq. (1). Other parts (i.e., S and I) are in this range if the given RGB values are in interval [0, 1].
The HSI colour space gives us an opportunity to describe colours in terms of more readily understandable. The intensity referred is the brightness and the hue is what we normally think of as "colour". The saturation is a measure of how much white is in a colour; for example, pink is red with more white, so it is less saturated then a pure red. People can concern to this methodology of defining colour, for example "a deep, bright orange" would have a large intensity ("bright"), a hue of ("orange") and a high value of saturation ("deep"). People can imagine a colour in their minds, but if the colour is defined in terms of its RGB colour space (for example, R=240, G=100, B=25), most people would have no idea how this colours view. Thus, the HSI colour model has been improved based on relating to the HVS [1].
A significant specification of the HVS is its distinct reaction to the intensity of the light and colours thoroughly concerned to the natural specifications of the human eye [22][23][24], which are used and reflected into the presented technique with varied percentages.

Discrete cosine transform
One of the most famous transformation methods is Discrete Cosine Transform (DCT) and is generally utilized for image compression. The DCT like Fourier transform uses sinusoidal functions. The main distinction is that the DCT based functions are not complex; it uses only cosine functions not sine functions. The 2D-DCT equation for N×N image is given by

Digital image histogram
The histogram of an image with intensity levels in the range [0, where v k is the kth intensity value and n k is the number of pixels in the image with intensity v k . It is general implementation to normalize a histogram by dividing each of its parts by all number of pixels in the image, defined by the product RC, where, as usual, R and C are row and column dimensions of the image. Therefore, a normalized histogram is written by p(v k )=n k /RC, for k= 0, 1, 2, … , L-1. p(v k ) is an estimate of the probability of occurrence of intensity grade v k in an image. The sum of all parts of a normalized histogram is equal to 1 [21].
Not only image pixel values but also whole number clusters can be depicted with a histogram. From this general concept, the first quarter part of the DCT coefficients of each HSI channel is used to obtain histograms in this presented study. This approach is an important aspect of the PIQA. Although, it is thought to be image qualities are the same considering the PSNR results (i.e., 27.67 dB), it can be viewed that this is not really true considering the histograms depicted after DCT ( Figure 5). H and S histograms have been obtained after normalization processes because H and S channels can be contain floating point numbers which are out of range. Thus, normalization process has been realized to adjust the values between [0,255].
Dissimilarities between the original image's histogram and its deteriorated version's histogram are given in Figure 5. Thus, this situation has been main motivation to develop the PIQA technique presented in this study.

THE PROPOSED PERCEPTUAL IMAGE QUALITY ASSESSMENT TECHNIQUE (PIQA)
The developed PIQA technique, with the steps of its four basic phases, i.e. the HSI color space, the DCT based Histograms computation for all HSI components, obtaining histograms of their first quarter parts' (H and  H ) and weighting of these considering the human perception are detailed. The basic flow of the PIQA method is given in Figure 6. The HVS based weighting PIQA Result Figure 6. Block scheme of the presented PIQA technique

Steps of the proposed PIQA technique
The presented PIQA technique's assessment result is obtained in four phases as depicted above. Each phase is listed and clarified below: • Phase 1: As it is previously described, intensity information and color information should be separated from the color image because human eye response is different for each of them [23,24,25]. Thus, HSI transformation equations given in Eq.
The Δ is a difference vector between test image and its deteriorated histogram version. Total alteration (Δ TA ) between them is achieved by summation of all indices of the Δ (Eq. 9) and 0 ≤ Δ ≤ 2 × R × C. In this equation, R and C refers row and column numbers, respectively.
Then, Δ TA F varying between 0 (worst) and 1 (best) is obtained using Δ TA as given in Eq. (10) [26].  The fovea part of the human eye contains cone cells and rod cells. The rod cells are responsible for vision at especially low light levels. The rod cells don't ensure color vision and have no effect on spatial perception [4,27,28]. On the contrary, the cone cells are responsible for vision at higher light levels and spatial perception. Rod cells' density is 200,000 rods/mm 2 (120 million) and cone cells' density is 150,000 cones/mm 2 (7 million) [21,29,30]. The corollary of this case, the proportion of the Rods (R P ) and the proportion of the Cones (C P ) can be easily obtained as seen below: C P = 7×10 6 / (120×10 6 + 7×10 6 ) = 0.0551, (12) R P = 120×10 6 / (120×10 6 + 7×10 6 ) = 0.9449.
(13) At the end of the all processes, the PIQA result can be calculated using Eq. (14).
Thanks to C P and R P , the HVS specifications are taken into account and more suitable results with human perception can be clearly obtained as detailed in the next section.

EXPERIMENTAL RESULTS
Generally used Lena reference image and its six deteriorated samples have been used for experimental applications and performance evaluations at the first step. It is distorted with well-known deterioration methods to yield the same Peak Signal to Noise Ratio (PSNR) values relative to the original Lena image.
The PIQA and the PSNR values for all of the images (512×512×3) are listed in Figure 8. It can be said that the PSNR results of the Figure 8b, 8c, 8d, 8e, 8f and 8g are 27.67 dB. This means that these deteriorations affected in the same way to the original test image contrary to the fact based on at least basic perceptual evaluation. In addition, it can be said that classical PSNR measure cannot differentiate all probable diversities between test image and its deteriorated versions [22]. But the proposed PIQA technique can easily differentiate and sort (i.e, from 0.4389 to 0.8537) them. This means that Figure 8g (blurred version) is worse than Figure 8a (sharpen) in terms of the HVS.
Considering the other IQA techniques mentioned above, it can be said that their results can differentiate to the images like the presented PIQA technique, with respect to the PSNR. But, the PSNR-HVS-M result implies that Figure 8d is better than Figure 8b. In addition, the MS-SSIM results are different from each other but according to this technique's results, Fig. 8g has better quality than Figure 8c, Figure 8d and Figure 8f. This means that it cannot sort to distorted images according to the HVS, too. On the other hand, the SSIM and UQI quality assessment values verified to be false, as they indicate that the image quality for Figure  8g is better than that of Figure 8c. Thus, it can be said that these results are not compatible with the HVS, too.    The calculation or computation speed is also a significant criterion in the most applications, such as the IQA while working on image data bases [4]. The experimental studies have been realized on Intel® Core i7 5500 CPU running at 2.4 GHz with 8 GB RAM, and the PIQA were implemented in Matlab®. As seen in Table 2, the PIQA is about 25 times faster than the PSNR-HVS-M and better than the other techniques.
In addition to some valuable conclusions mentioned above, 24 test images downloaded from [31] shown in Figure 11 have been used for other applications in order to measure the PIQA technique. All images were coded by Least Significant Bit (LSB) steganography algorithm (1 bit per pixel). Thus, sensitivity and the capability of differentiation of the IQA techniques have been measured, too.  The PSNR results are the same (i.e., 57.67 dB) for all images as seen in Table 3. In addition, the MS-SSIM and the SSIM results are the same. This means that these techniques cannot differentiate images because the quality results are the same. Moreover, the PSNR-HVS-M and the UQI methods generate repeatable and close IQA results. But, the presented PIQA technique can differentiate in terms of visual effects of low distortions on images. The PIQA results are between 0.4830 and 0.9740 as seen in Table 3.  Figure 11. SCISE images used for additional results.

CONCLUSIONS
Well-known IQA techniques play a significant role in almost all aspects of image processing algorithms. The HVS based technique and algorithm improvements needs are still continue because commonly used IQA techniques cannot properly differentiate the images.
There are two types of ciliary receptor, rod cells and cone cells in the human eye. Rod cells are sensitive to the light and the other one is sensitive to the colors. This means that if you want to get HVS based IQA result, a color image should be separated light and color parts.
In addition, many quality specifications cannot be calculated in time space, but this fact is different in the frequency space. The PIQA uses the color IQA philosophy and the histogram based IQA methodology with the help of the DCT. As a result of these fundamentals, the HSI color model based full-reference perceptual IQA (PIQA) technique is offered in this presented study.
IQA results' ranges are between 0 and 1 as the most IQA techniques. 1 means that top IQA result, 0 means that the worst IQA result for the PIQA. Experimental studies about the reliability of the PIQA on a wellknown test image under different deterioration types, both perceivable by the HVS and with the same PSNR value are given. On the contrary, the PIQA results are more suitable than the others in terms of the HVS and they can be calculated 1.46% to 2560% faster than its counterparts' results. Moreover, the PIQA can easily differentiate that impact of deteriorations on colour images. MATLAB ® implementation files of the proposed PIQA technique are available online at: shorturl.at/rvAB0