HISTOGRAM MANIPULATION
Time and time again over the past few blogs we have seen that a lot of image information can be found and subsequently utilized by generating its histogram. As we will see in the next few examples, we can apply simple techniques to manipulate the histogram to increase the contrast of an image, improve its quality, and mimic other imaging systems such as the eye.
To start, let us consider the following image.
 |
| Figure 1 . Test image 1 |
We see that the image appears very dark and depending on your monitor screen resolution, a lot of information appears to be lost in the shadows. Is there anything we can do to increase the brightness or contrast of the image? Let's take a look at it's normalized histogram (the importance of it being "normalized" against the number of pixels will be utilized later).
 |
| Figure 2. Histogram of test image 1. |
As shown by the test image and it's histograms, majority of the pixel values lie at the extremes ends of the histogram (0 and 255). Increasing the contrast and amount of detail observable in the photograph involves manipulating the pixel values of an image to cover a larger dynamic range of values. This can be done through two methods: contrast stretching and histogram equalization. Of the two methods, contrast stretching is simpler as it just involves increasing the the distance between the pixel values within the histogram and as such, is performed by normalizing the image over a larger range of values (the minimum and maximum possible values for an image). Can we implement this for the test image shown? Without even having to try it, the answer is no. As shown by the histogram in Figure 2, the pixel values mostly fall at the minimum and maximum ends of the possible range. Any change made to this range will only serve to decrease contrast further. Thus, the only other way available is to find some way of redistributing the pixel values. This is where histogram equalization steps in.
As opposed to the linear scaling imposed by contrast stretching, histogram equalization is a nonlinear approach that aims to redistribute pixel values. To start, note that if we normalize the histogram of an image (as was done in Figure 2) against the total number of pixels, we obtain the probability density function (albeit, a discretized form of the probability density function). As such, we can apply the properties of probability density functions (pdf) and more importantly, cumulative probability density functions (cdf) to somehow manipulate the histogram. I'll skip the mathematical derivation and head right into the technique! The technique basically involves matching the cdf of the original image to a desired cdf through backprojection. For more information on this, refer to [2].
Anyway, histogram equalization aims to redistribute pixel values over the entire possible range such that each pixel value is given equal probability. Thus, the pdf of a histogram that is perfectly (more on this later) histogram equalized is that of a straight horizontal line. the resulting cdf is a straight increasing line with slope equal to 1. Thus, the original image pdf is flattened. How do we do this? This is pretty much summed up in the following figure taken from Ma'am Jing's handouts [3]! Note that the example used illustrates general histogram manipulation and not just histogram equalization (using a straight increasing cdf).
 |
| Figure 3. Graphical explanation of histogram equalization. Image taken from Figure 7 of [3]. |
In Figure 3, T(r) represents the original image cdf while G(z) represents the desired image cdf (which can be any desired lineshape). The variables r and z are treated (roughly) as continuous variables and represent the possible pixel values of the original and transformed pixel values, respectively. The process involves 4 steps. The first step and second steps involve finding the T(r) value corresponding to a specific r value. The third and fourth step involve finding the corresponding z value in the desired cdf at which G(z)=T(r). The corresponding z value is the non linearly transformed pixel value! This process is done for all pixel values within an image (the "r" values) to obtain the transformed histogram equalized image (the "z") values. Because the process involves going into the histogram domain and back into the image domain, this is sometimes referred to as histogram backprojection (remember color segmentation?). As mentioned earlier, histogram equalization requires that we use a linearly increasing cdf as shown in Figure 4.
 |
| Figure 4. Desired cdf for histogram equalization |
So let's get to it! The code used to implement such a procedure is pretty straightforward. Because Scilab is optimized to deal with matrix operations, the backprojection can be easily done without having to implement for loops! The code used to perform histogram equalization is shown below.
Lines 1-8 just involve loading the image and taking its gray scale (let's take a look at colored images later!). Lines 10-15 involve getting the histogram of the image and normalizing it to obtain the pdf. The cumsum()function is then used in lines 17-22 to obtain the cdf of the original image. Next, lines 25-28 generate the linearly increasing cdf as shown in Figure 4. Lines 34-36 find the T(r) values mentioned earlier (steps 1 and 2). Steps 3 and 4 are summed up in line 40. Note that to find the z values at which T(r) = G(z), we had to use interpolation to find the exact z values at which this condition held. Lines 43 onwards then just involve reshaping the resulting image array, rounding the pixel values to the nearest integer (because the interpolation process led to decimal z values), and displaying the resulting image. Let's now take a look at some test images and some results!
In Figure 6, we use three gray scaled test images (top row) and perform the histogram equalization procedure to obtain their "enhanced" versions (bottom row). Immediately, we see that all of the images are significantly brightened with respect to their original images. Moreover, areas that appear to be black in the original images actually reveal a lot of information when viewed in the histogram equalized outputs. The increase in contrast of the originally dark regions, however, is accompanied by an apparent decrease in contrast of the originally bright regions (see the plants in the window of test image (a), the words on the menu board of test image (b), and the objects in the far background of test image (c)). Moreover, notice that there is significant noise in output (a) that is absent from both outputs (b) and (c). An immediate (but rather incomplete) answer to this question is that it is due to the fact that the histogram equalization procedure is indiscriminate. What this means is that the procedure that only enhances the contrast of the "desired image", but also of any noise present in the photograph. Furthermore, it is a global procedure which means that the values of pixels far away may affect the value of the current pixel being considered. Further discussion necessitates that we take a look at the cdf and pdf of originals and outputs.
Comparing the original and transformed pdf's, we immediately see that the shift of the entire pdf to higher values is responsible for the increase in brightness observed after the transformation. Moreover, the increase in contrast of the dark regions is due to the redistribution of the pixel values that were originally clumped up in the "dark" end to the "brighter" end. This thus also results in a reduction of contrast of the light regions. Lastly, we come to the point of noise. Why is it that there is a significant amount of noise in output (a), but not as much noise in outputs (b) and (c)? Note that in the original pdf of output (a), we see that the image pixel values are mostly concentrated within sharp regions at the extremes of the gray scale range. The image pixel values for cases (b) and (c), however, are more spread out despite being dominantly shifted to the darker portion of the gray scale range. The histogram equalization procedure can only remap pixel values to a certain extent and thus if the pixel values are concentrated on a very narrow range of pixel values, the resulting histogram will only be slightly flattened (see the pdf of the outputs!). Thus, in other words, it is harder to "flatten" the image if there are dominant peaks in the original image's histogram. The "noise" that results is an effect due to this limitation.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 | pic=imread('C:\Users\domini8888\Desktop\Activity 10\bad.JPG') //imshow(pic) graypic=rgb2gray(pic) //graypic is used to generate the histogram counts while graypic_d is for the backprojection graypic_d=double(graypic) //imshow(graypic) //imwrite(graypic,'C:\Users\domini8888\Desktop\Activity 10\best_gray.JPG') [counts,binloc]=imhist(graypic) //normalize the histogram to obtain pdf counts=counts./(size(graypic,1)*size(graypic,2)) //plot(binloc,counts) //ylabel('probability') //xlabel('pixel value') //obtain the cdf cdf_pic=cumsum(counts) cdf_pic=cdf_pic' x=binloc' //plot(binloc,cdf_pic)T //ylabel('T(r)') //////////////////////////////////// //CHANGE THIS TO CHANGE THE CDF USED cdf_new=0:255 cdf_new=cdf_new./max(cdf_new) x_new=0:255 //////////////////////////////////// //plot(x_new,cdf_new) //ylabel('G(z)') //find T(r) for each pixel value in graypic (+1 because indices start at 1) //graypic contains the INTENSITY VALUES and are the INDICES of T(r) with the added +1 T=cdf_pic(graypic_d+1) //find the new pixel values by interpolating the inverse G(z) function at the T points // inverse G(z) function is x=cdf_new, y=x_new z=interp1(cdf_new,x_new,T,'spline') //reshape z back into a 2d array z=matrix(z,size(graypic,1),size(graypic,2)) zint=round(z) zint=uint8(zint) //imshow(zint) //imwrite(zint,'C:\Users\domini8888\Desktop\Activity 10\best_equal.JPG') |
Figure 5. Code used to implement histogram equalization
Lines 1-8 just involve loading the image and taking its gray scale (let's take a look at colored images later!). Lines 10-15 involve getting the histogram of the image and normalizing it to obtain the pdf. The cumsum()function is then used in lines 17-22 to obtain the cdf of the original image. Next, lines 25-28 generate the linearly increasing cdf as shown in Figure 4. Lines 34-36 find the T(r) values mentioned earlier (steps 1 and 2). Steps 3 and 4 are summed up in line 40. Note that to find the z values at which T(r) = G(z), we had to use interpolation to find the exact z values at which this condition held. Lines 43 onwards then just involve reshaping the resulting image array, rounding the pixel values to the nearest integer (because the interpolation process led to decimal z values), and displaying the resulting image. Let's now take a look at some test images and some results!
 |
| Figure 6. Original images (top row) and histogram equalized results (bottom row). Each column of images will be referred to by the letters (a), (b), and (c). |
In Figure 6, we use three gray scaled test images (top row) and perform the histogram equalization procedure to obtain their "enhanced" versions (bottom row). Immediately, we see that all of the images are significantly brightened with respect to their original images. Moreover, areas that appear to be black in the original images actually reveal a lot of information when viewed in the histogram equalized outputs. The increase in contrast of the originally dark regions, however, is accompanied by an apparent decrease in contrast of the originally bright regions (see the plants in the window of test image (a), the words on the menu board of test image (b), and the objects in the far background of test image (c)). Moreover, notice that there is significant noise in output (a) that is absent from both outputs (b) and (c). An immediate (but rather incomplete) answer to this question is that it is due to the fact that the histogram equalization procedure is indiscriminate. What this means is that the procedure that only enhances the contrast of the "desired image", but also of any noise present in the photograph. Furthermore, it is a global procedure which means that the values of pixels far away may affect the value of the current pixel being considered. Further discussion necessitates that we take a look at the cdf and pdf of originals and outputs.
 |
| Figure 7. Original and transformed pdf and cdf graphs of the test images (a), (b), and (c). |
Now why do we encounter this problem, anyway? This is due to the fact that it was assumed earlier on that r and z are continuous random variables which is clearly not the case as they are restricted to integer values from 0 to 255. Histogram equalization can only "remap" gray values. It cannot separate large clusters. Thus, if there a dominant peak results at a specific pixel value, histogram equalization cannot do much to fix this. Thus, it can be said that the histogram equalization process performs adequately well when the peaks (if there are any) within a specific image's histogram are not too sharp.
Now that we're done with that, let's try using a sigmoid shaped cdf and the same test images. I thought of generating the sigmoid shaped cdf by taking the cumulative sum of a Gaussian of mean 127 and standard deviation of sqrt(30). The code snippet in Figure 8 just needs to replace lines 25-28 in Figure 5 and we're all set!
1 2 3 | x_new=0:255 cdf_new=exp(-((x-127)/30).^2/2) cdf_new=cumsum(cdf_new)./max(cumsum(cdf_new)) |
Figure 8. Code used to generate the sigmoid shaped CDF
Just for reference, here's the sigmoid shaped cdf used.
 |
| Figure 9. Sigmoid shaped cdf to be used |
 |
| Figure 10. Original images (top row) and histogram manipulated (sigmoid cdf) results (bottom row). Each column of images will be referred to by the letters (a), (b), and (c). |
Now interestingly enough, we can extend histogram equalization into the color domain. Let's use test image (c) for this. The uninformed and hasty student would straight up apply the procedure to each RGB channel and then recombine the results. So let's do that. (I used GIMP to compose the output R, G, and B images from the Scilab code).
 |
| Figure 11. Original (left) and incorrectly histogram equalized (right) images |
So what are we to do now? Recall normalized chromaticity coordinates (see my blog on color segmentation!). The point of normalized chromaticity coordinates (NCC) is that it facilitates the separation of intensity (brightness) from chromaticity information. Thus, if we convert our channels into NCC, we can apply the histogram equalization procedure on the brightness component I to obtain I' and then convert each NCC back to RGB (for example, R=rI'). We do exactly this to obtain the result in Figure 12.
 |
| Figure 12. Original (left) and correctly histogram equalized (right) images |
Color looks okay! 😆
Once again, I have finished yet another blog and I am immensely tired. I believe I tackled a lot of interesting points regarding histogram manipulation. Moreover, I feel I have pointed out interesting observations which include the limitations of the histogram equalization algorithm. I believe explanations were more than enough and I was able to provide the viewer with an intuitive feel of the topic being discussed. I was also able to find a way to code the backprojection without using for loops. For this, I'd give myself a 12/10.
References:
[1] Contrast Stretching. (n.d.). Retrieved December 5, 2016, from http://homepages.inf.ed.ac.uk/rbf/HIPR2/stretch.htm
[2] (n.d.). Retrieved December 5, 2016, from http://www.math.uci.edu/icamp/courses/math77c/demos/hist_eq.pdf
[3] Soriano, M. Activity 10 Manual. Enhancement by Histogram Manipulation. 2016.
No comments:
Post a Comment