Chroma Subsampling not a confusion anymore
I recently worked on implementing some of the functionalities that handle the YUV image. So this article is a by-product of the queries I had and the learning during the process.
I am writing this article hoping it might be helpful to someone who is or will be working on the same domain. So cheers to learning together!!
If we are working on a computer vision project or any tasks involving images, we are almost sure to work with the RGB color model, which most of us are already familiar with. An RGB color value is specified with red, green, and blue colors where each of the colors defines the intensity with values from 0 to 255. For example all parameters values as one i.e {1,1,1} defines black color and {255,255,255} defines white color.
When the monitor displays an image, every color in that image can be represented by three colors. RGB images are typically encoded in 8, 12, 16, or 24 bits per pixel. If we assume 24 bits per pixel, then every pixel in a monitor has 3 bytes (each byte for red, blue, and green) of information. This can be represented as RGB888 (an RGB color value encoded as 8 bits per channel). The standard byte format is:
r0, g0, b0, r1, g1, b1, r2, g2, b2, ...
To play around with what different parameters define which color, you can use the link below and check it for yourself.
Motivation behind the YUV Color Format
Now comes the real fun human being’s eye is very sensitive to brightness but it is not much to color. Taking advantage of this, here comes the YUV representation of the color or YUV color model. “Y” is for the luminescence (brightness), “U”(blue projection), and “V”(red projection) means chrominance (color). This is similar to the RGB color model for representing a pixel of an image. We may wonder why the YUV color model when we already have the RGB model with us. This is all about because of the file size :). We already know that we use 3 bytes for each pixel in the RGB model to record color. Just imagine if we could reduce half of the size of the file but still get the information that we need. This is where the YUV color model comes into the picture and dramatically decreases the file size as compared to the RGB model. So now that we have some understanding of why the YUV model is introduced let’s talk about it more.
Types of YUV Format
The YUV format has two categories which are called packed format (non-planar format or interleaved format) and planar format (non-packed or non-interleaved format). The former stores the YUV components in the same array and several adjacent pixels form a macro-pixel while the latter uses three different arrays to store Y, U, and V components separately.
Let’s take the following examples for understanding these two different forms. For the packed format, the YUV components(if we consider the UYVY format that we will discuss below) will be placed as the following.
u1, y1, v1, y2, u2, y3, v2, y4,....and for the planar format, the same values will place in the following order.
u1, u2 ,..., v1, v2,..., y1, y2, y3, y4,...To get a more clear picture of these two representations we can check the following two images.
Many color camera models support the transmission of images in a YUV color encoding scheme. This scheme assigns both brightness and color values to each pixel. There are different types of YUV depending on how many bits are allocated per pixel. Each unique Y, U, and V value comprises 8 bits, or one byte, of data. Out of the different types, there are three major types of YUV which are 24, 16, and 12-bit per pixel (bpp) formats also called YUV444, YUV422, and YUV411 data formats.
In 16 and 12 bpp formats, the U and V color values are shared between pixels, which frees the bandwidth and may increase the frame rate.
This technique is called chroma sub sampling which takes the account that we already discussed that the human eyes are greater sensitive to variations in brightness than in color. Now let us understand this format briefly one by one.
Different YUV Formats
YUV444 Data Format
Let us first discuss the most straightforward format which is called as YUV444 data format. This format uses or transmits 24 bits per pixel wherein each pixel is assigned a unique Y, U, and V value and uses 1 byte for each value. They are ordered in such a way in the image that a combination of U, V, and Y represents a pixel value.
This order results in the following data structure for each pixel:
YUV422 Data Format (or the UYVY format)
This YUV422 data format is the one that I have recently worked on and is an inspiration for this blog. This format shares U and V values between two pixels. So as a result, these values are transmitted to the PC image buffer only once for every two pixels, resulting in an AVT (average transmission rate) of 16 bits per pixel. If we compare it with the YUV444 data format, the AVT has reduced from 24 to 16 bits per pixel.
This order results in the following data structure for each pixel:
YUV411 Data Format
Now, talking about the YUV411 format, this format shares U and V values between four pixels, resulting in an average transmission rate of 12 bits per pixel. Yes!! you see how the transmission rate is decreasing with different YUV formats.
This order results in the following data structure for each pixel:
Displaying the YUV Image or Video
YUV is raw data that does not contain the height and width information needed. We need to provide that while playing a YUV video or displaying a YUV image. Take the following link to download the YUV player from GitHub.
After installing the software provide the height and width information and other details, as shown in the Image 8 below.
In the above Image 8, there are many fields can be filled depending upon which format we want to use. The mandatory fields needed to view an image or play a YUV video are Width, Height, and the YUV Format. Here in this example, I had chosen the custom YUV format and the Chroma sub sampling 4:2:2 (or UVUY format), and selected the packed format to display an image. However, we can always play around with these fields and check for our self.
Some more YUV Formats
The different formats of YUV color models are not limited as I have already discussed above. There are many other formats of YUV that you can find in the following provided link.
Conversion from RGB to UYVY
If we want to convert an RGB image to a YUV format we can use the following link to use that. I am not discussing that here in detail since that is not the scope of this article.
Conclusion
Thank you !! for reading this article. I believe there is no better feeling than sharing the knowledge we gain during our learning. It was so wonderful writing this article, and I hope you also loved reading this article. If you feel that you have acquired something new from this article you can give claps on this page or follow me in the medium. That will surely give me the motivation to write many more such articles. With this, I sign off. Stay Safe and Happy Learning :)
