Data Representation
1. Introduction
In computer science area, we can deem that the basic information component is the bit streams representing by 0, 1 string. Based on 0 and 1 bit streams, we have developed information patterns such as text, images, videos, and audio that can be intuitively understood by humans.
Text
Image
1. RGB Principal
1. The visual mechanism of the human eye provides a basis for describing color using three quantities.
Light possesses various spectral compositions. Upon entering the eye, light stimulates photoreceptor cells, and the visual system subsequently generates the perception of color based on these stimuli.
In humans with normal vision, photopic (daylight) vision relies primarily on three types of cone cells, each exhibiting a distinct response to different ranges of light wavelengths. Consequently, under specific viewing conditions, the stimulation of these three types of cells by a given light source can be characterized by three distinct quantities.
This implies that even if two light sources have different spectral compositions, they can still be perceived as the same color, provided they elicit identical levels of stimulation across these three types of cone cells.
This principle serves as the foundation for display devices to reproduce a wide range of colors using only a limited number of light sources.
2. A specific range of colors can be reproduced by adjusting red, green, and blue light.
By selecting red, green, and blue as primary colors and independently controlling the intensity of each, display devices can generate a diverse array of color perceptions.
For instance, an appropriate mixture of red and green light is perceived by the human eye as yellow.
Thus, any color that can be reproduced using this set of primary colors can be described by three individual components:
- Red Component
- Green Component
- Blue Component
3. Color Representation in Space Distribution
A picture in computer can be divided into the minimized unit: Pixel. We can simply deem a pixel is a point.
A picture is composed of many pixels.
We need to describe two things:
- Pixel Location
- Pixel Color
So a picture can be described as Pixel Location and Color information.
4. Color Component Precision
We have discussed above that three basic color components R, G and B can be used to compose different other colors. For each color component, there is a difference in color degree. For example, there is a difference between light red and dark red. In the computer, we divide the degree of distinction of each color component into an interval from 0 to 255.
1 | R: 0 ~ 255 |
Now a pixel’s color can represent as:
1 | (255, 0, 0) |
In a given RGB color space, 255 represents the maximum encoded value for a channel, while 0 represents a value of zero for that channel.
5. Color Precision Represented by Bit
Obviously, we can use 8 bits to represent 0 ~ 255.
e.g.
1 | R G B |
2. From Pixel to Image (Raster Image)
We have discussed the RGB principle and the image’s basic unit: pixel.
An image is in fact a 2-dimension pixel grid.
e.g. There are 6 pixels from A to F with corresponding RGB value arranging in 3 columns and 2 rows.
1 | x → |
Common Conventions:
- The upper left corner is the origin
- x increases to the right
- y increases downward
So the pixel E with coordinates (1, 1).
This image described by 2-dimensions pixel grid is called Raster Image.
So each pixel contains two information entry, the position and color:
1 | Position: (x, y) |
3. Image Storage
3.1 From Image to Bits
How to store an image in the computer?
1 | x → |
A common way is: from left to right in each row, and from top to bottom between rows. This is called row-major order.
1 | 2D Image: Pixel Sequence: |
Assume that each pixel has three color channels (RGB) and each channel takes 8 bits. So a single pixel needs 3B space. 6 pixels take 3 * 6 = 18B in total.
1 | Pixel: A B C |
Offset refers to the byte offset relative to the start of this pixel data, starting from 0.
So if we have the image’s binary information . If we want to know the 5th pixel’s color, we can infer that the 5th pixel’s offset number is 3 * (5 - 1) = 12.
1 | 5th Pixel(E): |
General Formula for Row-Majored Order:
1 | Pixel index = y × Width + x |
1 | x |
- X-Coordinate starts from zero (0, 1, 2 … width - 1)
- Y-Coordinate starts from zero (0, 1, 2 … height - 1)
Pixel A:
1 | x = 0 |
Pixel E:
1 | x = 1 |
3.2 From Bits to Image
Suppose that we have a string of RGB bitstreams:
1 | B_0 B_1 B_2 ... B_12 B_13 B_14 B_15 |
Can we restore the image ? Of course not, because
We do not know the space information of the image. e.g. 6 pixels can be arranged as 3 * 2 or 2 * 3.
1
2
3
4
5
6
7
8
9
10Width = 3, Height = 2
A B C
D E F
Width = 2, Height = 3
A B
C D
E FWe don’t know the RGB rule. The three color bytes can be described into different rules
1
255, 0, 0
RGB rule:
1
2
3R = 255
G = 0
B = 0
BGR rule:
1
2
3B = 255
G = 0
R = 0
So we need extra information to record full information:
1 | Image description: |
3.3 Stride
For a 3 * 3 image:
1 | Row 0: [ A ][ B ][ C ] |
The gap of the offset of each line is 9 which is called Stride.
However some hardware or image processing interfaces require row data to be aligned to specific byte boundaries.
Suppose each row is required to take up space in multiples of 4 bytes. A row of color data has 9 bytes, so you can add 3 bytes at the end to make it 12 bytes:
1 | Pixel data Padding |
1 | Pixel data per row = 9 bytes |
Formula for Row-Majored Order with Stride:
1 | Byte offset = y × Stride + x × Bytes per pixel |
e.g. the pixel E (1,1)’s offset:
1 | offset = 1 * 12 + 1 * 3 = 15 |
3.4 RGBA (Transparency)
RGBA is an extension format of RGB with an extra channer Alpha to describe the transparency.
Why we need Alpha to describe the transparency?
Suppose we want to place a red icon on a blue background. The icon might have areas that completely cover the background, or it might have transparent or semi-transparent areas.
RGB can only describe the color of a pixel; it can’t tell us how it should blend with the background on its own.
| Alpha Value | Meaning |
|---|---|
| 255 | Completely Opaque |
| 128 | Half Opaque |
| 0 | Completely Transparent |
For instance, a red pixel can have different alpha value:
1 | (255, 0, 0, 255) Fully opaque |
Place the red pixels on a blue background: - Alpha 255: The spot shows the foreground red, completely covering the background. - Alpha 128: The foreground red and background blue both affect the final color. - Alpha 0: The foreground doesn’t contribute color, so the spot shows the background blue.
With an extra channel, the pixel’s color information will increase from 3 to 4 bytes.
1 | RGB pixel: |
So the formula for RGBA8 (8 for 8 bits):
1 | Bytes per pixel = 4 |
If the stride is known:
1 | Byte offset = y × Stride + x × 4 |
3.5 Grayscale Image
As we said above, we can use RGB combination to illustrate a specific range of colors. There’s a phenomenon: when R = G = B, we will get black, white and gray.
| RGB value | Appearance |
|---|---|
(0, 0, 0) |
Black |
(64, 64, 64) |
Darker Gray |
(128, 128, 128) |
Gray |
(192, 192, 192) |
Lighter Gray |
(255, 255, 255) |
white |
For this kind image, we can use single-channel to represent the color instead of using three kinds.
Grayscale pixels usually only keep one gray value.
Using the common 8-bit grayscale format, usually called Gray8:
1 | Gray value: 0 ........ 128 ........ 255 |
Each pixel only includes a gray value:
- 0 for black
- 255 for white
- middle values indicate different shades of gray
4. Image Compression
4.1 Redundancy
Suppose an image’s specification is 1 * 8, and each pixel is red.
1 | Width = 8 |
So each pixel’s RGB value is: (255, 0, 0). Totally cost
3 * 8 = 24 Bytes
We can use one sentence to describe these pixels:
Reapt Red 8 times. This rule includes following
information:
1 | Color: (255, 0, 0) |
Reapt Red 8 times is said for human. In computer, we can
use the following format data to record consecutive
identical value:
Format:
1
2
3+---------------+--------------+--------------+-----------+
| Count: 1 byte | R Value: 1B | G Value: 1B | B Value: 1B |
+---------------+--------------+--------------+-----------+Example:
1
2Count R G B
8 255 0 0This record only takes 4B space comparing with the original 24B.
Decompress:
When the image dealing process read: (8, 255, 0, 0), it
knows that there are 8 consecutive (255, 0, 0)