Information about Jpeg

JPEG
A photo of a flower compressed with successively more lossy compression ratios from left to right.
File extension:	.jpeg, .jpg, .jpe .jfif, .jfi, .jif (containers)
MIME type:	image/jpeg
Type code:	JPEG
Uniform Type Identifier:	public.jpeg
Developed by:	Joint Photographic Experts Group

In computing, JPEG (pronounced JAY-peg; IPA: [ˈdʒeɪpɛg]) is a commonly used method of compression for photographic images. The name JPEG stands for Joint Photographic Experts Group, the name of the committee that created the standard. The group was organized in 1986, issuing a standard in 1992 which was approved in 1994 as ISO 10918-1. JPEG is distinct from MPEG (Moving Picture Experts Group) which produces compression schemes for video.

The JPEG standard specifies both the codec, which defines how an image is compressed into a stream of bytes and decompressed back into an image, and the file format used to contain that stream. The compression method is usually lossy compression, meaning that some visual quality is lost in the process, although there are variations on the standard baseline JPEG which are lossless. There is also an interlaced "Progressive JPEG" format, in which data is compressed in multiple passes of progressively higher detail. This is ideal for large images that will be displayed whilst downloading over a slow connection, allowing a reasonable preview before all the data has been retrieved. However, progressive JPEGs are not as widely supported.

The file format is known as 'JPEG Interchange Format', as specified in Annex B of the standard. This is often confused with the JPEG File Interchange Format (JFIF), a minimal version of the JPEG format that was deliberately simplified so that it could be widely implemented and thus become the de-facto standard. Most image editing software programs that write to a "JPEG file" are actually creating a file in JFIF format.^[1]

Image files that employ JPEG compression are commonly called "JPEG files". The most common file extension for this format is .jpg, though .jpeg, .jpe, .jfif and .jif are also used. It is also possible for JPEG data to be embedded in other file types, such as TIFF format images.

JPEG/JFIF is the format most used for storing and transmitting photographs on the World Wide Web. For this application, it is preferred to formats such as GIF, which has a limit of 256 distinct colors that is insufficient for color photographs, and PNG, which produces much larger image files for this type of image. The JPEG compression algorithm is not as well suited for line drawings and other textual or iconic graphics, and thus the PNG and GIF formats are preferred for these types of images.

The MIME media type for JPEG is image/jpeg (defined in RFC 1341).

JPEG codec

Although a JPEG file can be encoded in various ways, most commonly it is done with JFIF encoding. The encoding process consists of several steps:

1. The representation of the colors in the image is converted from RGB to YCbCr, consisting of one luma component (Y), representing brightness, and two chroma components, representing color. This step is sometimes skipped.
2. The resolution of the chroma data is reduced, usually by a factor 2. This reflects the fact that the eye is less sensitive to fine color details than to fine brightness details.
3. The image is split into blocks of 8×8 pixels, and for each block, each of the Y, Cb, and Cr data undergoes a discrete cosine transform (DCT). A DCT is similar to a Fourier transform in the sense that it produces a kind of spatial frequency spectrum.
4. The amplitudes of the frequency components are quantized. Human vision is much more sensitive to small variations in color or brightness over large areas than the strength of a high-frequency brightness variations. Therefore, the magnitude of the high-frequency components are stored with a lower accuracy than the low-frequency components. The quality setting of the encoder (for example 50% or 95%) affects to what extent the resolution of each frequency component is reduced. At a low quality setting, the high-frequency components are discarded altogether.
5. The resulting data for all 8×8 blocks is further compressed with a loss-less algorithm, a variant of Huffman encoding.

The decoding process merely reverses these steps. In the remainder of this section, the encoding and decoding processes are described in more detail.

Encoding

Many of the options in the JPEG standard are not commonly used, and as mentioned above, most image software uses the simpler JFIF format when creating a JPEG file, which amongst other things specifies the encoding method. Here is a brief description of one of the more common methods of encoding when applied to an input that has 24 bits per pixel (eight each of red, green, and blue). This particular option is a lossy data compression method.

Color space transformation

First, the image should be converted from RGB into a different color space called YCbCr. It has three components Y, Cb and Cr: the Y component represents the brightness of a pixel, the Cb and Cr components represent the chrominance (split into blue and red components). This is the same as the color space used by PAL, MAC and digital color television transmission (but not by NTSC, which uses the similar YIQ color space). The YCbCr color space conversion allows greater compression for the same image quality (or greater image quality for the same compression).

This conversion to YCbCr is specified in the JFIF standard, and should be performed for the resulting JPEG file to have maximum compatibility. However, many "high quality" JPEG images do not apply this step and instead keep them in the sRGB color space, where each color plane is compressed and quantized separately with similar quality levels.

Downsampling

The human eye can see more detail in the Y component (brightness) than in Cb (blue) and Cr (red). Using this knowledge, encoders can be designed to compress images more efficiently.

The above transformation enables the next step, which is to reduce the Cb and Cr components (called "downsampling" or "chroma subsampling"). The ratios at which the downsampling can be done on JPEG are (no downsampling), (reduce by factor of 2 in horizontal direction), and most commonly (reduce by factor of 2 in horizontal and vertical directions). For the rest of the compression process, Y, Cb and Cr are processed separately and in a very similar manner. Downsampling the chroma components saves 33% or 50% of the space taken by the image.

Block splitting

After subsampling, each channel must be split into 8×8 blocks (of pixels). If the data for a channel does not represent an integer number of blocks then the encoder must fill the remaining area of the incomplete blocks with some form of dummy data:

• filling the edge pixels with a fixed color (typically black) creates dark artifacts along the visible part of the border
• repeating the edge pixels is a common but non-optimal technique that avoids the visible border, but it still creates artifacts with the colorimetry of the filled cells
• a better strategy is to fill pixels using colors that preserve the DCT coefficients of the visible pixels, at least for the low frequency ones (for example filling with the average color of the visible part will preserve the first DC coefficient, but best fitting the next two AC coefficients will produce much better results with less visible 8×8 cell edges along the border).

Discrete cosine transform

Next, each component (Y, Cb, Cr) of each 8×8 block is converted to a frequency-domain representation, using a normalized, two-dimensional type-II discrete cosine transform (DCT).

As an example, if one such 8×8 8-bit subimage is:

The next step is to transform the subimage from a positive range to one centered around zero. For an 8-bit image each pixel has 256 possible values: . To center around zero it is necessary to subtract by half the number of possible values, or 128.

Subtracting 128 from each pixel value yields pixel values on

The next step is to take the two-dimensional DCT, which is given by:

where

• is the horizontal spatial frequency, for the integers .
• is the vertical spatial frequency, for the integers .
• is a normalizing function
• is the pixel value at coordinates
• is the DCT coefficient at coordinates

If we perform this transformation on our matrix above, and then round to the nearest integer, we get

Note the rather large value of the top-left corner. This is the DC coefficient. The remaining 63 coefficients are called the AC coefficients. The DCT temporarily increases the bit-depth of the image, since the DCT coefficients of an 8-bit/component image take up to 11 or 12 bits (depending on fidelity of the DCT calculation) to store. This may force the codec to temporarily use 16-bit bins to hold these coefficients doubling the formal size of the image representation at this point. The advantage of the DCT is its tendency to aggregate most of the signal in one corner of the result, as may be seen above. The quantization step to follow accentuates this effect while simultaneously reducing the size of the DCT coefficients to 8 bits or less, resulting in a signal with a large trailing region containing zeros that the entropy stage can simply throw away. The temporary increase in size at this stage is not a performance concern for most JPEG implementations, because typically only a very small part of the image is stored in full DCT form at any given time during the encoding or decoding process.

Quantization

The human eye is good at seeing small differences in brightness over a relatively large area, but not so good at distinguishing the exact strength of a high frequency brightness variation. This fact allows one to get away with greatly reducing the amount of information in the high frequency components. This is done by simply dividing each component in the frequency domain by a constant for that component, and then rounding to the nearest integer. This is the main lossy operation in the whole process. As a result of this, it is typically the case that many of the higher frequency components are rounded to zero, and many of the rest become small positive or negative numbers, which take many fewer bits to store.

A common quantization matrix is:

The quantized DCT coefficients are computed with

where is the unquantized DCT coefficients; is the quantization matrix above; and is the quantized DCT coefficients. (Note that this is in no way matrix multiplication.)

Using this quantization matrix with the DCT coefficient matrix from above results in:

For example, using −415 (the DC coefficient) and rounding to the nearest integer

Entropy coding

Main article: Entropy encoding

Entropy coding is a special form of lossless data compression. It involves arranging the image components in a "zigzag" order employing run-length encoding (RLE) algorithm that groups similar frequencies together, inserting length coding zeros, and then using Huffman coding on what is left.

The JPEG standard also allows, but does not require, the use of arithmetic coding which is mathematically superior to Huffman coding. However, this feature is rarely used as it is covered by patents and because it is much slower to encode and decode compared to Huffman coding. Arithmetic coding typically makes files about 5% smaller.

The zigzag sequence for the above quantized coefficients are shown below. (The format shown is just for ease of understanding/viewing.)

−26
−3	0
−3	−2	−6
2	−4	1	−4
1	1	5	1	2
−1	1	−1	2	0	0
0	0	0	−1	−1	0	0
0	0	0	0	0	0	0	0
0	0	0	0	0	0	0
0	0	0	0	0	0
0	0	0	0	0
0	0	0	0
0	0	0
0	0
0

If the i-th block is represented by Bi and positions within each block are represented by (p,q) where p = 0, 1, ..., 7 and q = 0, 1, ..., 7, then any coefficient in the DCT image can be represented as Bi(p,q). Thus, in the above scheme, the order of encoding pixels (for the i-th block) is Bi(0,0), Bi(0,1), Bi(1,0), Bi(2,0), Bi(1,1), Bi(0,2), Bi(0,3), Bi(1,2) and so on.

This encoding mode is called baseline sequential encoding. Baseline JPEG also supports progressive encoding. While sequential encoding encodes coefficients of a single block at a time (in a zigzag manner), progressive encoding encodes similar-positioned coefficients of all blocks in one go, followed by the next positioned coefficients of all blocks, and so on. So, if the image is divided into N 8×8 blocks {B0,B1,B2, ..., Bn-1}, then progressive encoding encodes Bi(0,0) for all blocks, i.e., for all i = 0, 1, 2, ..., N-1. This is followed by encoding Bi(0,1) coefficient of all blocks, followed by Bi(1,0)-th coefficient of all blocks, then Bi(0,2)-th coefficient of all blocks, and so on. It should be noted here that once all similar-positioned coefficients have been encoded, the next position to be encoded is the one occurring next in the zigzag traversal as indicated in the figure above. It has been found that Baseline Progressive JPEG encoding usually gives better compression as compared to Baseline Sequential JPEG due to the ability to use different Huffman tables (see below) tailored for different frequencies on each "scan" or "pass" (which includes similar-positioned coefficients), though the difference is not too large.

In the rest of the article, it is assumed that the coefficient pattern generated is due to sequential mode.

In order to encode the above generated coefficient pattern, JPEG uses Huffman encoding. JPEG has a special Huffman code word for ending the sequence prematurely when the remaining coefficients are zero.

Using this special code word: "EOB", the sequence becomes:

−26
−3	0
−3	−2	−6
2	−4	1	−4
1	1	5	1	2
−1	1	−1	2	0	0
0	0	0	−1	−1	EOB

JPEG's other code words represent combinations of (a) the number of significant bits of a coefficient, including sign, and (b) the number of consecutive zero coefficients that follow it. (Once you know how many bits to expect, it takes 1 bit to represent the choices {-1, +1}, 2 bits to represent the choices {-3, -2, +2, +3}, and so forth.) In our example block, most of the quantized coefficients are small numbers that are not followed immediately by a zero coefficient. These more-frequent cases will be represented by shorter code words.

The JPEG standard provides general-purpose Huffman tables; encoders may also choose to generate Huffman tables optimized for the actual frequency distributions in images being encoded.

Compression ratio and artifacts

The resulting compression ratio can be varied according to need by being more or less aggressive in the divisors used in the quantization phase. Ten to one compression usually results in an image that cannot be distinguished by eye from the original. 100 to one compression is usually possible, but will look distinctly artifacted compared to the original. The appropriate level of compression depends on the use to which the image will be put.

Those who use the World Wide Web may be familiar with the irregularities known as compression artifacts that appear in JPEG images. These are due to the quantization step of the JPEG algorithm. They are especially noticeable around eyes in pictures of faces. They can be reduced by choosing a lower level of compression; they may be eliminated by saving an image using a lossless file format, though for photographic images this will usually result in a larger file size. Compression artifacts make low-quality JPEGs unacceptable for storing heightmaps. The images created with ray-tracing programs have noticeable blocky shapes on the terrain.

Some programs allow the user to vary the amount by which individual blocks are compressed. Stronger compression is applied to areas of the image that show fewer artifacts. This way it is possible to manually reduce JPEG file size with less loss of quality.

Since the quantization stage always results in a loss of information, JPEG standard is always a lossy compression codec. (Information is lost both in quantizing and rounding of the floating-point numbers.) Even if the quantization matrix is a matrix of ones, information will still be lost in the rounding step.

Decoding

Decoding to display the image consists of doing all the above in reverse.

Taking the DCT coefficient matrix (after adding the difference of the DC coefficient back in)

and taking the entry-for-entry product with the quantization matrix from above results in

which closely resembles the original DCT coefficient matrix for the top-left portion. Taking the inverse DCT (type-III DCT) results in an image with values (still shifted down by 128)