Quantization is where the actual data compression occurs, by a shortening of the transformed data wordlength. The transform is invertible, so that no data compression actually occurs there. A quantizer is a nonliner, zero-memory device that choose representative values for ranges of input data coming from the transform one-at-a-time, called scalar quantization, or several-at-a-time, called vector quantization. The quantization operation is one of truncation or rounding. x->Q(.)->x'
r1 -- d_1 < x < d_1
Q(x) = ri -- d_i-1 < x < d_i
rj -- d_j-1 < x < d_j
A quantization matrix is used in combination with a DCT coefficient matrix. Quantization matrices are often specifically designed to keep certain frequencies in the source to avoid losing image quality. An example DCT coefficient matrix:
A common quantization matrix is:
Using this quantization matrix with the DCT coefficient matrix:
For example, using −415 (the DC coefficient) and rounding to the nearest integer