Quantization is really the second half of this story in digital signal processing. The first half being the discretization of time. We soon realize that digital devices can only deal with integers no matter how many bits we use inside each memory cell. And so we need to map the numeric range that discrete time samples live on or to a finite set of values. In so doing there is an irreversible loss of information because we're chopping these amplitudes according to the resolution that our system allows for. If we were to represent the situation graphically we have a sequence of discrete time samples here that belong say, to the set of complex numbers. These samples go through a quantizer, and the sequence of quantized samples come out where each quantized sample now belongs to the set of integers. We model input as a stochastic process. And to study the effects of the system, we have to consider several factors. How many bits per sample this quantizer will allocate? What is the storage scheme used to represent the quantize samples? For instance is it fixed point or floating point, and what are the properties of the input as a stochastic process? What is its range and what is its probability distribution? The simplest quantizer is the Scalar quantizer. In this quantizer, each sample is encoded individually. So we don't take into account, relationship between neighboring samples. Each sample is quantized independently, so there is no memory of previous quantization operations, and each sample is encoded using R bits. So the rate here is R bits per sample. Let's see what happens when we Scalar quantize an input. Assume we know that each input sample is strictly between A and B. Each sample is quantized over 2 to the R possible values, because we are using R bits per sample and this defines 2 to the R intervals over the range A to B. Each interval will be associated to a quantization value. Which means that whenever the sample, say, falls into this interval here, it will be replaced by this representative value for the interval and similarly for the other intervals. So let's look at an example for R = 2. The range A to B, would be divided into four intervals, and these are the boundaries of each interval. We would associate a representative point to each interval, and we would encode each interval using two bits, so the sequence zero, zero would be associated to the first interval, and so on and so forth. In other words, the quantized values would be one of these four possible values. And internally, the quantizer would know how to associate this binary value to this real value. The two natural questions at this point are, what are the optimal interval boundaries IFK and what are the optimal quantization values for each interval [INAUDIBLE] to find an answer, let's consider the quantization error. So this is defined as difference between the quantized value, namely the representative value for each interval, minus the real value. We model the input as a stochastic process, as we said in the beginning, and we model the error as a white noise sequence. In other words, we assume the samples are uncorrelated. And we assume that all error samples have the same distribution. These are rather drastic assumptions, but as a first approximation, they will give us a good feeling for the effects of a quantizer. In order to proceed further, we need the statistical description of the input samples. Let's also make some assumptions on the general structure of the quantizer. And let's consider the simple, but very common case of uniform quantization. The range in this case is split into 2 to the R equal intervals of width delta, which is equal to B- A, the range of the input samples. Divided by 2 to the R, the number of levels afforded to by a rate of R beats percent. So in the case of R equal to 2, as before, our range would be split into four equal width intervals. The Mean Square Quantization Error is the variance of the the error signal, namely the expectation of the difference between the quantized samples and the original samples. If we know the probability distribution function for the input, we can right that as an integral from A to B of the PDF of the input times the error function, which is the quantized value of the integration variable minus the integration variable squared. This is the standard application of the expectation theorem. Then we can finally split the integral over the independent quantization intervals. And we get this last formulation, here. Now, in order to proceed, we need to know the probability distribution function of the input to compute these integrals. So now we make a further hypothesis on the input, we assume that it is uniformly distributed. That means that the probability distribution function is just a constant from A to B, a value 1 over B- A. The mean squared error becomes the sum of 2 to the r- 1 independent integrals. Each on in which is the integral over the quantization interval of the representative value for the interval, which we haven't determined yet, minus tau squared divided by B- A. In order to find the optimal quantization points we minimize the mean square error with respect to the quantization points themselves. We take a partial derivatives of the error with respect to x hat of m. When we take partial derivatives of the sum, the partial derivative will kill all terms of the sum except the one that depends on the derivation variable so we're left with an integral over the interval I m of 2 times the quantization point minus tau divided by B- A in the tau. We have to compute this integral over the quantization interval number m and you remember the range is from a to b, we divide this into 2 to the r equal intervals of size delta and a lower and upper boundary for the interval number m will be A+m delta and A+ m delta + delta. In order to minimize the error we set the partial derivatives to 0 for all quantization intervals, and we find that this happens when the quantization point Is, the interval's midpoint. With this we can plot the quantizer's characteristic. Here we show it for R equal to 3, three bits per sample. And you can see that the quantizer associates with quantization interval, to its midpoint. And you have a typical staircase characteristic of the uniform quantizor. Back to the mean square error, we now replace into the expression for the mean square error the values that we found in the previous analysis. Namely, the boundaries for each quantization interval, the value for the midpoint and the expression for the probability of distribution of the input. And if we compute this integral, we obtain the fundamental result of uniform quantization. The mean square error for uniform quantizer is equal to delta square over 12 where delta is (B-A)/(2^R). If we analyze the R a little bit further, we can relate the expression of the error to the expression for the signal's energy. Since we assume that information is uniformly distributed, we can compute its variance, i.e. it's energy, as (B-A)^2 over 12. It’s the variance over uniformly distributed variable and so we can compute the signal-to-noise ratio as, the power of the signal divided by the power of the error and the signal-to-noise ratio happens to be 2 to the 2R. So if the input is uniformly distributed and the quantizer is a uniform quantizer, which means it's matched to the input, the signal to noise ratio is only a function of the number of bits per sample that we allocate. We can express this result in decibels by taking 10 times the log in base 10. Of 2 to the power of 2R, and we get the famous and handy formula of 6 dB's per bit. In other words, every bit we add to the internal representation of a quantized signal adds 6 dB's of signal-to-noise ratio. So for instance, a compact disk has 16 bits per sample, so the maximum signal-to-noise ratio that you can achieve in a CD is 96 dB's. A DVD on the other hand has 24 bits per sample so your signal-to noise ratio grows to 144 dB.
