We can represent any signal, such as images, in the frequency domain. Then, we can perceive the segments in this frequency domain, the so-called sub-bands, and examine the behavior of the signal in each of these sub-bands. The characteristics of an image differ in each sub-band. Some are low-frequency sub-band, bands, some are high-frequency in one dimension. And low in the other, and so on. Then, the main idea in sub encoding is that we can now adapt the compression scheme to each sub-band. This means that different parameters can be utilized for the same encoder, or that a completely different decoder can be used in each sub-band. Therefore, there are a large number of possibilities. One of the issues, of course, is to design appropriate filters that would allow us to perform this sub-band decomposition. Filters that would allow perfect reconstruction of the signal. We discuss and analyze all these concepts in this segment. So let's have a specific look. The idea of sub-band coding is quite interesting and intriguing. So we first use a bank of filters to divide the image into sub-bands. And each sub-band contains a set of spatial frequencies. So each sub-band has reduced bandwidth, with respect to the original image. If the sub-bands are of equal size then we're talking about equal rate sub-band decomposition. If, on the other hand, they have different bandwidths then we're referring to as multirate decomposition. The main idea from this decomposition is now, we can exploit the properties of each band. And use a different compression scheme to compress the various bands. So different bitrates or different coding techniques can be used for each sub-band, by taking advantage of the properties of each sub-band, and also by allowing for errors to be distributed across sub-bands in a visually optimal manner. So this is a celebrated approach that has found applications both in still image compression as well in video compression. As well as in speech and audio compression. Some of the most successful audio compression standards are based on sub-band decomposition. So the principle of sub-band decomposition is demonstrated here. We show it for a one dimensional signal, but also show it to the signal in a few slides later. So, we have the analysis filter bank and the synthesis filter bank. So, we see that the original sig, signal is input here. Then, we use a band pass filter 0 to band pass filter M minus 1. After filtering, the signal is decimated, down-sampled by a factor of capital M. We assume again we have M such sub-bands. And after this decimation, we'll call these sub-bands separately, using a different encoder per sub-band. The compressed signal is either transmitted or stored to important applications in image and video compression. At the other end, the decoder. We have the synthesis bank of filters. So each sub-band is decoded separately. Then, it's up-sampled by a factor of M. And then it's band pass filtered and all the sub-bands are added to give an estimate of the original signal. These are operations that should make perfect sense to you at this point, because we covered them earlier in class. We saw what happens if we, we down-sample is the spectra overlap. We have the problem of aliasing. That's why we first low pass filter or band pass filter before we down-sample to avoid aliasing. On the other hand, when we up-sample, we introduce M zeros in between the samples. And then we use a band pass filter to extract one replica of the spectrum. Because what happens, in the frequency domain, I have replicas of the spectrum, through this introduction of zeroes. And through this band pass filtering, I pull out one replica, and I pull the signal in the original resolution. Here is the block diagram of a two-channel sub-band decomposition. And having this actually a two-channel I can then cascade it and end up with multi-band as well as multi-dimensional signal sub-band decomposition. So, we see the H1, H0 filters here. The H0 is a low pass filter, H1 is a high pass filter. So, the summation by a factor of two, encoding, up-sampling by a factor of two, and here are the up-sampling filters, F1, F0. Conceptually, the up-sampling filters are, are, are undoing what the down-sampling filter did, you might say. And there is a lot of work on how to design these filters H0, H1, F0, F1, so that in the absence of compression, I have perfect reconstruction. We don't have per, perfect filters, if we implement the spatial domain they have a pass band, a transition band and a stop band. And there, for example the quadrature mirror filters that give you perfect reconstruction is a topic that has seen a lot of activity. In signal processing and there are quite a lot of results that one can utilize. So this is what is done in practice when, with two band decomposition system I want to build a multiple sub-band decomposition system. So, I start again, I show this one-dimensional signal, you break it into the low-frequency band and the high-frequency band. So X, X of omega. If it's shifted by pi it gives me, if the first one's a low pass filter it results in a high pass filter. So downsampling takes place here, and now I will take the low band or the low frequency band and split it also in its low frequency and high frequency component. I can take this low frequency band, and fur, further split it into two. I can take this high-frequency and further split it into two. So, in this particular example, I end up with two, four, six, eight different sub-bands, and they have the name of, of this [UNKNOWN] essence. So this is the LLL band, it, only low pass filters were applied to this one to, to obtain this band. While this one, if the first filter's low pass the second is low pass but the third one is high pass. So pretty much we have all eight combinations of these different letters. And the last one is HHH. So the, the, the filter was filtered three times by how, high pass filter to give us this band. And this band is here, the highest frequency band. The first band is down here, the lowest frequency, and so on and so forth. So, again, having one good pair of decomposition, I can cascade it this way, and I end up in an equal rate as shown here. Obtain as many bands as I want in this multiband decomposition system. And since this pair has also its reconstruction, perfect reconstruction pair, I can also go from this eight band decomposition here, for example, to perfect reconstruction using the pairs that I started with. Here's the picture of sub-band decomposition in two dimensions. If I have a filter that can break the frequency spectrum of a signal into four bands, so it's a four band split decomposition, here. Then, I can use that in a cascade to obtain as many bands as needed. So, in this particular example here, you have a four band split. And this is the four band split. This is the LL, band and then LH is, is here, right? Because it's centered at zero you can do the periodic extension and see how these bands look like. And then we have the HL band and the HH band. So in the first pass I split the image into four bands. And then, for example, I can take the low frequency band and split it into four additional bands. So that's how the picture's going to look in the frequency domain. Here are the four original bands and then the low frequency band was split into four additional ones. So this is actually an example of multirate sub-band decomposition. Because the bands are not all of the same size or of the same rate. And in practice, that's how we obtain sub-bands, either equal rate or multirate, just using one pair. Either two filters for a one-dimensional signal or four filters are shown here for the two dimensional signal. Here's one example of the use of sub-band decomposition. All these examples again were used generated using DC Demo. The original image is shown here, again. And this is a four band decomposition using 16-tap filters, so the number of taps is the number of samples in the impulse's [UNKNOWN] filter. So you see here the four sub-bands, the low, low sub-band is just a decimated version of the original image. It was just low pass filter. This is the, these are the low H and HL bands, so they show the high frequencies in the horizontal and the vertical direction. And this is the HH band, which shows the high frequencies in both directions. By and large you see some structure of course in the low frequency band, and then. In the LH and HL bands and the HH band, by and large, looks like white noise, doesn't have much structure in it. So for these four bands, for the first one the LL DPCM was used to encode the coefficients. And we talked about DPCM, it's clear what we are doing here. It's differential pass code modulation, so a prediction model was built to predict and then correct the intensity values of the low, low band. Now for the other bands, the combination of high and low frequency and high, high band, PCM was used. So, no transformation was done, but just reduced the number of bits representing the samples. And the rate was 1.6 bits per pixel. So here is the resulting encoded image. The PSNR is 36.5 dB, so it's definitely of high quality. As I mentioned, anything above 30 dB represents a very close representation of the constructed image, very close to the original one. Here's a second example of the application of multirate, now, sub-band decomposition. The original image. So it's split first into four sub-bands. And then for the low sub-band we further subdivide it into four sub-bands. So it's certainly multirate, it's a seven band decomposition, and also 17, I'm sorry 16 tap filters were used for this decomposition. Then here are the quantized sub-bands, the first sub-band, the low, low DPCM was used for that. And for the rest, PCM at 155 bits per pixel. And we see here the encoded one is of similar quality, slightly higher, higher quality than the previous one for a very similar bit rate. But certainly there is more computation that is involved when you go to more than four sub-bands. Here we have again seven sub-band decomposition. So we have reached the end of week nine. Three more weeks to go. 75% of the course is over. Congratulations to all of you. You should really feel proud about your accomplishment. During this week, the second week out of three dedicated to compression, we talked about lossy image compression. This is a very important topic, since we are surrounded by a plethora of applications depending on image compression. We discussed and analyzed the fundamental topics, and covered some of the intellectually interesting and practically useful techniques. We talked about the important concept of quantization, both scalar and vector, and covered DPCM, fractal encoding, transform encoding and JPG and finally we briefly covered sub-band encoding. Prediction and transformation of the data are two very fundamental and important topics encountered everywhere in signal processing. We presented an encoder in terms of three important building blocks, and tried to point out the differences and similarities of all techniques we described. There was nothing terribly mathematical in the material of this week, except maybe for the material on fractal encoding and the derivation of the optimal quantizer. So, I hope everybody feels comfortable with this material. The material, I believe, allows you to understand the image compression technology around you, use it, and also be able to potentially take the next step in pushing the envelope, and developing the next image compression technique. Next week, we'll talk about the system problem of im-, image compression, that of video compression. So, see you all next week.
