CS180 Project 2: Filters and Frequencies

Name: Shuai Victor Zhou

SID: 3035912996


Overview

In this project, we first utilize filters and frequencies to adjust our images. With filters, we can create outlines of photos by highlighting the edges. Next, by playing around with frequencies and using low-pass and high-pass filters, we can generate sharper images, hybrid images, or even blended images.


Finite Difference Operators

We define the following two operators:

Dx=[11],Dy=[11],D_x = \begin{bmatrix} 1 & -1 \end{bmatrix}, D_y = \begin{bmatrix} 1 \\ -1 \end{bmatrix},

with D_x calculating our photo’s “rate of change” (in terms of pixel values) in the x direction, and D_y calculating the rate of change in the y direction. By convolving our images with these two kernels, we’re able to identify vertical and horizontal edges. Next, we combine these two gradients by taking the L2 norm, or

imagedx2+imagedy2.\sqrt{image_{dx}^2 + image_{dy}^2}.

Finally, we can binarize the L2 norm by normalizing the image (mapping all pixel values to be in the range [0, 1] and then binarizing the image (mapping all values below a threshold to 0 and the others to 1).

Original photo
Dx
Gradients combined
Dy
Binarized

Derivative of Gaussian (DoG) Filter

We notice the noise in the results when we directly convolve our image with the kernels, and so this time, we first convolve the image with a Gaussian filter before repeating the steps in the previous part. This convolving with a Gaussian filter effectively blurs our image without affecting the sum of its pixel values, allowing us to preserve the image.

Dx
Gradients combined
Dy
Binarized

Not only does this resulting binarized image have less noise than our previous result, but it’s also more smoothed out with less jagged parts to it. This can be attributed to the blurring on the image smoothing things out for us.

We can also convolve our Gaussian filter itself first with D_x and D_y before convolving the image with both results.

Dx
Gradients combined
Dy
Binarized

Comparing the two binarized results here, the images seem more or less the same, though the second one has a little more noise (still less than our original result). This may be due to more operations being done, resulting in greater exponential growth of our noise.


Image “Sharpening”

The Gaussian filter we described in the previous part can also act as a low-pass filter. This means we can attain the high frequencies of an image by subtracting the low-passed image from the original image, or

ImgHigh-pass=ImgImgLow-pass=ImgGaussian(Img).\text{Img}_{\text{High-pass}} = \text{Img} - \text{Img}_{\text{Low-pass}} \newline = \text{Img} - \text{Gaussian}(\text{Img}).

By adding our high frequencies times some alpha back to our original image, we can make our images seem “sharpened.”

ImgSharpened=Img+αImgHigh-pass\text{Img}_{\text{Sharpened}} = \text{Img} + \alpha \cdot \text{Img}_{\text{High-pass}}
Original photo
α = 0.75
α = 2.0
α = 0.5
α = 1.0
α = 5.0

Original photo
α = 0.75
α = 2.0
α = 0.5
α = 1.0
α = 5.0
Original photo
α = 0.75
α = 2.0
α = 0.5
α = 1.0
α = 5.0
Original photo
α = 0.5
α = 1.0
Original photo blurred
α = 0.75
α = 2.0

When we blurred our photo of nature and resharpened it, it seems similar to the original. However, the blurring made us lose some information which couldn’t be reattained with our sharpening, and so the result isn’t the complete same as the original — this results in a still slightly blurry look.

For visualization purposes, here are the high frequencies for our images:


Hybrid Images

To generate hybrid images, we use the Gaussian filter as a low-pass filter again. Taking in two images, we first align the two to ensure that they are lined up well for “merging.” Next, we take the low frequencies of one image, the high frequencies of the other image, and then we add them together. The result is our hybrid image, where if we’re close to the image, we see the photo of which we have the high frequencies; if we’re far away, we see the photo of which we have the low frequencies.

Below are some of our results; the left image is the image which we take the low frequencies of, the middle one is the image which we take the high frequencies of, and the right image is the result.

Low-pass
High-pass
Hybridized result
Low-pass
High-pass
Hybridized result
Low-pass
High-pass
Hybridized result
Low-pass
High-pass
Hybridized result
Low-pass
High-pass
Hybridized result
Low-pass
High-pass
Hybridized result
Low-pass
High-pass
Hybridized result
Low-pass
High-pass
Hybridized result

Let us further examine this last result. Below is the log magnitude of the Fourier transform of the input images, filtered version of these images, and the final result.

Original low-pass image
Original high-pass image
Filtered low-pass image
Filtered high-pass image
Hybridized result

If we examine the hybridized image of Erlang and Wukong, it seems to be a bit of a failure. It is difficult to make out both images without seeing a bit of the other one, making it hard to parse the frequencies apart from each other.

Failed hybridization of Erlang and Wukong

Gaussian and Laplacian Stacks

To create our multiresolution blending, we first have to create Gaussian and Laplacian stacks for images. The Gaussian stack is defined such that the 0th element is the original image, and then each subsequent ith element is the result of the Gaussian filter applied on the (i-1)th element. Our Laplacian stack is defined such that for a Gaussian stack of length n, our Laplacian stack’s ith element for i in [0, n-2] is equal to Gaussian[i] - Gaussian[i + 1]. At the end of our Laplacian stack, we append the last element of our Gaussian stack; this ensures the two stacks are the same length. For this project, I made the stacks to be of length 8. Consider the following two images:

Below are the results in the Laplacian stack for the two images at the chosen levels of 0, 2, 4, and 7:


Multiresolution Blending

Having created our Laplacian stacks for our images, we now choose a mask that allows us to create a blending. Our mask should generate the boundaries upon which we choose to blend the photos. In the case of the “oraple” (orange + apple), we want the left half to be the apple and the right half to be the orange. The corresponding mask for this would be an image with all white on the left half and all black on the right half.

Similar to what we did for our selected images, we also have to generate a Gaussian stack for our mask. Once we have our mask’s Gaussian stack, we can generate the resulting blended image’s stack like so:

resultImageStack[i]=maskGaussian[i]firstImageLaplacian[i]+(1maskGaussian[i])secondImageLaplacian[i]\text{resultImageStack}[i] = \text{maskGaussian}[i] \cdot \text{firstImageLaplacian}[i] \newline + (1 - \text{maskGaussian}[i]) \cdot \text{secondImageLaplacian}[i]

Using our mask, we get the following oraple:

Mask
Oraple

By using the mask, we can also generate the following results:

Image 1
Image 2
Blended result
Image 1
Image 2
Blended result

For the blend of Bronny and Boruto, here are the results from the stack at the chosen levels 0, 2, 4, and 7:

0
2
4
7

We can also utilize a custom mask to make a beautiful even more gorgeous by placing a Costco in it:

Image 1
Image 2
Mask
Blended result

Conclusion

This project taught me about how low and high frequencies are perceived by the human eye, and I learned how to manipulate the two using low-pass and high-pass filters to alter how we process images. I also now know how to create tools that allow the endless generation of memes!