In this video we will discuss Spatial Operations in Image Processing In this video we will review: Convolution sometimes called Linear Filtering, Edge Detection, Median Filters In this video we will show the one channel representation, but unless specified we can apply these operations to each channel independently Spatial Operations consists of a neighbourhood, in this example we take a neighbourhood of 2 by 2 pixels we apply a function that involves each pixel in the neighbourhood and output the result we then shift the neighbourhood and repeat the process for each pixel in the image The result is a new image that has enhanced characteristics, these spatial Operations take advantage is spatial patterns in the image Convolution or linear filtering is a standard way to Filter an image the filter is called the kernel the different kernels perform different tasks Convolution is analogous to this linear equation the input is the image X output will be another image Z, we have the parameter W this is known as a kernel or filter The star this operation is the convolution operation, there is actually an analytical expression for it, but it's a little confusing, so let's just go over an example to understand what's going on. here's our image X, here's our kernel w, it’s just a smaller array The array is made up of values that have been pre determined for some operation. Let's go over the operation of convolution, We start off in the top right corner of the image, and we'll overlay the kernel with that region of the image. We see the kernel values in red, we'll multiply every element of the image by the corresponding element of the kernel. For the first row we'll get the following operation, multiplying the intensity value and summing the results this process is repeated for the second row, multiplying the intensity value and summing the results. Finally for the final row we multiplying the intensity value and summing the results, we sum those elements together. The result is the first element of the image z We shift the kernel to the left represented by the different colors in red. We multiply all the elements of the kernel with the image. This gives us the second element of the output image z We'll shift the kernel one more column and perform the exact same operation, multiplying every element of the kernel by the corresponding intensity value of the image, adding them all together. This gives us the next value We then shift the kernel down and repeat the process until we get to the end column. Repeating the process until we get a new image One problem is the images are different sizes we can fix this my changing the size of the image x We change the size of the image by padding. In zero padding we add two additional rows of zeros and two additional columns of zeros. We can also replicate the values of the border. Low Pass Filters are used to smooth the image getting rid of noise Consider the following image of a square. We have the following noise pixel, we can reduce this noise by applying a smoothing filter or kernel. Smoothing filters average out the Pixels within a neighbourhood, they are sometimes called low pass filters. For mean filtering the kernel simply averages out the kernels in a neighbourhood. Let's see what happens to the pixel intensities
To explore how the kernel effects different areas we can plot the output image and its relationship to specific regions of this input when we apply the kernel to this area of the image the intensity values remain unchanged around the edge of the box the values change as the values of 255 are averaged out with the zeros Finally, when we get to the region with the noise pixel, we see the noise value is smaller Comparing the original image and the output image we see the intensity of the noise has reduced but the edges appear less sharp In general there is a trade off between sharpness and smoothness, in the lab we will review several other kernels that explore this trade off Edge Detection is an important first step in many computer vision algorithms Edges in a digital image are where the image brightness changes sharply. Edge detection uses methods to approximate derivatives and gradients for identifying these areas. Consider the following image with a vertical and horizontal edge. Let's take a look at the intensity values Let’s plot the first row of the image, the horizontal axis is the column index, and the vertical axis is the intensity value If we move in the right direction from pixel 1 to pixel 2, the intensity value increases. In the image, we can represent this as a vector, in the same manner, when moving from pixel 2 to pixel 3, the intensity value decreases we can represent this as a vector pointing in the opposite direction. The direction of the vector represents if the adjacent pixel is increasing, we can represent this change by applying the following difference equation, and this works by subtracting the intensity value of the adjacent columns j and j+1 in row I. This computes an approximation of the gradient in the x-direction. In the following table, each row applies the equation to the intensity values for column 1 and 2, the final column is the result of using the equation. We can overly the result over the image as vectors We can apply the same logic to the next column, this time the values are negative It turns out we can perform similar Horizontal Derivative Approximations using convolution; the Horizontal changes or gradient are computed by convolving the image with a kernel. These kernels are called Sobel operators. We can represent the output in the array of intensities G x, all the values are zero except the elements corresponding to the horizontal edges, in this image the gray values have different ranges where black is negative, gray is zero, and white is positive. These represent vectors we discussed earlier We can use the same process to find vertical changes. This is computed by convolving the image with a kernel to get the image "G y " This time the vectors point in the vertical direction We can combine the output of each filter into a vector, if you are familiar with calculus this approximates the gradient. Taking the magnitude of the vectors we get the intensity values, we can plot it as an image this represents the edges, we can also calculate the angle representing the direction Median Filters are another popular filter, they are better at removing some types of Noise but may distort the image The median filter outputs the median value of the neighbourhood, consider the yellow pixel, consider the region in this three by three neighbourhood the resultant value of the output image is the median value of the 9 pixels Depending on the padding we see the median is identical to the image in most regions In the region with the noise sample we see the output is as follows Overlaying the image values unlike the median filter we see the noise is no longer there and the edges are straight, but the square is distorted as its missing a pixel Let’s apply some Spatial Operations in open cv, PIL is relatively simple so check out labs for PIL examples We will create an image “new image” that is a noisy version of the original first we create a kernel for mean filtering. The function filter2D performs 2D convolution between the image and the kernel on each color channel independently. The parameter values are in the lab we see the new image has less noise, but the image is blurry Image Sharpening involves smoothing the image and enhancing the edges. We can accomplish image sharpening by applying the following Kernel We then apply fillters2D, comparing the image before and after we see it is much crisper
We can perform edge detection consider the following image. We smooth the image using GaussianBlur, a low pass filter we will be discussing in the lab; this decreases changes that may be caused by noise that would affect the gradient. We can approximate the derivative in the X or Y direction using the Sobel function “ddepthis” is the output image depth, dx, dy is the order of the derivative in each direction in this case it will be one. Finally, “k-size” size of the extended Sobel kernel; we can output the gradients as images We can approximate the magnitude of the gradient. We calculate absolute values, and convert the result to 8-bit using convertScaleAbs Then apply the function ”addWeighted” to calculate the sum of two arrays and assign it to grad we plot Grad as an image the areas with high intensity values represent edges. Check out the labs for more