Homework 1

Implement and Experiment with Canny Edge Detector

We discussed in class today the Canny edge detector. You can also get the original paper from Canny and from page 71 in the Trucco/Verri book. Implement all steps of the Canny Edge detector, apply it to the images that you found on the web. Play with the parameters. Plot the edge images for each example image. Here is a summary of all steps:

• Compute the gradient at each pixel location using the 2D Gaussian Derivative filter
• Mark edge pixels that are local maxima along gradient direction, using "non-maximum-suppression". You can either just check along one of the 4 discretized orientations (X, Y, or 45 degree diagonal) that is closest to the gradient direction, or use a more elaborate check.
• Check if gradient magnitute is above threshold H.
• Include all neightbor edges that are above threshold L. (as briefly discussed in class). This step is optional and is worth extra credit if it works well.

You might find the matlab diaries from class useful for this assignment: (look at top-level class page for latest links)

Deadline for electronic submission is 5pm Thursday, Oct 14, 2004

2nd Part of this HW: Implement and experiment with "Lucas-Kanade" flow estimation

If you use matlab do following steps:

• We provide example image sequences from Michael J. Black's web site at Brown. We will be working with the SRI tree sequence. Also, a test sequence is provided. We have converted the images from the web site above into the matlab readable png format for you to use: SRI dataset, Test dataset
• Allow user to mark region of interest. (either use ginput and a fixed window size or specify a tracking mask with arbitraty shape with mask = double(roipoly);
• Track that area over the multiple frames. Use the matlab plot functions to visualize your results.
• If your code doesn't work, debug it with artifical input images that you generate.
• On your web site, post results for the SRI image sequence, marking the region of interest in the first frame and in successive frames.
• If you have extra time, you can try adding the ability to track regions with affine transforms, rotations, and scalings.
Deadline for electronic submission is 5pm Thursday, Oct 21, 2004

Outline of "Lucas-Kanade" flow estimation using Matlab. (See class page for the Lucas-Kanade paper)

Step 0 (Initialization):
Label in the first image of your sequence (Image F) the region of interest (ROI). Either pick a center + some rectangular area with imagesc(im); xy = ginput(1) or choose an arbitray ROI shape with imagesc(im); roi = double(roipoly);

In order to get all the indices of the region of interest, you can use inds = find(roi); . For example to sum over all Fx you can do sum(Fx(inds).^2) .

Fx(x,y) is the spatial gradient along the x-axis of image F at location x,y.

Fy(x,y) is the spatial gradient along the y-axis.

You can compute those gradients with the convolution we discussed in class. Pick a sigma (start with 3 for the first few iterations and then 1.5), and compute the convolution kernel. You can create that kernel by computing two vectors: gauss (the gaussian function), and dgauss (the first derivative of the gaussian function). Compute the outer product: kernel = gauss * dgauss' . Here's what I get if I do that for sigma = 2 and plot it in matlab with surf(kernel):

Convoluting an image with this kernel can be done using conv2() in matlab. You compute Fx with kernel=gauss * dgauss', and Fy with kernel=dgauss'*gauss;

Ft (the temporal gradient of the iamge) can be computed in just substracting G-F.

Step 3 (Re-warping). Rewarp G (or F). Remember to rewarp the original image each time. Do not rewarp a rewarped image. The rewarping should be done in sub-pixel accuracy. You can do that with interp2(... ,'cubic'); The XI and YI coordinates (as specified in matlab help) can be computed in the following way: first use meshgrid() and then add to them the new U and V.

Step 4 If error too high, goto Step 1.