CS 194-26 Project 5 - Depth Refocusing and Aperture Adjustment with Light Field Data

Austin Luong

Overview

One of the major issues with photography is that is that the parameters such as the depth of field and aperture size must be set as desired when the photo is taken. But as this paper by Ren Ng demonstrates, capturing multiple images over a plane orthogonal to the optical axis enables effects such as depth refocusing and aperture adjustment in post-processing using simple operations such as shifting and averaging. The images used in this project are taken from the Stanford Light Field Archive.

Depth Refocusing

Using the grid of light field images, it is possible to create a realistic depth refocusing effect with simple shifts and averages. The key idea behind the technique is that which are far away from the camera do not vary their position significantly when the camera moves around while nearby objects vary their position significantly. As a result, averaging all the images in the grid without any shifting will produce an image which is sharp around the far-away objects but blurry around the nearby ones. Similarly, shifting the images 'appropriately' and then averaging allows one to focus on object at different depths.

Algorithm

The general algorithm is surprisingly simple. Let the grid of images be indexed by i, j. A translation factor (u[i, j], v[i, j]) is computed for each image[i, j]. Each image[i, j] is then shifted by C * (u[i, j], v[i, j]) where C is some specified constant that controls the focus depth. The resulting image is formed by averaging each of the shifted images. Below are some of the results.

Chess Board

C = 0 (this is the same as just averaging all the images)

C = 0.1

C = 0.2

C = 0.3

Bracelet

C = -0.1

C = 0

C = 0.1

C = 0.2

Treasure Chest

C = -0.2

C = -0.1

C = 0

C = 0.2

Animated Refocusing

I also created some animated refocusing gifs by varying C in small steps and combining the resulting images into a gif. Looks like what happens when you refocus the camera in person.

Animated refocusing of the chess board. C has values from -0.3 to 0.7 in 0.005 step intervals.

Animated refocusing of the treasure chest. C has values from -0.3 to 0.7 in 0.005 step intervals.

Aperture Adjustment

It is possible to mimic a smaller and larger aperture by taking averaging more and more images starting from the center of the image grid. For example, taking just the center image in the grid would be considered a aperture size = 0 image. A aperture size = 1 image would created by averaging the center image and the images within a distance of 1 away from it (so the cetner 9 images). Below are some of the results. Notice that the pictueres get blurrier with a larger "aperture size" just as they would when taking images with a camera with a larger aperture size.