Chinese Journal of Computers   Full Text
Title	Building Panoramas from Photographs Taken with An Uncalibrated Hand-Held Camera
Authors	CHEN Hui LONG Ai-Qun PENG Yu-Hua
Address	(School of Information Science and Engineering, Shandong University, Jinan 250100)
Year	2009
Issue	No.2(328—335)
Abstract &   Background	Abstract This paper presents a method for building a full view cylindrical panorama from uncalibrated photos taken naturally with an ordinary hand-held camera held at an approximately fixed location. Such photos usually have large perspective distortion, small overlap, brightness differences, and camera rotations such as tilt and roll. These characteristics make both image alignment and panorama building more difficult than using photos taken by cameras calibrated by special equipment. To align such images, simple registration techniques cannot be applied directly. This paper shows how these images can be registered by a particular robust feature matching scheme, which contains robust information of gradient direction; this paper also proposes a reliable initial parameter estimation method as well as an iterative approach with linear steps to perspective transformation parameter optimization for pairs of adjacent images. Experiments show that the method yields good results even when the overlap region of adjacent photos is as small as 16%. Keywords image registration; panorama; image mosaic; feature matching; virtual reality Background Constructing a full view panorama of a 3D scene from a sequence of partially overlapping photos is a critical modeling task in building an image-based virtual reality system［1-2］. Such photos will also be referred to as images, views, or mosaics in the following. Photos can be captured by special equipment［3］, such as a panoramic camera or a fish eye lens camera with a large field of view, or by less expensive equipment such as an ordinary camera or video-camera［4］. Literature［5］ gives an comprehensive review of the panorama technique. If using ordinary camera, the problem is to stitch together adjacent views of a scene into an image having a larger field of view. Recently, a popular approach is to apply the classic optical motion estimation to find the perspective alignment parameters, see Szeliski and Shum［4］，the parameters are resolved by nonlinear optimization. The method is effective to most of the image sequences with identical lighting condition. However, with the limitation of the optical method itself, it is not applicable to images with large intensity changes, and it also needs other method to provide initial parameters. The authors also see other panaroma literatures published while our paper is in reviewing: they are FFT-based registration method［7］ and Brown’s［8］ SIFT-based method. The FFT-based method runs fast but it has low resolution and requires large overlap； Brown et al apply the invariant SIFT feature matching method for registration of adjacent image sequence. It is fully automatic and robust, but it requires to compute the histogram of gradient direction at each scale to construct the feature vector, which has relatively much higher complexity and very time consuming. Here, the authors present a simpler approach for feature matching that makes use of the robust information of gradient direction as well, but in a more efficient way. In this paper the authors assume that the photographer uses reasonable efforts to keep the camera at a fixed location while taking a series of natural photos. Tilting and rolling are allowed but should be kept to a minimum. Panning angles can be relatively large, but must be small enough to allow some overlap between adjacent views. They require an overlap of at least 16% of the width of each photo. The focal length of the lens should be kept the same for all photos. They believe that these requirements are not overly stringent or inconvenient to a photographer. The authors present here a new method for compositing natural photos, as most methods developed for compositing photos taken with special calibration devices are not applicable to natural photos. Our method consists of three main steps: (1) automatic identification of corresponding feature points; (2) computing a perspective transformation for image alignment from corresponding feature points; (3) mapping the panoramic mosaic onto a cylindrical model, after all adjacent images have been stitched together by applying steps (1) and (2).