Image sharpness in stitched panoramas

John Strait
December, 1999

• Introduction
• Images used in this article
• Sources of blurriness in panoramic images
• Strategies for improving sharpness
• What sharpening filters do
• Other stitchers sharpen automatically
• Controlling the blend region
• Conclusion

Introduction

Some users have expressed concern about a perceived lack of sharpness in images produced by The Panorama Factory when compared to images from other panoramic stitchers.

This article investigates the apparent lack of sharpness in The Panorama Factory images by comparing a panoramic image made with The Panorama Factory to the same image made by another panoramic stitching application.

It will be seen that the apparent lack of sharpness in The Panorama Factory images is simply due to the fact that the other panoramic stitching application automatically applies a sharpening filter whereas The Panorama Factory does not.

top

Images used in this article

The panoramic images used in this article were made by James Rigg, the author of The Guide to Panoramas and Panoramic Photography.  The panorama was made from 10 frames shot using a 35mm film camera on a tripod but without a pano-head.  The images were scanned from prints.

• The "effective" focal length of the images is greater than 35mm, possibly as much as 38mm.
• The images may suffer from parallax.
• The day was breezy with the result that some of the foliage on the bushes and trees moved between on image and the next.  Different applications stitch in different ways and therefore may seem to produce better or worse results because of this.

These images are © 1999 by James Rigg.   Unauthorized reproduction is a violation of copyrights.

The only image processing applied to the images was to crop them and to apply the Panorama Factory's sharpening filter.

Figure 1a - An image stitched by another application

Figure 1b - Image stitched by The Panorama Factory but not sharpened

Figure 1c - Image stitched by and sharpened by The Panorama Factory

top

Sources of blurriness in panoramic images

Making panoramic images from multiple images can be a tricky business.  Attention to detail is important.  There are many sources of  loss of sharpness in the resulting image.

First, and perhaps most significant, small misalignments between adjacent images can lead to loss of sharpness in the overlap region.  There are quite a few source of misalignment:

1. Wrong focal length setting
2. Distortion introduced by imperfect lenses
3. Parallax (camera not rotated around its nodal point)
4. Camera tilted from level in either of two axes (pitch and roll).
5. Camera moved between images.

A second source of blurriness is image resampling.  A single image resampling step is unavoidable when using an ordinary camera with a flat film plane.  The images must be reprojected onto the surface of a cylinder or sphere so that they will match correctly.   (The Panorama Factory uses spherical projection).

Each consecutive image resampling operation introduces more blur.  Depending on how you make your panoramas, there may be as many as 5 resampling steps:

1. Trimming/rotating scanned images to extract rectangular images from a photographic print or a strip of negatives.
2. The spherical or cylindrical warping.
3. Small-scale image warping to compensate for local misalignments.
4. Rotation of the stitched result to make the horizon line horizontal.
5. Resizing the image.

For an excellent discussion of how multiple generations of image resampling produces blur, see Prof. Helmut Dersch's paper Testing Interpolator Quality.

top

Strategies for improving sharpness

The Panorama Factory uses three strategies for improving image sharpness.

1. Small-scale image warping to compensate for local misalignments,
2. Sharpening filter.
3. Control the width of the blend region.
4. Recalculate the final image in a single generation.

For more information about how to use the first strategy, see The Panorama Factory help topics Fine tune command and Fine tune the stitched image.

The last strategy, recalculating the final image in a single generation, is discussed in the article Improving the quality of multi-generation images.

The remainder of this article discusses the use of a sharpening filter.

top

What sharpening filters do

Sharpening filters work by increasing contrast at edges.  Figure 2 shows how a little sharpening can improve the appearance of an image.

Figure 2a Detail, unsharpened

Figure 2a Detail, sharpened

The most common sharpening filter use a neighborhood of 3x3 pixels.  For each output pixel it computes the weighted sum of the corresponding input pixel and its 8 surrounding pixels.   The weights are positive for the central pixel and negative for the surrounding pixels.  By arranging the weights so that their sum is 1, the overall brightness of the image is unaffects.

For example, Table 1 shows the weights used by the default sharpening filter in The Panorama Factory

Table 1 - Default sharpening filter

Figure 3 shows four gray rectangles unsharpened and then sharpened with The Panorama Factory's sharpening filter with the central value set to 2.0, 3.0 and 10.0.

Figure 3a Four gray squares

Figure 3b Sharpened 2.0

Figure 3c Sharpened 3.0

Figure 3d Sharpened 10.0

Figure 4 shows the images from Figure 3 magnified by a factor of 3 to better show the sharpening artifacts.  Notice that transitions develop a light fringe on the lighter side of the transition and a dark fringe on the darker side.  This edge enhancement produces the effect of increased sharpness by highlighting edges. 

Figure 4a Four gray squares

Figure 4b Sharpened 2.0

Figure 4c Sharpened 3.0

Figure 4d Sharpened 10.0

These images also show what happens with excessive sharpening.

top

What sharpening filters do to noisy images

Sharpening filters have a definite downside.  If your image is noisy, sharpening it will accentuate the noise.  This is particularly noticable with JPEG image.   Figures 5 and 6 show the effect of sharpening the gray rectangles after they have been stored in JPEG format.

Figure 5a Four gray squares (JPEG)

Figure 5b Sharpened 2.0

Figure 5c Sharpened 3.0

Figure 5d Sharpened 10.0

Figure 6a Four gray squares (JPEG)

Figure 6b Sharpened 2.0

Figure 6c Sharpened 3.0

Figure 6d Sharpened 10.0

top

Other stitchers sharpen automatically

Most other stitchers sharpen automatically to hide the blurriness introduced by the spherical or cylindrical warping.  Figure 7 shows several versions of a detail section of the panorama.  Click on any of the images to display a magnified version.

Figure 7a shows a detail from the original, unprocessed image at the righthand end of the panorama.  Figure 7b shows the output from another popular stitching application.

Figure 7c shows the output from The Panorama Factory before sharpening and Figure 7d shows the result of sharpening with the central value set to 2.0.

In Figures 7b and 7d you should be able to see the telltale fringing at the edge between the light colored pavement and the dark wall at the lower right.  Note also the increased JPEG noise in the pavement and in the green garage door.

Figure 7a original image

Figure 7b stitched result from another application

Figure 7c stitched result from The Panorama Factory

Figure 7d stitched and sharpened result from The Panorama Factory

top

Controlling the blend region

Where an image pair overlaps, panoramic stitchers blend the two images to produce a gradual transition from one image to the next.  The width of the overlap is determined by the relative aligments of the images.  The size and placement of the transition from one image to another may cover the entire overlap region or it may be narrower.

One other popular panoramic stitching application uses a very narrow blending region, only a few tens of pixels wide.  It also has a smart method for making the blend line wander so that it follows a path of "least blurriness."

You can adjust the width of the blend region in The Panorama Factory to get a similar effect.  For example, changing the righthand edge of the blend region as shown in Figure 8a and 8b improves the appearance of the brick between the first and second story windows.

Figure 8a default blend region

Figure 8b narrow blend region

Figure 9a shows the overlap region as produced by another panoramic stitching application.  Figure 9b shows the overlap region as produced by The Panorama Factory with no sharpening.  Figure 9c shows the result of sharpening with the central value set to 2.5 and Figure 9d shows the sharpened result after narrowing the blend region.  These figures show that, through manual control, The Panorama Factory produces an image that is superior to the one produced by the other application, even in the blend region.

Figure 9a from other stitcher

Figure 9b from The Panorama Factory

Figure 9c from The Panorama Factory, sharpened

Figure 9d from The Panorama Factory, narrow blend region and sharpened

top

Conclusion

Panoramic stitching applications use a variety of strategies to control the loss of sharpness that is an unavoidable result of image processing.  Most panoramic stitchers automatically sharpen the stitched image.  Automatic sharpening is undesirable if you need to rescale, rotate or otherwise post process the stitched image because it cannot be reversed nor reapplied without creating artifacts. Sharpening should be applied only as the final image processing step.

The Panorama Factory provides several strategies for controlling loss of sharpness.   The sharpening filter is only one of them.  Adjusting the blend region, fine tuning to correct misalignments and recalculating the final image in a single generation can also be used to achieve the highest quality result.  The Panorama Factory provides manual control over all image processing features in order to leave the final esthetic judgement to the user.

By showing how manual sharpening and manual adjustment of the blend are used to control image sharpness, this article has shown that images produced by The Panorama Factory compare very favorably with images produced by another popular panoramic image stitcher.

top

Revised: May 16, 2000