virtual tour example

home gallery product info downloads buy now support learn more search

learn more

A fine tuning challenge

John Strait
March, 2001


Introduction

Recently a user wrote to me with this question:

After stitching I open each overlap and do a fine tune on each of the seams. I get each of them looking good with the tiles in place, then step to the next one using the dialogue box, it blends each one as I move to the next but when I have done them all and click on approve it goes back to looking a mess i.e. as it was before fine tuning.

I am new to this....what am I doing wrong?

This article examines the user's images and how he fine tuned them as a way to illustrate some potential difficulties with fine tuning and how to avoid them.

top


A challenging pair of images

Figure 1 shows two of the images in the sample stitching project.

imported1.jpg (2964 bytes) imported2.jpg (3144 bytes)
Figure 1 - A challenging pair of images

These images are particularly challenging for The Panorama Factory because of the relative lack of detail in the foreground snow and in the sky. The most significant part of the image from the point of view of the stitcher is the middle (top to bottom) and this is in the foreground snow! (This is by no means a criticism of the photos, just an explanation of how The Panorama Factory looks at them.)

It is harder for the software to correctly align low-contrast images and ones that lack detail. The fine tuner has particular problems with areas like the sky and foreground in thse images because particular fine tuning tiles may include insufficient detail to identify a match. This is the sort of thing that people do much better than programs. Of course, people find it much more tedious to do it themselves.

With images like these, you may find, in addition to poor fine tuning performance, that The Panorama Factory doesn't do a good job of aligning the images in the first place. Images like these often require more manual adjustment than ones that are rich in high-contrast edges.

Of course, most image sets will not experience these problems at all or to such a great degree.  Most images will stitch automatically with no problems.  But reading about these very difficult images should help you understand how to correct problems if they should arise, even if they are relatively minor.

top


How The Panorama Factory can go astray

Opening the overlap region shows three ways The Panorama Factory can go wrong.

First, the overall alignment (before fine tuning) is incorrect. When the alignment is correct, there should be very little image offset (between the left and right images) near the center of the overlap region and the offsets should increase outward from the center.

Figure 2 shows the overall alignment from the sample stitching project.  In this example, the alignment was dominated by the strong detail in the building at the left of the overlap.

auto_aligned.jpg (2674 bytes)
Figure 2 - Automatic alignment

To see the overall alignment in your own projects and to subsequently adjust it, you must first open the overlap region by clicking on the top or bottom edge of an overlap rectangle or double-clicking on one of its sides when the stitched image is showing (see the Stitch command).   Then you must clear the fine tuning with the Clear fine tuning command (Image menu). After clearing the fine tuning, you can see the whether the overall alignment is correct.

Second, The Panorama Factory probably chose the wrong value for focal length. When the focal length is correct, the image offsets should be relatively small across the middle section (top to bottom) of the overlap region.  But in this example, if we were to adjust the alignment so that it is correct at the center, the image offsets would be relatively large -- about the width of a window in the building at the left side of the overlap region. This happens because the automatic focal length refinement gives poor results when images are low-contrast or lack detail.

Finally, the automatic fine tuning produced incorrect results.  You can see this in the sky region where there is a discontinuity in the fine tuning results between the left and right portions of the overlap. In the lefthand portion there are relatively large fine tuning offsets. In the righthand portion the fine tuning offsets are close to zero.   Figure 3 shows the results from the automatic fine tuning in the example project.

auto_fine_tuned.jpg (5530 bytes)
Figure 3 - Automatic fine tuning

The fine tuning discontinuity resulted from the way that the The Panorama Factory's successive-subdivision fine tuning method works. After its initial division of the image into left and right portions, the fine tuner came up with very different results in the two regions. These different results were propogated into the successive subdivisions of the regions. You watch the subdivision in action by executing the Fine tuning command (Image menu) when the overlap region is open -- it shows the fine tuner operating in real time.

top


Why a little editing may not help

OK, suppose you open the overlap region of the sample project and remove and/or adjust a few fine tuning tiles until the result looks good. When you close the overlap region, the result may still look wrong.

For example, Figure 4 shows how you might edit a few of the tiles, and Figure 5 shows how what it might look like after you close the overlap region.  You can see that everything is mixed up right in the center of the overlap.

edited_fine_tuned.jpg (5526 bytes)
Figure 4 - Edited fine tuning

edited_result.jpg (3070 bytes)
Figure 5 - Edited result

 

So why did the result turn out the way it did? Because there were still many inappropriate tiles in your overlap region. These were, in all cases, tiles that were created by the automatic fine tuning.  To understand why this matters, you need to know a little about how The Panorama Factory uses the fine tuning tiles.

Each fine tuning tile defines how the images must align at the very center of the tile. The Panorama Factory deforms the overlapping images to fit each tile. The fit is perfect at the center of each tile. Between tiles the images are deformed smoothly so that all tiles are fit exactly.

Here is the way I envision it: Take a sheet of rubber and lay it on a flat surface. Insert a thumbtack through the sheet of rubber to anchor it down. Stretch the sheet of rubber and insert another thumbtack. Stretch it again and insert a third thumbtack. And so on. Each thumbtack represents a fine-tuning tile.

The problem is how to deform the image so that all tiles fit exactly in their centers. Between tiles, the image deformation is directed by the influence of the tiles. Nearby tiles have a greater affect, and distant tiles have less effect. Although I think of this as like a sheet of rubber with thumbtacks, the deformation is not the same as you would get with this physical model.

The image alignment is absolutely correct at the center of each tile you drew yourself. However, the inappropriate tiles cause the image alignment to become more and more incorrect as you move away from the centers of the correct tiles. Because there are so many incorrect tiles, and they are in such close proximity to the correct tiles, the overall result is dominated by the incorrect ones.

top


What you can do to get better results

To get good results with images like these, you may need to control all of the steps yourself.

  1. The Panorama Factory's focal length refinement procedure gave me a value of 24mm working with just one pair of images. Using the manual technique to determine focal length described in A manual procedure to determine focal length, I arrived at a value of 28mm (the original project was stitched with 34mm). Using 28mm gives better alignments with smaller offsets away from the centers of the overlap regions. Perhaps there is a better value than 28mm based on all of the images.
     
  2. Disable automatic fine tuning. For images with large areas of low contrast or large areas that lack detail, manual fine tuning may be faster and will probably give better results.
     
  3. Open each overlap region in turn and adjust the overall alignment so that it looks best at the center of the overlap.  Figure 6 shows the correctly aligned images.

correctly_aligned.jpg (2940 bytes)
Figure 6 - Correctly aligned but not fine tuned yet

  1. Draw fine tuning tiles over the image to fine tune the alignment. Be sure to distribute them over all areas where you care about the alignment, including areas that are already correctly aligned. Remember that the centers of the tiles are important to defining the image deformation, not the sizes of the tiles.  Figure 7 shows the correct fined tuning.

correctly_fine_tuned.jpg (3237 bytes)
Figure 7 - Correctly fine tuned

When you get everything right, the result looks pretty good.  Figure 8 shows the result of correct focal length, correct alignment and correct fine tuning.

final.jpg (3415 bytes)
Figure 8 - Correctly aligned and fine tuned

top


Revised: April 28, 2004

 1999-2005 Smoky City Design, LLC and John Strait