What is reconstruction?

Reconstruction is the abstract "rebuilding" of something that has been torn apart. In the medical imaging context, it is often necessary to acquire data from methods that essentially "tear" data apart (or acquire the data one piece at a time) in order to be able to view what's inside. Also, a big part of reconstruction is then being able to view, or visualize, all the data once it's been put back together again.

Now this seems pretty abstract, but here are some real-world examples:

Serial-Section Microscopy- In Serial-Section Microscopy, the tissue being studied is sectioned into a number of slices and each slice is put into a microscope. Then, images are captured of each slice. To recreate how the tissue looked before we sectioned it, we must put all the images of all these slices back together again, just as if we were putting the real slices of tissue back together again.
Confocal Microscopy- In confocal microscopy, the microscope can obtain a single plane of image data without having to slice the tissue. In this case, we don't need to realign the images of the slices much, but just stack them back together and then visualize the result.
CAT- In CAT, the scanner acquires a number of projections, much like an X-ray, from different positions. Then, these different views through the object (or person!) must be "deconvolved" (meaning combined) to reconstruct the 3 dimensional object.
MRI- In MRI, the imaging device acquires a number of cross-sectional planes of data through the tissue being studied. Since all of these planes must be stacked back together to obtain the complete picture of what the tissue was like, MRI entails some amount of reconstruction and lots of visualization, too.

So, since it sounds like putting the slices back together is easier for confocal, MRI, or CT, then why do serial-section reconstructions at all? Well, it turns out that the smaller the thing you're looking for is, the more difficult it is to get an imaging technique like MRI or confocal to "see" it, because the information from surrounding areas blurs out what you're looking for (there are alot of other technical reasons, but let's just stick with this for the moment). So the smallest thing that MRI can "see" is about 1mm cubed. For confocal microscopes, the smallest object that's detectable is about 1/10 um (1/10000 mm). But once you slice the object up, you can use other forms of microscopy such as an electron microscope to be able to see objects almost as small as 1/10000 um (1/100000000 mm). There are even newer "atomic force" microscopes that even let you detect individual atoms, but not many people (as yet) have done 3D reconstructions at this minute level. The problem is, though that the smaller you go, the more artifacts can be introduced, so the reconstruction process gets much more difficult.

Deconvolution: Postprocessing 3D (from Kevin Ryan [kryan@cts.com])

An alternate method to confocal microscopy is reconstruction of the image data via deconvolution. Basically, you divide out the blur introduced by the system (usually a microscope). The data (d) is produced by the object (o) modulated through (convolved by) the optical transmission function of the imaging system, the point spread function (p). This is standard linear shift invariant filtering, resulting in:

                d = o (*) p, where (*) is a convolution.

In the frequency domain, given the Fourier of these items, the equation is:

                D = O * P, where * is a multiplication.

Deconvolution is the inverse operation:

                O = D * P^(-1)

In microscopy, where many of these techniques are implemented, it should be noted that the blur in Z is much greater than the blur in X and Y. Also note that this is an ill-conditioned problem: the inverse operation results in dividing by zero in many places. There are various signal processing approaches to this problem, but the end result is a noise limited approximation to the original object, with some spatial frequencies missing.

Full deconvolution is computationally expensive, but there are approximations that give decent results. The most common is 'serial plane deconvolution', where the images +/- in Z from the plane of interest are used to approximate the blur in the center plane, and the approximate blur is subtracted.

Full deconvolution, for more accuracy, is usually done in an iterative decent fashion, homing in on an answer O' that when convolved with P results in a close approximation of the data D.

Full deconvolution is accurate, but as you are doing 10-50 iterations of 3D Fouriers, it's slow. There is only one commercial product with full deconvolution I am aware of at the moment. Serial plane (or adjacent plane) deconvolution is less accurate in its reconstruction, but computationally cheaper - four 2D FFT's per plane for the first plane, two 2D FFT's per plane for each additional plane.

In microscopy, there are several players: Oncor [was BDS] (Macintosh), Vaytek (Mac and PC), and Scanalytics (PC, full iterative deconvolution). There are also apparently deconvolution packages for Khoros and some other imaging platforms.

Methods in Cell Biology Vol. 29 or 30 (sorry, can't remember at the moment) has a good review of the state of the art in this subject.

Deconvolution allows full field imaging with a standard camera, provided care is taken to account for uneven illumination and background bias. Deconvolution is better than confocal acquisition with small point objects (as they resemble the noise spectra of confocal acquisition), while it is weaker on large structureless objects (as it is dependent on image delta to calculate the object, and structureless objects give less to work with).

I won't go into the pros and cons of these packages; I wrote the BDS package and currently work for Oncor, and it would be a biased opinion...

Cryo-electron microscopy (Virus reconstruction) (from Stephan Spencer [sspencer@rhino.bocklabs.wisc.edu])

Cryo-electron microscopy in combination with image reconstruction can yield a three-dimensional structure that includes both surface and internal features and contains a large degree of structural detail (currently at resolutions as low as approximately 20 Angstroms). Image reconstruction is a computationally demanding process based on the assumption that each virus image from the raw data - the two-dimensional electron micrographs - is a two-dimensional projection of the same three-dimensional object. The reconstruction process takes advantage of the icosahedral symmetry of viruses in assigning an orientation to each projection. The projections are subsequently reconstructed into a three-dimensional array of variable electron density values. The array is then rendered using visualization techniques. Because of the assumption of icosahedral symmetry, icosahedrally symmetrical structures are reinforced and any non-icosahedrally symmetrical (e.g. flexible) structures pre sent on or within the particle are averaged out. The gain in overall detail is marked, despite a lack of detail in non-icosahedrally symmetrical structures.