This command associates structure tensors with each input pixel, these recording intensity differences and their likely orientations. Such differences, if sufficiently coherent, can be regarded as 'edge-like features', hence the notion that this command makes explicit the 'structure' of an image.
The command produces a structure tensor field of its operand image; this may have three (with 2D inputs) or six (with 3D inputs) channels with each pixel in the output field containing the coefficients definining a structure tensor at the corresponding input pixel.
When the -display command renders structure tensor fields, detected edges are the dominant feature, appearing as thin lines not unlike other edge detecting methods. In contrast to those methods, however, -structuretensors also encodes the orientation of those edges. Faux coloring appears where blue indicate horizontal, red vertical, and white oblique orientation. These are visualization conveniences. The key output of this command is the structure tensor associated with each input pixel and the local image structure it conveys.
The structure tensor itself is, for images of one pixel depth, a 2x2 square symmetric matrix, and a 3x3 matrix of similar form for images of depth greater than one. The eigenvectors of these matrices furnish the orientation of local differences, while the eigenvalues furnish their magnitudes. These vector and scalar values are latent in the structure tensors. -eigen computes explicit scalars and vectors from given structure tensor fields.
To determine the local orientation of a feature, this command estimates the gradients along the width, height and depth axes in the immediate neighborhood of the pixel being assayed. The command's single numeric parameter. {0, 1, 2} chooses one of three estimating schemes:
The command builds structure tensors from intensity differences – gradients – detected in small, 3x3 pixel rectangles (2D) or 3x3x3 cubes (3D) centered on an assay pixel. The end result of the assay is a 2x2 (2D) or 3x3 (3D) symmetrix matrix, so-called because the coefficients in the upper right corner triangle mirror the values of those in the lower left. The schematic above depicts a 3x3 structure tensor. Providentially, the 2x2 structure tensor also appears as a sub-matrix in the upper left corner of the 3x3 matrix. When -structuretensors notes an image depth of one, it only builds the upper left corner.
The command finds the structure tensor coefficients, Sij, in two phases. The first entails taking differences. Scheme 0 differs from its two counterparts in that it takes only three 'centered differences', to wit: the differences of the immediate neighbors to the assay pixel along each cardinal axis, divided by two, yielding at least two differences with images of depth one, ix, iy, and a third difference iz for images of greater depth.
Schemes 1 and 2 take finer-resolution forward/backward differences involving the assay pixel itself. Under this regimen, the command takes "forward" and "backward" differences between the assay pixel and each of its neighbors along the cardinal axes, yielding at least four differences, ixf, ixb, iyf, and iyb for single pixel depth images, and izf, izb for images of greater depth.
In the second phase, the -structuretensors command takes various products of differences, which become the coefficients of the matrix. Products of differences are apt for detecting any variations in local image intensity. If these differences are small, the products are even smaller, but concomittantly amplify variations for differences larger than one.
For all schemes the main diagonal coefficients are set to products of differences taken only along one the cardinal axes, with S00 set to those along the X axis, S11 for those along the Y axis and S22 for those along the Z axis. For scheme 0, these are simply the square of the center difference, Schemes 1 and 2 employ different averaging methods of the products of forward and backward differences, details given in the preceding chart.
Off-diagonal coefficients arise from products of differences garnered from pairs of axes, the indices of the coefficient selecting particular pairs. For example, coefficient S02 and S20 arise from products of differences taken along the X and Z axes. Since the structure tensor is a symmetrix matrix, the computations for these two coefficients are identical, and, generally it is only necessary to calculate the main diagonal and one corner of off-diagonal coefficients. Scheme 0 off-diagonal coefficients are just the products of the two differences from each axis. Schemes 1 and 2 both compute averages of forward and backward differences from each axis. The preceding chart documents the particular calculations.
So far, this discussion of matrix computation suffices for gray scale images. The multispectral extension is straightforward. The command calculates the coefficients for each channel and sums these together for the image aggregate, as indicated by Σc in the schematic of the tensor matrix in the preceding chart.
VisualizationThe -display_tensors command visualizes 2x2 tensor fields, taking such as input operands. For each pixel (tensor) in the field, the command renders an ellipse with axes that are proportional to the two eigenvalues. The semimajor axis aligns with the steepest gradient in the locale and the eigenvector pointing along it.
Especially flat or highly eccentric ellipses reflect large differences in the magnitude of the two eigenvalues. The relative discrepancy of the two eigenvalues reflects the degree of anisotropy or bias toward a particular orientation. Rounder ellipses arise from eigenvalues closer together in value and reflect gradients that are not as strongly oriented in a particular direction.
The overall saturation and luminance of the ellipse reflect the magnitude of the largest eigenvalue and the rapidity of change of the local gradient.
In the preceding illustration: (upper far left) An image with edges, some edgier than others. (upper near left) After -structuretensors, in faux colors. For 2x2 structure tensors, the command stores coefficient S00 in channel 0, S01 in channel 1 and S11 in channel 2. With this arrangement, structure tensors around horizontal edges render blue and vertical edges render red, the obliques white or nearly black. The S01 coefficient may be positive or negative and ranges in values around a median of zero, so black - the color no particular edge - renders as a median gray. (near right and far right) -eigen reads structure tensor fields and produces two images, each with two channels. The first image stores the two eigenvalues, the second the components of the unit eigenvector pointing along the steepest gradient. The other eigenvector is rotated 90° clockwise, so its components may be easily derived. (bottom) -display_tensors visualization of the -structure tensor field, upper near left.
Computing structure tensors is the first stage in finding -diffusiontensors, which, in turn, furnish -smooth with datasets of blending vectors. Users primarily interested in anisotropic smoothing usually do not need to use either -structuretensors or -diffusiontensors directly, unless they wish to tailor these fields for specialized or artistic purposes.
Garry Osgood