A Full-Featured Open-Source Framework for Image Processing

rbf

Given a sparse set of keypoints, rbf reconstructs an approximation from a weighted set of radial basis functions, (RBFs), each taking an observed keypoint as a parameter. -rbf finds weights from a least squares fitting; the solution interpolates keypoints and estimates fits for locations in between these. -rbf can reconstruct approximations from about 1,000 keypoints taken from 1000 × 1000 images, the results dependent on both the sampling pattern and the particular kind of RBF used. Among other uses, G'MIC employs -rbf to compress color look up tables (CLUTs), where a sparse sampling of the original provides a compact representation. The behavior of the reconstructed CLUT differs little from the original. Another congenial setting for RBFs may be found in simulated neural networks. Here, RBF meshes integrate stimuli and respond to particular patterns by "firing" an output pulse.

Given two numerals, 3 and 7, one could guess at an in-between value of 5. On what basis does that guess sit? Well — none, really. It stems from some heuristic that, knowing nothing else about extremes then in-betweens are as likely as anything else. Thus the art of reconstruction: given (a very few) hints, interpolate the whole as so many averages. The technique here entails basis functions, a kind of simple-to-compute template function that scales, slides around and from a very few keypoints estimates what goes on in between them.

Radial basis functions (RBFs) are one such type of basis function, so-called because these functions, take only a radius r and return a "relative contribution". That reflects what a known keypoint contributes to an approximation at a radial distance r. This radius is itself a proxy for a difference that we take between two locations: the place where we are making our assessment, x, and the place where the sample has been "observed" c. A suitably weighted set of RBFs can approximate an entire image through a small set of keypoints so long as some loss is acceptable.

rbf, given a set of keypoints, returns a range image computed from the keypoints, as processed by the set of weighted RBF's. The keypoints may be drawn from one, two or three dimensional spaces and the range has a like number. We can take an inconveniently large data set — a color look-up table (CLUT) — and save selected keypoints, these consuming just a fraction of the storage of the original. Later on, we could reconstruct the CLUT from the keypoints and obtain something that almost matches the original. Is it the original? No. Does it work like the original? It could work in a pretty fair dinkum sort of way, with an adroitness to the sampling and an apt choice of RBFs; that is a deep topic upon the edge of which we teeter. For safety's sake, let's assume a sampling fair and RBFs robust. This brings us to particulars:

1.	We obtain keypoints — "spot-checks" — for an otherwise unknown image.
2.	-rbf plots approximations into one, two or three dimensional images: the range.
3.	Arguments to -rbf quantify the range for ·: x, y or z.
a.	d· arguments specify range dimensions and lengths; these are required.
b.	·0 arguments specify origin offsets; their absences imply zero offsets.
c.	·1 arguments specifies the dimensional lengths of the sample space; their absences imply that the sample and range spaces have equally sized dimensions and that scaling is 1:1.
d.	A final, optional string is taken to be a math expression and defines a custom RBF. Its absence implies the default RBF, the thin plate spline, described further on.
4.	The images referenced in -rbf's selection decorator are one dimensional vectors encoding keypoints. Each pixel encodes keypoints in spatial coordinate-sample value order; two coordinates and three values codify the position and color of an RGB sample, for example. Each datum occupies separate channels. -rbf separates coordinate from value channels by determining the number of dimensions of the output range. This arises from particular argument signatures:
a.	one, three or four arguments: one dimensional ranges. Signatures may consist of 1 argument: Only the length of the range: dx. They may consist of 3 arguments: Range length with origin offset x0 and the sample space dimensional length x1; dx:x1 sets scaling from range to sample space. They may consist of 4 arguments: Those before plus a custom RBF, written as a math expression.
b.	two, six and seven arguments: two dimensional ranges. Signatures may consist of 2 arguments: Only the length of the range in dx and dy. They may consist of 6 arguments: Range length in dx and dy, origin offsets, x0 and y0 and the sample space dimensional lengths x1 and y1. They may consist of 7 arguments: Those before plus a custom RBF, a math expression.
c.	nine and ten arguments: three dimensional ranges. Signatures may consist of 9 arguments: The length of the range in  dx, dy and dz, origin offsets x0, y0 and z0 and the sample space dimensional lengths x1, y1 and z1. They may consist of 10 arguments: Those before plus a custom RBF, a math expression.
5.	"Decompressing" a sparse set of keypoints into an approximation range entail finding weights for each of the RBF's, so that associated keypoints may be appropriately scaled. This is taken as a least squares problem; G'MIC composes a design matrix D by taking the absolute values of pairwise differences among the given keypoints and evaluating these with the chosen RBF. The design matrix which arises from this exercise is symmetric, the lower diagonal mirroring the upper. A linear least squares approach solves for the weights of the RBFs:

With weights found, -rbf performs its reconstruction, visiting every pixel in the range and estimating its magnitude from the weighted contributions of the given keypoints. The particular RBF chosen can have considerable influence on the character of these estimates, as shown in the following one dimensional reconstruction, all using the same six keypoints, but different RBFs:

1.	Blue dots		Six sample points
2.	Purple		Lerps the keypoints.
3.	Olive		Unscaled Gaussians.
4.	Light Green		Scaled Gaussians. The frontpiece animation illustrates a progression of ε from 1.0 → 0.01.
5.	Orange		Thin plate spline, the default radial basis function. Its behavior between keypoints is reminiscent of spring steel strips.

Examples

Reconstruct an image twice from 16 random samples, using the inverse multiquadric RBF, then the thin plate spline.

Reconstruct an image, first sampling it with a regular grid pattern, then reconstructing it with two different RBFs

Generate an orientation field from six random keypoints; reconstruct with the thin plate spline RBF. Use the field to draw some oriented ellipses. See Tiled Art

Command Reference

Notes

dx,dy,dz	Sets the output range, in pixels. dx, in isolation, implies a one dimensional output range. dx,dy implies a two dimensional output range. dx,dy,dz implis a three dimensional output range. Must be specified (no defaults).
-x0,y0,z0	Origin: Sets what output coordinate, ·0, corresponds to input origin 0. Provide as many origin coordinates ·0 as there are output ranges d·. Defaults to 0.
-x1,y1,z1	Scaling: d·:·1. Sets what input range, ·1, corresponds to output range d·  Provide as many input ranges ·1 as there are output ranges d·. Set ·1 = d· for 1:1 scaling from input to output. Set ·1 = 2*d· for 50% downscaling from input to output. Defaults to d· – 1.
phi(r) (math expression)	Set the radial basis function. The default, r^2*log(1e-5+r) provides the thin plate spline. The math expression recognizes the symbol r (radius) as representing the Euclidean distance  xy — c between image points xy and keypoints c. During evaluation of the basis function, G'MIC's math parser substitutes for r, wherever it may appear in the expression, the Euclidean distance. Other expressions may be provided, over-riding the default. Ensure that the math expression is written in terms of just one variable, r.