Table of Contents  ▸  Managing 3D Vector Objects

◀  Installing the G'MIC-Qt Plug-in For 8bf Hosts    |    Scientific Publications  ▶

Managing 3D Vector Objects

G'MIC is able to create, manage and display 3D vector objects. Although it may be considered as a minor feature for an image processing framework, it has in fact many implications in the way new commands can be developed, even for applying effects on usual 2D images.
We describe here the various aspects of this interesting feature.

1. In which case do I encounter 3D vector objects ?

Basically, 3D vector objects in G'MIC can be generated from one of these sources :

1.1. Reading a file (.off, .cimg[z] or .gmz formats)

For now, there are very few file formats storing 3D vector objects that G'MIC is capable of loading/saving:


Here is a simple example of loading a 3D vector object, and displaying it with the command line tool gmic of G'MIC:

1.2. Extracting a 3D object as an image feature

G'MIC has some commands to transform/extract/convert features of regular images as 3D vector objects. For instance, the command imagecube3d creates a 3D cube where the elected image has been mapped on each of the 6 faces of the cube.
Command elevation3d generates a 3D vector object that represents a 3D elevation of the selected image, as shown below:

1.3. Generating 3D objects from scratch

Some other commands are able to generate 3D vector objects "from scratch", without needing additional image data. For instance, commands torus3d or circles3d are able to do that:



Look at the 3D Meshes section of the reference documentation to see all available commands to create 3D objects from images or from scratch.

2. How 3D vector objects are internally stored.

2.1. Why should I care about this ?

In G'MIC, a 3D vector object is stored as a regular image. The only thing is, this kind of images has a special structure (image size and ordering of pixel values) that makes it recognizable as a 3D vector object, by the G'MIC interpreter. As a consequence, anyone can write custom commands able to generate 3D objects, as such objects are not different than any other image. Knowing this allows to quickly create complex 3D objects, for instance without having to use explicit loops (which are bottlenecks for performances in many interpreted languages). As 3D objects can be drawn into images (with command object3d), it becomes also interesting to use them instead of using loops even for dealing with 2D images. We will show an application of the use of 3D objects to create a custom (and fast!) version of the histogram command afterwards.

2.2. The different parts of a 3D vector object.

What makes a 3D object data recognizable from an usual image ?
First, it is stored as one-column scalar image, with dimensions 1xNx1x1 (where N is usually very big). Then, they are basically vectors which contains all the necessary informations to entirely define a 3D vector object.
These informations appear in this order, to define the 3D object features :


Each of these 6 parts composing the whole 3D vector object are detailed below.

Before going further, one has to notice that the command split3d is exactly able to split any 3D object into 6 different images that correspond to these different parts of the object. Using it on an existing 3D object is a really interesting way to do some reverse engineering of the data structure, and see how the object has been encoded as different parts in a one-column image:

2.1. Magic numbers

The magic numbers define a sequence of 6 numbers which represent a header for recognizing an image as a 3D vector objects in G'MIC. These numbers are actually the ASCII codes of the string CImg3d, eventually with a float-valued part (usually, 0.5 is added to each of the ASCII values for being even more random and unlikely to appear in real image data). To get the exact values for these ASCII codes, you can use G'MIC of course :)


which means that the magic sequence of numbers is

(+epsilon, with 0<=epsilon<1).

An one-column image which starts with these values is likely to be a 3D vector object in G'MIC. But, it is not sufficient. The remaining object data must have a valid structure too. If only one of the data is missing or incorrectly structured, the image will not be seen as a 3D vector object, but as a regular image instead.
Be aware that there is a very strict checking of the image structure to determine whether an image represents a 3D vector object or not.

2.2. Number of vertices and primitives

A 3D vector object is basically composed of :


So, the number of vertices (V) and primitives (P) are two global variables that are mandatory to define a 3D vector object. These two values appear in the image data just after the sequence of magic numbers.

2.3. Vertices data

Each vertex of a 3D vector object is represented by a triplet (x,y,z).
The vertices data are then stored as the sequence of the three coordinates (xi,yi,zi), for i from 0 to V-1. These coordinates can be anything you want, as they are float-valued. They don't have to be bounded.

2.4. Primitives properties

That's the more complex part of a 3D vector object, as primitives can be defined in a very flexible way. They may have different shapes (dots, segments, filled triangles, with or without textures, etc...). So the difficulty is one primitive will be defined by a sequence of values which size depends on the type of the primitive to encode.
The first value N of a primitive sequence represents at the same time, the type of the primitive, and the size of the encoding.
A primitive is then described by a sequence of N+1 values.
Depending on the first value N, the next encoding values have different meanings :

2.5. Primitives materials

Just after the definition of the primitive types comes the definition of the primitives materials (color or texture data). As for the definition of primitives structures, the material of each primitive can be encoded with a variable number of values.
The sequence of numbers defining a primitive material can be:

It can be used to associate a texture to a primitive (applies for primitives with N=6,9 or 12), or an image sprite to a pointwise primitive (N=1), or to define a color primitive that may have more than 3 channels (for N=1,2,3,4 or 5).

That way, there are several possibilities to define primitive materials.

2.6. Opacities

Then, comes the definition of the primitives opacities. It is pretty close to the way materials are defined. Each primitive opacity can be defined by :

2.7. Putting all this together

Now, you know how to define every part of a 3D vector object in G'MIC.
Once again, remember that command split3d is able to decompose an existing 3D objects as a set of these 6 property vectors.
To generate a 3D vector object from all these 6 property vectors, just append them along the y axis, with command append y.

The example below shows how we can write a command that create a 3D vector object from scratch, without having to use anything else than input expressions :


and then
will generate this object:

Note that the command below would generate almost the same 3D object, and is definitely more comprehensive and easy to code:


But in the latter case, each primitive will be merged independently, without sharing any vertices, so at the end, the object generated by my_object3d2 will be bigger (14 vertices) than the one generated by my_object3d (7 vertices).

So, generating your own 3D objects using the internal structure coding is not mandatory, unless you have good reasons to do so. It demands some efforts, and often the vector image we obtain do not comply with the very strict memory structure of a 3D vector object (one byte missing is enough to make it not recognized as a 3D vector object!).
A good way of understanding where the problems are coming from is to try forcing display the 3D object, with command display3d (shortcut: d3d). When this command fails, it outputs a quite descriptive error message that helps to understand the issue.

In the next sections, we show an example on how it can be useful sometimes.

3. Using 3D vector objects to speed up 2D image processing

One of the nice thing with 3D vector objects in G'MIC is that they can be drawn on regular images,  using command object3d. And of course, to be fast enough, this drawing command is a native one, and to draw the 3D object, it loops over the list of all primitives and draw them at their 2D locations (that are the projections of the 3D coordinates regarding the orientation of the 3D object).
That means that the object3d command is able to quickly draw generic primitives (points, segments, ...) on quite random locations of a 2D image. This characteristic of the command can be advantageously used to replace usual loops in some G'MIC commands.
It may be sounds a bit tricky (and it is), but this often allows to speed up the execution of a command surprisingly.

We show here a simple example that code a simpler version of the histogram command, with an usual loop, and with the use of 3D vector objects.

3.1. Computing an histogram: the naive way

Challenge : Suppose you have one input image with integer values in [0,255], which I want to compute the histogram of, and of course without using the histogram command.
A quite naive way of doing this would be :


If we call this command, and see how long it takes (lena.bmp is a 512x512 scalar image):


It works as expected (as it gives the same result as histogram actually), but it is painfully slow because of the two nested loops over the image pixels that are interpreted again and again (no, G'MIC has not JIT compiler !)

3.2. Computating an histogram: using 3D objects tricks.

Now, what if we decide to use 3D vector objects in our histogram computation commands ?
Basically, the idea would be to consider that each pixel value i(x,y) of the original image should be drawn as a point (with additive opacity) onto the target image (histogram), at position X=i(x,y). Looks like the problem of histogram computation can be see as the drawing of a 1d point cloud on an initially black image.
A 1d point cloud can be obviously declared as a 3D vector object, where each primitive is a single colored point (with value 1), and each primitive opacity is set to -1 (additive drawing).
So here we go :


And now, we get :

We avoided loops in our code, making them internally executed by the object3d command, and the gain of time spent is impressive, as we are 3000% faster now !
That is a typical example where it is interesting to know how 3D vector objects are internally stored in memory, as we can generate them to speed up our algorithms.

Actually, lots of existing G'MIC custom commands are using this kind of technique : grid, pointcloud, display_polar, distribution3d, and so on...

Conclusions

G'MIC defines a very flexible way to deal with 3D vector objects, as anyone can generate them from scratch (sometimes this may require a lot of efforts!).
For advanced G'MIC programmers, knowing how to generate these objects by ordering the object data correctly in memory will allow them to optimize their custom commands.
Probably not for used everyday, but it can greatly optimize your G'MIC scripts. Definitely one of the trickiest thing to know about the G'MIC script language, but a powerful way to gain computational time !