You have the ability to bake various maps in TopoGun using the Bake feature. You can bake normalmaps, displacement maps, hardware ambient occlusion maps, software ambient occlusion maps, diffuse maps transfers and cavity maps.
In order to use the Bake feature, you should have a highpoly model (the reference mesh) and a lowpoly mesh (created in TopoGun or in any other application). These should be in .obj format. Note that the lowpoly mesh should have a valid UV mapping and should consist of triangles and/or quad faces.
Before going forward, you should have at least a minimal knowledge of normal mapping techniques and you should understand some of the basics of baking maps. You can easily find tutorials on these topics on the Internet.
Start by loading the high resolution reference or subdivided mesh (File->Load Reference) and then loading the lowpoly retopologized mesh (File->Load Scene, change the file type to .obj). Note that after loading the lowpoly mesh, you can only modify the topology by moving the vertices of the lowpoly cage. You must not modify the topology by adding or deleting geometry. If you do so, the geometry's UV's will become invalid and you won't be able to bake your maps, or if you do, the baked maps could contain errors.
In order to bring up the main Bake window, click on the Generate Maps entry from the Bake menu. Notice that a duplicate black version of your lowpoly is rendered in the viewport. The black mesh is called the baking cage, and is generated from the scene geometry.
The Generate Maps window is divided into several important ares. The first one is the General Settings tab.
You usually start by setting the Map width and Map height.
Choose between one of the raycasting sampling quality presets from the Sampling drop down list (MEDIUM should be just fine in most cases, setting it to HIGH could be a waste of time).
The pixel values along seam edges are expanded by a distance equal to the Edge Extent value. This is in order to avoid future rendering issues like texture seams appearing at render time, due to the various sampling methods, (such as mipmapping).
The Search Dist limits the raycasting search distance, in order to maximize the baking speed. Setting a too low Search Dist value could cause empty pixels on the rendered maps because the fired rays won't reach the highres reference model.
The cage mesh's vertices are shifted along their normals by a distance equal to the Cage Dist value. This is done in order to optimize the baking process (the reference mesh might have overlaying geometry layers and leaving the Cage Dist parameter set to zero could create a lot of problems in the baking process). This is due to the fact that the lowpoly cage would intersect in different areas of the highresh geometry layers at a time. Setting this parameter to a non-zero value helps the baking module "see" the highpoly mesh from a certain distance, avoiding the possible confusion that would have arisen if the highres areas are "seen" from too close.
Remember that the reference mesh is normalized during import (it's scaled down so it can fit into a cube unit). The Search Dist and Search Dist parameters are given in TopoGun 1/1000 units.
The raycasting can be done BOTH SIDES (TopoGun fires intersecting rays along both sides of the cage mesh normals), INSIDE CAGE or OUTSIDE CAGE (fires intersection rays inside the cage mesh or outside - along the direction of the normals).
If you want to generate displacement maps, you should check the Subdivide levels option and set a bigger than 1 value for this parameter (usually 2-3). That will subdivide and smooth the cage mesh, in order to get a valid usable displacement map. (Displacement maps are usually applied over a subdivided version of the lowpoly cage, at render time. That is why the displacement should be created somewhere between the subdivided cage and the reference mesh).
By default, TopoGun smooths out the normals of the cage mesh, (there are no hard edges). However sometimes you could use some hard edges. In order to have more control over the direction of the normals, and thus the directions the baking rays are fired to, you should use a Custom Cage. A custom cage is a mesh that is loaded directly from an external file, not from the existing scene geometry. The main difference is that the custom mesh can have custom normals (as a result the normals are not smoothed when loaded by the baking module).
You can also use a Custom Reference mesh (with custom normals). An advantage of using a custom reference mesh is that the mesh can be much heavier than a normal reference mesh. A standard reference mesh has to be loaded into the video memory, while a custom reference mesh will not be loaded into the video memory and as a result, it doesn't have to be rendered in the viewport. The disadvantage is that you can't bake hardware ambient occlusion maps using custom reference meshes, because hardware ambient occlusion maps are baked is using the geometry data from within the video memory.
Another important feature is the usage of an ambient occlusion Dummy Object. An ambient occlusion dummy mesh will occlude the reference mesh, but it won't be taken into account when generating the maps (it's invisible to the raycasting engine).
The Load AO Dummy button opens a file load dialog allowing you to add a new AO Dummy Object that you want the baking module to take into account. After loading a new AO dummy, it will be rendered in the viewport as a black mesh.
The Delete AO Dummy button deletes the selected AO Dummy from the AO Dummies List.
This is what the ambient occlusion map would look like with and without the torus ambient occlusion dummy:
You'll have to enable the Generate checkbox for each map you want to bake (each map has it's own parameters group within the Generate Maps window).
Another parameter you will find in each group of parameters is Filter. TopoGun can apply a convolution filter on each of the baked maps, in order to improve the map's quality. You can choose between applying no filter to the baked map by leaving the Filter parameter set to NONE, or you can apply a GAUSSIAN BLUR or a SHARPEN filter. The default filter intensity is 50% (the Filter power parameter).
You'll have to set a file name for each of the baked maps, if you don't want to use the default ones. 8 bit maps can be saved as .tga files and 32 bit floating point maps can be saved as .tiff files. You can press the Set File button for setting the file name by using a file save dialog box, or you can also write the file name directly into the text box next to the Set File button.
After choosing which maps should be baked and setting the right parameters, all that's left to do is click on the Generate button at the bottom of the window, (scroll down). Once you are doing baking or if you just want to close the baking parameters window, you can press the Close button underneath the Generate button.
In order to bake a normalmap, enable the Generate checkbox found in the Normal Map options group. You can choose between a variety of engine specific options within the normalmap's options group. The parameters names are pretty self explanatory: The Flip Red Channel and the Flip Green Channel options will invert the normalmap's red and/or green channels. The Swap Red-Green option will interchange the red and green channels data.
You can choose between generating a Tangent Space normalmap (used for skinned/deformable objects) or a World Space normalmap (used for static objects).
In order to bake a displacement map, enable the Generate checkbox found in the Displacement Map options group. You can choose between generating 8 bit displacement maps (256 possible height values) or 32 bit floating point world space displacement maps.
You should enable the Subdivide Levels option in the General Settings group and set a non unitar value for the subdivision levels number (something like 2 or 3).
The following image shows an example of a displacement map created without subdividing the cage, and another one with the same cage subdivided 3 times:
If you bake an 8 bit displacement map, you should write down the displacement min/max and/or intensity/offset values, in order to correctly set the rendering parameters in your displacement rendering engine. Look for these values in the Log Window, after the baking process is over. You can display the Log Window by clicking the View Log Window entry from the View menu.
The displacement values are written in the Log Window and are within four lines that should look like this:
Displacement max value is 0.212334
Displacement min value is -0.183476
Displacement value is 0.395811
Displacement offset is -0.183476
If you bake a 32 bit floating point displacement map, you should first convert the resulting .tiff file to a 32 bit floating point image format supported by your rendering engine, if it doesn't recognize 32 bit floating point .tiff images. Then you should simply plug the displacement file into the displacement shader and set the displacement value to 1.0. You will get real world displacement values. If you scale the rendered object, you should scale the displacement amount value accordingly.
Ambient occlusion is a way to approximate the incoming amount of ambient (environmental) light a surface point receives. The amount of light can be diminished by other objects in the proximity of the sampled object points, occluding them.
You can bake ambient occlusion maps using TopoGun. The intensity of each pixel in the ambient occlusion map represents the visibility of that pixel in space of the surrounding environment (a black pixel will be totally hidden from the environment and a white one represents a pixel totally visible to the environment).
The hardware ambient occlusion baking method uses an Outside In occlusion algorithm. That means the occlusion is computed by tracing rays randomly sampled from different positions in the environment to the sampled surface points.
Ambient occlusion maps are a great addition to the texture creation process (the ambient occlusion map is often multiplied over the final colormap), improving the final textured object's realism.
In order to bake a hardware ambient occlusion map, enable the Generate checkbox found in the Hardware Ambient Occlusion Map options group. You cannot use a custom reference mesh when baking hardware ambient occlusion maps.
You can add custom ambient occlusion occluder objects as instructed above (via AO Dummies).
The Quality parameter defines the number of ambient occlusion iterations, (the number of iterations roughly equals the Quality value multiplied by 100).
The Sample offset parameter defines the hardware ambient occlusion shadow depth's buffer offset.
The Precision parameter helps bringing out some of the finer details, but the baking process can take longer.
The Skylight feature mimics the light coming from the sky (the light will come only from above, there will be a virtual ground plane blocking the light from below).
The Skylight coverage changes the environment's (sky) solid angle value. A value of 100 for this parameter means the environment is set to a full sphere around the object (this is the same thing as disabling the Skylight option). A value of 50 means the environment will be a 180 degree hemisphere around the object - 50% of the full sphere). The image below shows some examples of coverage values and their corresponding environment angles.
The Lambertian option is useful only when using the Skylight method. The environment hemisphere will act as a real light source, the ambient occlusion will be computed using a lambertian term, producing a more detailed and realistic result. This way the fine details will become visible on the baked ambient occlusion map, unlike the regular ambient occlusion baking method.
The software ambient occlusion baking method uses an Inside Out occlusion algorithm. From each surface point, a number of rays are shot in random directions within a hemisphere. Each time a ray hits a surface, the occlusion value is increased for that surface point, thus darkening the current sampled point. In the end, the ratio between the number of rays that never hit a surface point and the total number of rays that were shot will give the final occlusion value for the given point.
The hardware occlusion baking method should be used instead the software approach, due to the better quality maps it provides and because of the lower amount time it takes to bake a map.
In order to bake a software ambient occlusion map, enable the Generate checkbox found in the Software Ambient Occlusion Map options group.
The Samples parameter defines the number of rays that will be shot from each surface point. The Spread parameter defines the solid angle value the rays will be shot within. A value of 180 means the rays will be shot in random directions within a hemisphere.
The Search Distance controls the length of each ray. A value of 0 makes the ray's length infinite. Any other value sets the ray length to a finite value. If an intersection occurs beyond the given length, the intersection will be discarded and the ray will be considered as if it missed any piece of geometry and will not occlude the surface point for the given ray.
The Skylight option adds a virtual ground plane to the scene, producing the so-called "skylight" effect.
The Lambertian option adds a cosine term to the occlusion calculus. Each ray will have a weight, based on the angle to the normal vector's cosine value.
If the reference mesh has a texture (a possible way of working is to create the reference mesh and then texturing it), you may then want to transfer the reference mesh's texture to the lowpoly cage's UV space.
You should save the reference mesh texture as a .tga file first, then load it in TopoGun by clicking on the Load Reference Texture command in the File menu. If you want to see the texture on the reference mesh, activate the Show Textured Reference option from the Options menu.
Obviously, you can't apply the original texture on the lowpoly cage, because most likely the lowpoly cage has different UV coordinates, compared to the reference mesh.
You'll have to transfer the color map from the reference mesh's UV space to the lowpoly's UV space. In order to do that, open the bake window and activate the Generate checkbox in the Color Map options group.
After baking the maps, you get a new texture, representing the original color map, translated to the lowpoly's UV space.
Another useful feature is baking group colors. If the reference mesh has multiple groups, you can bake the groups colors, (these are randomly assigned at the reference mesh's loading time). This could be extremely useful when creating a texture, since the different colored groups can make it easier to create pixel selections. (An example would be, if you wanted to apply different texture elements to the different groups). To bake group colors, activate the Groups colors option. You can also preview the group colors in the viewport, by activating the Colorize Groups option from the Options menu. You can activate the View Groups Window option in the View menu and you can select one of the groups from the list.
After baking the colormap with the Groups colors option on, you will get a texture representing the reference mesh's group colors:
Finally, the Vertex colors mode helps you bake reference mesh vertex color data. The reference mesh should be saved as a .ply file and contain per vertex color data, saved as unsigned char red, green and blue properties. You can also import ZBrush 3.5 per vertex color data, saved within the .obj meshes you export from the application. Make sure you check the Colorize option before exporting the mesh from ZBrush.
Cavity maps can be used to enhance the very fine details on the final texture. They're usually applied as a multiplied layer over the final color map, in order to bring up very fine details, such as skin pores, fabric patterns, hair.