Normal Map Generation from Handpainting

Vous cherchez la version française ? C’est par ici.

The exercise here is to start with a hand-painted texture, to get an normal map from it. For this job we’ll need an image editor (Photoshop may sound familiar, or The Gimp for the alternative), and ZBrush 4 at least, for its TimeLine feature. The goal here is to keep artistic freedom of manufactured texture, contrary to a procedural one, without sacrificing the benefits of normal map. For the first step, we’ll start with a square document, typically a 1024 pixels wide. Then it’s up to your imagination, or your references, to describe the pattern. Good ? Next step.

2014-02-13 10_11_29-AncientPavement_A_1024

Then, you’ll need to make this pattern tileable, without any unpleasant seams. To help us in this task, Photoshop offers a special, non-artistic filter : you can find it in
Filters -> Others -> Offset. When prompted, you should indicate values equal to the half of your document. Here, we’ll type in 512 pixels for X and Y. We’ll get the borders of the document in the middle, for an easier correction.
2014-02-13 10_10_06-AncientPavement_A_1024

Then, you’ll just have to draw to rest of the pattern, but try not to draw on borders, otherwise you’ll have to correct them again. I encourage you to draw irregular lines, because it will help to break the similarities of the pattern, and make it less artificial. When done, don’t forget to get back to your initial pattern, by applying Offset again. It should be the latest filter used, so you can recall it with CTRL + F.

2014-02-13 10_16_53-AncientPavement_A_1024
Before exporting your pattern for ZBrush, we need to turn this image into grayscale, and increase contrast. For that task, I’ll use Hue/Saturation, and Curves.

2014-02-13 10_16_55-AncientPavement_A_1024

Now the stencil map is ready ! Export in whatever format you want, and we’re done with the image editor.

Let’s switch to Zbrush now !

Preparation Part (ZBrush)

First thing to do in ZB, is to setup the document size : in 99% of cases, it would be rectangular. For the sake of this task, we need to make it square. It will greatly help the stencil. Let’s go to Document palette, and find the pixel values for X and Y.
Beware, uncheck « Pro« (-portional) to unlock the image ratio, and type 1024 for X and Y. When done, click on « New Document » button, three stories above, and validate. It will reinitialize your canvas, so that’s why you should start by this.


On the second time, you can now activate the TimeLine, if not already visible. You need to go to « Movie » menu, open SubPalette « TimeLine« , and click « Show« . That’s it. For further information, I suggest you to read Pixologic’s documentation :

2014-02-12 11_26_33-ZBrush
I anticipate a crucial question, which is :

but what the Timeline is for ? Isn’t it question of animation here ?

It’s right, we won’t be discussing of any animation, but we’ll use at our advantage a cool particularity of the TimeLine : as it is able to memorize location and rotation of any given object, that’s a very smart feature in ZB, because we don’t have such parameters per object like in a conventional 3D package. If we need to recall the same location to apply our stencil, the TimeLine will reveal quite handy !

Creation Part

Next, we need the simplest 3D primitive available : a plane. Once drawn on the canvas, we’ll immediatly edit it (T), and make it editable by clicking « Make PolyMesh 3D » in the Tool palette.
Now, we’ll have to import our texture pattern, to make it work as a stencil. But first, we need to load it in Zbrush as an alpha texture. Go to Alpha palette, Import,

2014-02-12 11_38_08-ZBrush

select the proper file, and go. After, open a sub-palette called Transfert, located below, and click on « Make St » (Make Stencil). If this option is unavailable, check if the alpha is actually selected, you never know. 😉

2014-02-12 11_38_35-ZBrush

And why using a stencil instead of a regular alpha mask ?

For the simple reason that we can lock a stencil on canvas borders, whereas an alpha mask is not, and solely depends on your stroke. That makes an additional source of approximation we can avoid. We could try Spotlight, but I did not yet. It would be worth to check if it can we locked as a stencil…
Now we can go to the Stencil palette, to activate it : in addition to « Use Stencil » button, you should press « Stretch« , and increase the Alpha Repeat value to 3. That gives us a visible pattern with sufficient space around.

2014-02-12 13_51_23-ZBrush
2014-02-12 13_51_08-ZBrush


2014-02-12 13_53_04-ZBrush
We need now to save the current state of the plane, via a Morph Target. Go to Tool palette, under the MorphTarget sub-palette, and click StoreMT. I’ll explain you why a little later.
2014-02-12 13_58_00-ZBrush 2014-02-12 13_58_06-ZBrush
The next task will be to put the plane under the central pattern, with an overlap on neighbour patterns. That done, we won’t find ourselves in lack of pattern information later when we will be going back in Photoshop. When you consider the plane meets every requirements, you can record its position in the Timeline. For that, simply click in the lower scale, at any place. A new solid orange circle would appear.
That’s our key, which I recommend to check if it had recorded the desired location and rotation. If correct, you can poursue.

2014-02-12 13_55_17-ZBrush
Before projecting the stencil, you should go in the Tool Palette, and deactivate the « Smt » (Smooth) button, next to the « Divide » button. That way your corners won’t get smoothed on the first iteration. In my case, the plane borders were creased, which is a sort of protection against the smoothing effect, but in case of doubt, it doesn’t hurt.

2014-02-12 13_56_53-ZBrush

For this job I’ve used the Layer Brush (shortcut : B + L + A), because it preserves existing details, and does not mess surface normal (and make the plane unnecessarily wavy), plus we can draw several times on the same spot, and it won’t accumulate.

The worst choice would be the Standard brush, because it would do exactly the contrary of aforementioned perks.

You could use Clay Brush (B + C + L), but not ClayBuildup, because of the « buildup » option.
The following is pretty straightforward : you apply the stencil all over the plane surface, then you subdivide, you paint the stencil, you subdivide, and so on, until you reach the seventh level of subdivision. There, we’ll get a total of 4 millions of polygons, which more than enough for our texturing needs.

2014-02-12 14_15_39-ZBrush
A 1024 pixels square texture contains circa 1.05 millions pixels. At level 6, we have already reached that number, but remember, we do not use every polygon of our plane for the final export, so it is always better to see larger at first, and then shrink to fit your needs. You won’t lose quality.
I could also say that the default texture size generated by Zbrush is 2048 px, which is fine, because we’ll have much more what we need, especially if we take in account the security margin, and the final adjustment in Photoshop.

Now the stencil part is over, we’ll start the sculpting. Texture has left crisp, jagged edges on the geometry, and we need now to smooth them, but caution, do not abuse of smoothing brush, otherwise you’ll wash away your details.

2014-02-12 14_29_52-ZBrush

It’s better to polish edges, with a TrimDynamic brush (B + T + D), which give a subtle but convincing look of worn stone. This task is a meticulous one.

You could also use the PenMash brush (CTRL + draw), to protect the surrounding edges from unwanted polishing. Moreover, I’ve used a Clay Brush, without alpha, to level up some areas of stones, and smooth them after. The DamStandard Brush (B + D + S) is a cool one to create scratches and grooves. I should recommend using alpha 40.
After doing, this is what I’ve obtained :
2014-02-12 14_37_45-ZBrush


For the normal map export procedure, we need first to go to the lowest level of subdivision, I mean level 1. At this point we can notice that our plane is not so flat anymore… That’s now the Morph Target would reveal very precious, and in one click, our prefect plane would come back.

But why is it so important ?

Because normal map calculation is made from low-poly state and high-poly state, the difference of location vertices has a direct impact on the resulting normal map. A non flat plane would create a less accurate map, instead of a real flat one, unless we want our geometry not to be flat.
In the Normal map panel, don’t forget to activate the « Tangeant » button, otherwise it would be the world space normal map that would be generated, and not the tangeant one, which is the one we want.

Also activate « FlipG » (flip Green) button. This later button inverts the Y axis on computation, and therefore our normal map would render correctly, and not reversed.
Click on « Create Normal Map« , and let Zbrush do its work… And the little blueish thumbnail appears, which informs us of the completion of the process. But we’re still not out of Zbrush !
Just a few steps left… If you are new to ZBrush texture management (old school way), here is the trick : in the same panel, click on « Clone NM« , and then go to the Texture Palette. Check if the Normal Map is not upside-down. In my situation it was not the case. But Zbrush is known to work upside-down. So, beware. You can flip the texture vertically inside ZBrush, or in Photoshop, that’s up to you. Click on Export, and… give it a name.

Back into the Image Editor

The following is pretty simple. Import your newly acquired Normal Map file into your original texture, decrease opacity, and shrink it and move it to match the underlying pattern. If you do it carefully, you may have only a 1-2 pixels error, which is really low, and unnoticeable at runtime. This is the only manual crafting part, I hope to find later a better way to avoid this trick.
2014-02-12 15_07_30-AncientPavement_A_1024

And now, a render in a realtime engine, without and with the Normal Map.

2014-02-13 11_35_312014-02-13 11_35_56

I needed about three hours to make normal and painting, plus the tries and errors. With some training, I could have added more fine details, the current surface look quite plastic… But it has bumps !
I hope this tutorial would have make you feel like to try other patterns, or crank up your painted textures !