How to compute mesh normals

If you make a mesh of vertices, which is supposed to approximate a single smooth surface, how do you create a "reasonable" surface normal vector at each vertex in your mesh?

The key is to realize that when a vertex is shared by several different faces, its normal should be more aligned with the normals of the larger faces, and less aligned with the normals of the smaller faces. For example, suppose we approximate a rounded cube by bezeling the corners:

Here is the same cube, with the edges between the mesh faces rendered visible:
It is clear that we want each vertex to take on more of the shading (and therefore more of the surface normal) of the very large faces adjoining the vertex, and only be influenced slightly by the smaller faces.

Here's how to do it:

1. In each vertex of your mesh, make room for a surface normal. So at every vertex you would store [x,y,z,nx,ny,nz].

2. Initialize all the vertex normals to zero (so that nx = ny = nz = 0). In the next step, you're going to gradually accumulate values into these normal vectors.

3. Loop through all the faces of your mesh. For each face, consider all pairs of successive edges around the face. For example, if your face has vertices A,B,C,D, in that order, then you would consider the four pairs of successive edges:
A→B and B→C
B→C and C→D
C→D and D→A
D→A and A→B
For each pair of successive edges e1 and e2, take the cross product e1 × e2. Add the result of this cross product to the normal of every vertex that adjoins the face (for example, in this case you would add to the normals of vertices A,B,C,D).

4. Loop through all the vertices, and normalize the length of the normal vector of each vertex.

Note that the above algorithm only works if you've defined your mesh faces so that the vertices of each face go around in a counterclockwise direction, when you're looking at the mesh from the outside. Otherwise, your normals will end up facing into the surface, rather than out of it.

It is always a good practice to define mesh faces so that their vertices are sequenced in a counterclockwise direction. Many algorithms in computer graphics rely on the use of this convention.