Devil in the Shadows

I've taken the first steps towards implementing shadows for miru -- glorified 3d extensions (and a handful of arbitrary features) for pyglet. I had avoided implementing a shadowing effect for miru for some time, for fear that my brain wouldn't be able to handle the math, or I'd get lost in the gnarlier details of the OpenGL API and just throw my hands up after hours of time trying to get it right. Last night I was able to finally hammer out a proof of concept which still needs a lot more work (and maybe even more underlying library support) before I can build something decently generalized.

The technique I used to generate the shadows is referred to as shadow mapping - which is simply in theory but gets somewhat complicated in the implementation - in particular when you consider limitations of texture sizes for many common OpenGL drivers and other potential texture artifacts in a scene. My own implementation was borrowed mostly from an examples in the excellent OpenGL(R) Super Bible--I guess I should return this to my company's library someday since I've been hogging it for several months now. Shadow mapping entails creating a depth texture which means copying the z-buffer from the perspective of a light looking down onto the scene into a texture region to be applied to objects in the scene using eye-linear texture coordinates.

(It should be noted here that the process I use is a somewhat old school way of shadow mapping, though not as old-school and crude as stencil testing. With modern graphics hardware, vertex shaders and per-pixel lighting can be used to achieve much nicer and softer shadows and their penumbra. My goal is to first implement something using a traditional multipass technique and then try something more sophisticated with shaders--again, if my brain can deal with the math.)

To illustrate the first step in the shadow mapping process, the following code (which assumes there is single globally bound texture object) could be run at start of a program and whenever the light moves in the scene:

        scene_radius = 95.0
        light_to_scene_distance = math.sqrt(
                light_pos[0]**2 + light_pos[1]**2 + light_pos[2]**2)
        near_plane = light_to_scene_distance - scene_radius
        field_of_view = miru.math3d.radians_to_degrees(2.0 *
                math.atan(scene_radius / float(light_to_scene_distance)))

        tmp = ( GLfloat * 16)()

        glMatrixMode( GL_PROJECTION )
        glLoadIdentity()
        gluPerspective( field_of_view, 1.0, near_plane,
                near_plane + (2.0 * scene_radius) )
        glGetFloatv( GL_PROJECTION_MATRIX, tmp )
        light_projection = euclid.Matrix4()
        light_projection[:] = tmp[:]

        # Move to light's perspective
        glMatrixMode( GL_MODELVIEW )
        glLoadIdentity()
        gluLookAt( light.pos.x, light.pos.y, light.pos.z,
                  0.0, 0.0, 0.0, 0.0, 1.0, 0.0 )
        glGetFloatv( GL_MODELVIEW_MATRIX, tmp )
        light_mview = euclid.Matrix4()
        light_mview[:] = tmp[:]

        # Clear the depth buffer only
        glClear( GL_DEPTH_BUFFER_BIT )

        # Remember the current shade model and setup driver state
        ...

        # do a render pass
        render()

        # Copy depth values into depth texture
        glCopyTexImage2D( GL_TEXTURE_2D, 0, GL_DEPTH_COMPONENT,
                         0, 0, shadow_width, shadow_height, 0 )

        # Restore prior driver state
        ...

        # Setup up the texture matrix which will be use in eye-linear
        # texture mapping
        tex_matrix = euclid.Matrix4()
        tex_matrix.translate(0.5, 0.5, 0.5).scale(0.5, 0.5, 0.5)
        tex_matrix = (tex_matrix * light_projection) * light_mview
        tex_matrix.transpose()
        # Give us immediate access to ctypes arrays
        tex_matrix = (
            (GLfloat * 4)(*self.tex_matrix[0:4]),
            (GLfloat * 4)(*self.tex_matrix[4:8]),
            (GLfloat * 4)(*self.tex_matrix[8:12]),
            (GLfloat * 4)(*self.tex_matrix[12:16])
        )

In the above I omitted some details such as setting up and tearing down the required driver state. (I'll post a link to the completed source code at a later time.) The import parts are all intact, though, which include copying over the depth buffer to the current bound texture:

        glCopyTexImage2D( GL_TEXTURE_2D, 0, GL_DEPTH_COMPONENT,
                         0, 0, shadow_width, shadow_height, 0 )

It is important to note that this is done after rendering the scene, of course, and setting the view matrix according to the light's perspective. This are just simple API details, but the math comes at the end (saving the best for last):

        tex_matrix = euclid.Matrix4()
        tex_matrix.translate(0.5, 0.5, 0.5).scale(0.5, 0.5, 0.5)
        tex_matrix = (tex_matrix * light_projection) * light_mview
        tex_matrix.transpose()
        ... translate into ctypes array

All the matrix operations are done using pyeuclid which is a pure-python library well-suited for graphics programming being focused on vector, matrix and quaternion operations; however, I've submitted a patch to add transpose() methods which are not currently provided. In first two lines we translate and scale an initial identity matrix by 0.5 along each axis to create a bias which allows us to translate the actual coordinates captured in the range (-1,1) to the range (0,1)--normalized device coordinates, or screen coordinates. We multiple the bias matrix by the light projection and model view matrix acquired in earlier steps and transpose the result to produce the final texture matrix we'll use to map the texture onto objects in the scene.

Once we have the texture matrix, we're ready to apply the texture. First of all, the texture should be bound as follows:

    glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE)
    glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE)
    glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR)
    glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR)
    glTexParameteri(GL_TEXTURE_2D, GL_DEPTH_TEXTURE_MODE, GL_INTENSITY)
    glTexGeni(GL_S, GL_TEXTURE_GEN_MODE, GL_EYE_LINEAR)
    glTexGeni(GL_T, GL_TEXTURE_GEN_MODE, GL_EYE_LINEAR)
    glTexGeni(GL_R, GL_TEXTURE_GEN_MODE, GL_EYE_LINEAR)
    glTexGeni(GL_Q, GL_TEXTURE_GEN_MODE, GL_EYE_LINEAR)

Now we can draw the objects in the scene with the usual dizzying array of OpenGL calls:

    if not ambient_shadow_ext:
       # do an ambient pass
       ...

    glLightfv(GL_LIGHT0, GL_AMBIENT, ambient_light)
    glLightfv(GL_LIGHT0, GL_DIFFUSE, diffuse_light)

    glPushAttrib(GL_ENABLE_BIT)

    glEnable(GL_TEXTURE_2D)
    glTexEnvi(GL_TEXTURE_ENV, GL_TEXTURE_ENV_MODE, GL_MODULATE)
    glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_COMPARE_MODE,
                    GL_COMPARE_R_TO_TEXTURE)

    glEnable(GL_TEXTURE_GEN_S)
    glEnable(GL_TEXTURE_GEN_T)
    glEnable(GL_TEXTURE_GEN_R)
    glEnable(GL_TEXTURE_GEN_Q)
    glTexGenfv(GL_S, GL_EYE_PLANE, tex_matrix[0]))
    glTexGenfv(GL_T, GL_EYE_PLANE, tex_matrix[1]))
    glTexGenfv(GL_R, GL_EYE_PLANE, tex_matrix[2]))
    glTexGenfv(GL_Q, GL_EYE_PLANE, tex_matrix[3]))

    ... render objects in scene

    glPopAttrib()

The most interesting part of the above is the fact we get to use all texture coordinates: S, T, R, and Q. For the uninitiated, only S and T coordinates are explicitly set for more common texture operations which correspond to (row, column) pixel coordinates in the image. The OpenGL programming guide gives a brief overview of the R, Q coordinates and their role in projective textures.

So I only have more hurdles ahead to get this into decent shape for use in most any 3d application. The current form so far is more or less a direct translation of the example from the aforementioned book and inherits its limitations. First of all, by convenience all the objects rendered use color as their sole material for shading. Were we to have textured objects in the scene some additional work would have to be done (namely, better multitexture abstractions in miru or pyglet) and some details regarding this would be incorporated in the resulting API for ShadowMap requiring specification of a texture unit as not to interfere with other multitexture applications beneath the shadow. Next, the current application requires the screen to have dimensions that are powers of 2 - hence, the non-standard resolution of the screenshot above. This is less than ideal of course, but can be remedied by rendering to an FBO - which also calls for some more underlying library work. I've seen some very cool work done with FBOs in the cocos framework and hope to see this sink down into pyglet someday. Otherwise I might try ripping pieces of it out for miru ;)

On a final note, I'm in the midst of some drastic rewriting and reorganization of miru, phasing out the dirty bits--miru.mesh and miru.environment in favor of miru.graphics and miru.context, respectively--and changing, or actually removing, some interfaces to defer directly to pyglet.graphics instead. This means things are proceeding slowly with some subtle breakage here and there. In the long run, I think it will make miru a more coherent and useful library and provide smoother integration for other projects already exploiting pyglet for OpenGL state management.