Ambient Occlusion & You: A guide to creating lightmaps

[Mike]

Hey everyone! I wanted to write up a quick 'n dirty primer on how we handled level shading/shadowing in RPP to make it easier for you guys to make lightmaps of your own!

For those who aren't aware, a lightmap essentially overlays shading information over a level. It's analogous to a separate layer in Photoshop. I'd normally start a guide by writing a primer on what ambient occlusion (AO) is, but that's what Wikipedia is for, right? You're more than welcome to experiment with other methods of geometry shading, so long as they follow the same principle.

For those who don't want to read up on Wikipedia, AO is what makes this...

...look like this!


What you'll need:
-DeleD (for the time being...we're trying to eliminate this from our workflow)
-A 3D modelling program (Maya, Blender, 3DS Max, et al.)
-An image editor (Photoshop, GIMP, et al.)
-LightmapMerger.exe (included with the game)
-Patience

One thing to keep in mind is that lightmap generation is the very last step of level creation - once you've applied it to the stage, you really shouldn't be making any modifications to geometry. We also highly recommend that you save a backup of your level prior to this - lightmaps are fickle in nature, so it's never a bad thing to keep spare copies.

Another thing to consider is that it IS possible to bake lightmaps from DeleD directly, but this will produce less-than-stellar results. That method is quite simple in comparison - just hit "create lightmap" inside of DeleD, and let it go to work! If this guide appears somewhat overwhelming, I would recommend you simply take this approach for now.

For the sake of this guide, we're going to assume the level has already been created. I'm going to use the first Sowbug stage (level114 in your main folder) as our example throughout.

The guide!

Step 1: Download Collada Exporter plug-in for DeleD
A link can be found here.

Step 2: Export level as DAE
Open up level114.dxs in DeleD, and head to the Plugin->Collada Exporter option in the menu. You don't need to touch any of the other options - just hit OK.

Step 3: Import stage into 3D modeler of choice
A lot of 3D modelers should have native .dae support, but if they don't, Google around for an importer. I can guarantee that Maya supports this format, as does Blender. The reason we go for DAE as opposed to OBJ is so that you can still manipulate the stage elements - OBJ would combine the mesh and lose a lot of important metadata in the process.

Step 4: Remove irrelevant geometry
Your goal here is to eliminate any geometry that you don't want in the lightmap. In our example case, this includes:

• The level boundary box
• The water plane
• All the flagpole elements
• The bottom of the hole
  

Note that on stages with moving geometry, translate those sections far from the stage so that the AO bake remains accurate. You don't want to bake them for a single position, after all.

Step 4.5 (optional): Improve geometry normals
If you want to spend some time and smoothen out your level normals, now's the time. Just be sure not to physically modify any geometry when you're at it.

Step 5: Combine the level into one mesh
This is becoming harder and harder for me to write without being program-specific....every 3D modeler should have the ability to combine all level elements into one group mesh, even if they aren't intersecting polygons. The option is called 'combine' in Maya. After that's done, you're also going to want to strip out the previous texture information - in Maya, this means applying a fresh lambert overtop the entire stage, and I'm sure there are similar methods in other programs that accomplish the same task.

Step 6: UV Mapping!
In Maya, this is as simple as going to Create UVs-->Automatic Mapping (unless you're hardcore enough to want to use planar mapping and lay out the UV map manually, but that is way outside the scope of this guide). Your options should be set to this (apologies for terrible image quality):



Blender users, well....Google is your friend. If someone familiar with Blender can fill this part in, I'd be very grateful! In any case, you should see something like this appear:



You're almost there!

Step 7: Bake the lightmap!
In Maya, go to Color-->Batch Bake(Mental Ray). If you don't see this option, make sure you have the mayamr plugin enabled in your options. Here are the settings you want to have:



As for other programs...here's a quick and dirty version of how to do the same thing in Blender:

You can increase the texture size to 4096x4096, should you need the extra space. Physically baking a lightmap will depend on your PC specs, but it typically took me anywhere from 5 to 15 minutes to bake. The complexity of the geometry determines the length, so just sit and wait for your 3D modeler to do its thing. Patience is key here.

Step 8: Refine the lightmap
The bake is complete, phew! Smells delicious, don't you think?

Now is when you want to take that output file (which should look similar to the UV map posted earlier, but with some shading against a white surface) and throw it into your image editor of choice. I can't really explain why this happens, but ambient occlusion generation has a tendency to produce artifacts...errant little pixels in the strangest places. A simple tap with the heal brush and you're set.

Feel free to take this time to 'stylize' your lightmap as well - just don't make any modifications where the pixels exceed the regions of the UV map they belong to. I would recommend limiting your edits to exposure/contrast modifications.

There is one part of this step that's necessary, though - once you're done, be sure to vertically flip the lightmap texture. Save your final image as either a PNG or a JPG, depending on your preference.

Step 9: Save lightmap texture as .BUZ
Open up BUZMaker and drag'n'drop your lightmap texture in. Click "save as .BUZ". Navigate your way to /textures/LightMaps and ensure the filename corresponds to your level name (ie. the lightmap for level100 is level100.buz). When you get to BuzOptionsDlg, tick "embed" under the format of your choice (most likely PNG) and hit "save". Depending on the dimensions of your lightmap and the number of cores within your PC's processor, this may take a while to complete...the dialog box may appear to be frozen. My little 'trick' is to tap above the scrollbar a few times while I'm waiting - once the texture has changed, you know that the save has completed, since BUZMaker is being responsive,.

Step 10: Export map as OBJ
You know that 3D model of the stage you were using to make the lightmap? Yeah, we still need that - go to the export options and select OBJ. If you see any settings about retaining groups/materials/smoothing/normals/etc. settings, disable them.

Step 11: Import OBJ into DeleD
Yes, unfortunately, we have to return to this awful program one last time. Under plugins, hit "Waveform OBJ importer" and throw in the OBJ you just had your legit 3D modeling program spit out. Save as DXS and quit.

Step 12: Merge the lightmap DXS with your level DXS
For this, you need to open the LightmapMerger tool in the /utilities/ folder provided for you. I would recommend cleaning your regular course DXS before this step, just to be on the safe side. Set the correct filepaths, hit Go, and yet again, be veeeeeery patient. Just let the program do its thing, go outside for a walk in the meantime or something.

Step 13: There is no step 13, you're done!
You should now have ambient occlusion shading applied to your stage. Give it a run in-game and see how it looks!

As I mentioned above, applying AO to your stage should be the very last stage in its development process. The only things you'll be able to modify is the collision mesh, unless you want to start back at step 1. You can also modify textures, but with one caveat: You'll have to open the DXS file in a text editor and change it manually, or else the lightmap merge won't survive the trip.

Should you ever want to REMOVE a lightmap from a stage, there's an option in DeleD to do just that.

Now, please keep in mind that this is a live document, and our methodology is always changing to accommodate new features (ie bumpmapping, multiple AO texture support, etc.), so be sure to always check back for any changes or updates!