How to create source mod characters.
Please note: This is not a tutorial. This is just a recount of my experiences making characters for use with the Source Engine and the Hammer Level Editor. You may learn of a few things to keep your eye out for when doing this yourself, but this is not a step by step guide.
The characters were modeled in Maya. The could have been modeled in any software package, so I decided to use the one I was most familiar with. You just have to keep in mind the limitations of the engine. Through research I found that a good rule of thumb is around 8000 tris for a poly count, and a 1024x texture sheet.
I also imported a character model from Half Life into maya to see what sort of scale to model the character in. If you model your characters in the XSI Mod Tool, you can use the Source Biped Guide to make sure your character is around the same size. You can always scale your character later, but getting it roughly close does help.
This was when research became more important, as I needed to learn how to rig the characters properly in for the animation from existing characters to be used properly, and also so the game would recognize the skeleton. I looked through many online resources, including the Valve Developer Wiki (http://developer.valvesoftware.com/wiki/Main_Page) and the XSI Mod Tool website (http://www.softimage.com/products/modtool/inside.aspx).
From there I learned that it would be best to use the Mod Tool to rig the characters. It came with tools built in to create guides and rigs specifically for the Source Engine. This way I did not have to rely on any third party plugins for Maya. The Mod Tool is a free download for any to use, and it was not hard to learn.
It was at this point that I learned of the Noesis training videos (http://www.noesisinteractive.com/). They are a series of video tutorials made to teach you the process of creating assets and compiling them for use in a Source game mod. I used the one called Character Design & Integration with Half-Life®2 (XSI). I used this video for most of my reference material. I found it was very well made and easy to follow. There were a few discrepancies, as the video was a few years old, and was not for the specific game that we were using, but it provided a very good base.
Rigging the characters was fairly straight forward, as it was very similar to rigging in Maya. You first line up your character as close to the guide as you can, and then you can move the guides markers to the specific joints of your character. It is important not to move joints near the torso of the body too far from their starting point, as I found that the animation from existing characters will actually try to move them back, thus distorting your character in ways that you don't want.
Once the guide is in place you can tell the Mod Tool to create the actual rig from the guide. It will place all the bones in controllers in the proper places following the guide's placement. Then you begin the weighting. It works much the same as smooth skinning in Maya. You can paint weights on each bone, and you can also use the weight editor to specify the weights for each verticy specifically.
There are a few more steps in the Mod Tool before you can export it for compilation. You have to make sure that your model is pointing to a texture file that is inside the proper Steam folders on your computer, and that that file is a TGA format.
At this point you can export your character as an SMD, which is the starting format for Source engine models. You need this file, and the TGA texture file in order to create the compiled version.
This was the area of creation that required the most trial and error. The video tutorial was very helpful in this area, but as mentioned before it was not very up to date, and there were quite a few areas that had changed.
Compiling the texture was rather easy. All you had to do was place the TGA file in the proper materialsrc folder inside the Source SDK Content folder. You then dragged that file over on top of the vtex.exe which is contained in the bin folder of the engine version you are using. That's really all you need to do. The corresponding VTF file is placed in the game Material folder that you are using. You then create a folder of the same name as the file, and in that folder you place a VMT file also of the same name. This is basically a text file that tells the model where the texture is located, along with some other basic shader information. There was a sample of this file included with the video tutorial. Some editing in notepad was all that was needed.
Compiling the models correctly proved to be much more difficult. Most of the information for compiling the model is contained in a QC file. This is also a text file that references the SMD model, where the new MDL model will go, what certain bones do, facial information, animation references, etc. I used the basic QC from the Source SDK Content folder for the Male 06 model.
The main problem I had here was trying to get the models to compile using animation from existing Source models. There were various strings of code that seemed to reference different files for animation and other information, so it took various trial and error to figure out which ones to change, and which would result in errors during compiling.
You compile using the studiomdl.exe which is in the same folder as the vtex.exe. Though when trying to compile various iterations of the qc file, the screen did not stay up long enough for me to see the errors and troubleshoot them. Luckily I found a program that created an more user friendly interface for the studiomdl.exe, called GUIStudioMDL.exe (http://www.wunderboy.org/apps/guistudiomdl2.php). This allowed me to see the errors and come up with solutions.
I discovered that the best way to include animation in the custom models was to use the string $includemodel before the filepath of the model. This way it would include the animation from an existing model from any Source game you wanted.
It also took some experimentation to find proper characters to use for the base animation. The original plan was to try and include animation from a newer game called Left 4 Dead, which characters more closely resembled my custom ones in actions and behavior. However the Development Kit was not released at the time for this game, and decompiling the models myself did not yield a usable file.
So I found some suitable characters that existed in Half Life to use, and compiled the characters accordingly. Another error I came across when compiling them, was that it appeared that the files did not have write permissions. This was due to the folders that it was trying to write to were none existent. Unlike most programs that will create folders if they don't exist, the studiomdl will not.
Importing into Level
This was the final step in making custom characters for our game mod. I expected to be able to import my characters and have them act as the characters that their animation was based on. I was mistaken, as my custom characters lacked AI to control them in the game engine, and there was no way to get them into the level otherwise.
After researching some more, and posting on the Steam forums asking for help, I was down to two options. I could either try renaming my characters' files as the files for existing characters, essentially tricking the engine into using my character models instead of the originals, with the AI of the existing characters, or learn c++ and create my own AI code. I opted for the former.
I tried this method a few times, with the result of utterly breaking the SDK on my computer, and also breaking the characters that I had compiled and had working in the Model Viewer. I did not understand why. It took some more trial and error to realise that I was renaming my custom files to the very same name of the files I was referencing for the animations, thus creating an infinite loop, which is what caused the SDK to crash.
I solved this issue by copying the original Source characters, renaming them to Character_Original, using that as the reference in the QC for the animation, and then naming my custom characters as the original Valve characters. That solved the issue, and allowed me to bring in my custom characters, under the name of pre-existing Valve characters in the Hammer editor.
The last few issues had to do with the kinds of characters I was using for the animations. I originally wanted to use zombie like animations for my characters, but it even though they worked fine in the Model Viewer, when in the hammer editor, and in a running map they did not work correctly. Animations would not play, characters would not move, and some would crash the game when trying to use an animation that wasn't there.
I believe this problem comes from the skeleton I used. I don't think it is exactly the same as the skeletons used for the zombies. It is for humans only. So I solved this issue by recompiling the characters using the animation from pre-existing human characters. This worked for 2 out of 3 of the characters. The last still would crash the game, but I could not fix it by using another character's animation, as there are only so many characters available, and only a certain number of those seemed to work with my characters.
So in the end I managed to get 2 out of three characters in the level and working properly, save for a few minor bugs that I have yet to rectify, like some animations still not working properly, and weapons and creatures spawning when they should not be.