Custom animations with Kinect, from start to finish

This tutorial will tell you how to create custom animations for HL2 (and EP1, not EP2) Machinima (specifically Garrys Mod) with the help of the Kinect.
This has been one of the longest tutorials I’ve ever made, and apart from a few stupid mistakes on my end I’m quite proud of it 😛
There are a lot of requirements and it will take lots of time before you are having your animation up and going for the first time. For this tutorial I used bits of other tutorials: Modifying and Recompiling Source Models from Noesis and Custom Props and Animation with Softimage, from Design 3. The tutorial was supposed to let you record animations and edit them, but due to a stupid mistake of me the data was lost and re-doing was going to take a couple of months. So in this tutorial I assume the BHV recording you make will be perfect 😉

Index:

  1. Requirements
  2. Model Extraction
  3. Model Decompiling
  4. Model Importing in XSI
  5. Model Correction
  6. Weight mapping, Rigging & Tagging
  7. Creating BHV data < Kinect part = fun
  8. Animating with the BHV data <This is not in the tutorial, since this part is so diverse and freakin’ difficult 😉
  9. Exporting from XSI
  10. Using the custom animations

Requirements:

  • Kinect, hooked up to your PC, the XBOX version, not the PC version.
  • Brekel Kinect (Use auto installer to install all Kinect drivers etc. otherwise Kinect won’t work) LINK
  • FAAST LINK
  • GCFScape LINK
  • HL2 games and SDK
  • MDL Decompiler  (place in steamapps\\sourcesdk\bin\ep1\bin) LINK (its the .exe in the root of this zip, there are also some helpful files in them)
  • STUDIOMDL compiler (part of SDK)
  • VTEX compiler (part of SDK)
  • VTFEdit LINK
  • XSI mod tool LINK (and make sure you install the Valve addon)
  • XVI32 Hex editor LINK
  • Model WeightMap LINK

Model Extraction:

Open GCFScape and with that open steamapps\source models.gcf, look for your model you want to animate and drag and drop the 7 files with the modelname into an empty folder. Then open steamapps\source materials.gcf and put the following files with the ones you extracted: Dark_eyeball_l.vmt to pupil_r.vtf Then put some other files with them: citizen_sheet.vmt / vtf and one of the 2 facemap pairs. I used Ted’s. These might be difficult to find, use the search function.

Model Decompiling:

Make sure Steam is running, open the MDL decompiler and point it to the mdl file of the model you put in that empty folder. Then choose another output folder (preferably a subfolder in the folder you extracted the original models in) and click Extract, you should get a message that the model has loaded correctly. Click Ok, then wait for it to decompile. Then you get another message that the work is done! Close the decompiler and open VTFEdit, go to Tools> Convert Folder, choose the folder with the .vmt files you extracted with GCFScape, for output choose a new folder. Make sure to tga is selected and that in the box behind it stands this: *.vtf Just leave all the other settings as they are and click convert. Now in that output folder you should see 12 .tga files. (Optional: In order to see without a TGA editor what’s inside the TGA (texture) files, do exactly the same conversion but instead of tga select bmp as output).

The folder structure I have uptill now:

GCFFiles > Male_02 (extracted files with GCFScape)
GCFFiles > Output_decompiled (decompiled model files)
GCFFiles > Textures_Decompiled (decompiled with VTFEdit)

Model Importing in XSI

Open XSI (this might take a couple of minutes…)

Go to the ValveSource menu > Import SMD,  in the import file bar choose the _reference.smd file and as output folder select Textures_Decompiled. Now you should see a wireframe appear in the various screens. In the right corner right of the word “Wireframe” is a button, click it and the panel (called Camera) will maximize. Now click on the Wireframe option and choose Textured Decal. If your model looks like the first image, go to the Error part, if it looks like the second go to the next chapter (Model Correction) When you press S you can zoom in and out and move the camera with the mouse. Experiment with it.

Error
If your model looks like the first picture here’s how you fix it: Go to View > Rendering/Texturing > Material Manager . Open one of the textures on the left of the screen by double clicking on them. Somewhere in the middle, below the General tab is the path to the tga file. It should be in red, click on the three dots next of it and select the correct file again. Do this with all the files and the model should look like the second picture.

Model Correction
The model  looks OK but has lots of small errors and artifacts left over from decompiling (it isn’t perfect science) so we have to remove them. Open the explorer (either by pressing 8 or by clicking on Scene on the right part of the screen) Now select Mesh > Polygon Mesh > Clusters > EnvelopWeightCls and press delete. This is a duplicate weight map that might cause problems later. Now rename the Mesh folder to something else, a name preferably. Then click on the button Cut (under constrain on the right). Now click on ValveBiped (with middle mouse button if the next thing doesn’t work) and press delete. Now go back to Shaded mode. Go to the explorer, to Clusters and select mouth, now press H to hide that part to prevent us from screwing up the mouth. Now under Select on the right choose Edge and in the dropdown below choose Border Edge and press Ctrl + A. Now Click on the word Select > Tools > RaycastNow zoom in on the neck and while holding Ctrl click on all the neck edges (but not the collar!) so in the end the selection ends up like this:

Now while still having these selected go to Poly. Mesh and select Weld Boundary Edges/Points and in there select All Edges. Close this screen, deselect (Select > Object, ESC key or click outside boundary) your model and go back to shaded, now select the model again and all the edges (apart from the neck) are gone. Switch back to wireframe. Now go to Select and the Raycast tool, Edge selection and Border Edge filter like before. Now this is a tricky part, select a few of the edges you want to have welded together (not all of them!) You’d have to guess what looks the most natural (the three on the bottom for example). Now go again to the Weld Boundary Points/Edges option and the system should have welded the model so a smooth line has formed! 😀 If some of them don’t snap automatically increase the Distance slider in the menu. To quickly get the menu use the middle mouse button on Poly. Mesh (this opens the option last used with that dropdown menu) Switch back to Textured Decal a copule of times to see if the weld’s look ok and that you’re not screwing up. Now the model has no double edges and can be animated without having the chance of seeing holes in the model.

Weight Mapping, Rigging & Tagging

On the top left where it says “Model” click on the dropdown and choose Animate. Open the Scene menu again (press 8) and go to the mouth Cluster again, and press H to unhide. Now select the model, go to ValveSource > Character Rigs > Biped Guide. Now open the Scene menu and middle click on ValveGuide. Go to Wireframe mode and you’ll see some sort of skeletal structure! Now go to ValveSource >Character Rigs > Rig from Guide. The process takes a while and small windows will flash open and stuff will happen to your model. If you get an “Unexpected error” delete the ValveBiped in the Scene window (8…) and go to Character Rigs > Male Rig and it should produce the same result only faster. The rig from guide option is more precise since some models require you to move the Biped Guide elements around. But with humans (HL2 humans to be precise) this option is the fastest. Now open the Scene window > ValveBiped and select everything from Bip01 to UpperBodyParent. On the left go to Store > Transformations – Current Values, and click ok in the window that opens. If you screw up with animations you can revert to this default pose the model is having atm.
Now on to Weight Mapping. This process is to prevent bodyparts to start moving in weird ways if you move, say an arm. In the middle window, on the top left you see a small eye with an arrow beneath it, click on it and select Weight Maps. Then go from Wireframe (or whatever you have) to Hidden Line Removal and you should see a strange disco kinda thing appear over the model:

If you click on one of the arm bulbs (near the hands) and start moving them around (remember to go into default position with Ctrl + z) you’ll see what I mean with bodyparts behaving weird, humans don’t move this way (I hope 😉 )
Now select the model, go to ValveSource > Import WeightMap and select the .xsiwm file you downloaded from the requirement list above (to make it easy to find place it with the model files). Wait a while and the model should get a slightly different disco suit:

Now we are done with all the modelling stuff! It is ready to be animated!
Extra: You can of course fiddle around with the textures with a tga editor and create your own model based on the Citizen! To get it back in the game a QC file is needed. Which will be explained in the Export chapter later on.

Tagging the rig:
We have to do a couple of things in order to let the model move. First we have to tag the rig, to tell XSI which bone is what bodypart. Then we load in a BHV in the next chapter and we have to tag that one as well in order for XSI to create a link between the BVH file skeleton and the tagged rig (tagged hand on the rig  gets linked to the movements from the hand node from the BHV skeleton). It sounds more difficult than it is. First open the explorer (8) and hide the model. Now with the middle mouse button click on ValveBiped, now, on the left click on MOTOR > Tag Rig. In the screen that opens you have to tag these bones by clicking on the name and then on the corresponding bone or control-node of the rig. Try to (or retype it to) get it as these pictures:

Save this to a file (Save option) for future reference, saves a lot of time 😉 Now the tagging of the rig is completed and we are ready to create the BHV file!

Kinect:
First of all we’re gona have some fun, the Kinect is made to be played with 😉 When you go to the brekel site download and install the auto driver installer and then Brekel 0.49 (I don’t recommend 0.5 since the motor/LED driver is having a bit of a problem there) Then you should be able to run programs like NiViewer which is included with the driver installer. Open brekel, we’re not going to work with it but open the Help menu and read trough the topics there, if the motor/LED does not work (slider on the right of the screen) then use the HELP page from Brekel which tells you how to enable those two. Also those help files explain a bit how the program works. Now that the Kinect is up and running close Brekel and download FAAST and extract it somewhere. FAAST lets you link keyboard and mouse commands to movements. Their project page (here) has a list (scroll down to Input Emulator Usage) of all the commands you can use. For example here is the cfg file which I use for minecraft: HERE With FAAST you can play games on ur PC with Kinect 😀
Now back to business. Open Brekel Kinect. Now in the BHV Capture pane select Poser & TrueBones, if you want Write Positions that shouldn’t give any problems and in general they’re a little more precise. Now select a delay and step in the view of the Kinect, and a skeleton will appear on your body! Now press Start Capture BHV and wait for the delay to end by standing in a T pose with your arms. Now act your scene, make sure you don’t move too fast or get bodyparts behind other bodyparts. Then (do it yourself or let someone else(preferred!)) stop the capture.

Basic BHV into XSI!
Now open up XSI again (and our scene of course) and go to MOTOR > Mocap to Rig. In step 1 choose BHV, in 2 find the BHV file you just made with Brekel Kinect, in Step 3 choose Create and another skeleton will appear + another tagging screen. Tag them according to this:

Now save this to a file as well. Now in step 4 enter ValveBiped and finally hit Apply! Now press Shift + H and our model should appear again. On  the bottom are 2 bars with 2 number inputs next to them. I still don’t know what they are used for but  in the top one enter 500 and then press Enter key.  Now press Home and then in the bottom of the screen press the button with a ‘>’ on it and BAAAM our model should do exactly as you did in front of the Kinect! 😀 If it still doesn’t play entirely increase (or decrease) the number you just entered. If you had to stop the capture in Brekel yourself a good way to stop the model form freaking out is to slide the red bar on the bottom, and set an ending point by looking at the frame number it displays. Now enter that number where you just typed 500.

Animating:

This part is too difficult to add to this tutorial, fact is I’m still struggling with it myself! For the sake of this tutorial (otherwise it will never be published :P) we assume that what you recorded in Brekel Kinect is perfect. Of curse that is impossible but still…

Exporting from XSI:

This is very easy, make a new folder (in my case in the GCFFiles) and copy its path. In XSI, in the top menu bar go to ValveSource > Export SMD, in file paste the path and then”\<Modelname>.smd” . Make sure that file type is SMD and leave all the 3 boxes checked. Now click OK. Model exported! 😀 Now, there are Level of Detail models, to render approximations of the model when looked at from a great distance. Concidering we are using this for film making we won’t need them.
Now go to the EP2 source folder form sourcesdk_content\hl2mp\modelsrc\humans_sdk or something similar. And copy:

  • Bodyrules_xsi.qci
  • commonbones.qci
  • hitbox.qci
  • lowesthumanohelpers.qci
  • lowesthumanskeleton.qci
  • ragdoll.qci
  • male.vrd
  • ragdoll.smd
  • ragdoll_pose.smd
  • removehands.qci
  • skeletonanimation.smd
  • standardflex_xsi.qci
  • standardhierarcy.qci
  • standardikchains.qci

Basically everything form humans_sdk and the Male folder in there(not the subfolder in there!). The more the better so I dont have the chance that I forgot one. Dump all these files with the SMD file you created. The example QC file you can find here: http://pastebin.com/4itpPLqt Just save the text as a QC file with the SMD. This default file was created by Noesis Interactive. Now close XSI, start Steam, open the folder you have the TGA files in. And with the help from the QC file delete all (including bmp) unwanted files. Now using https://developer.valvesoftware.com/wiki/Vtex as help compile these 8 TGA files. The folder hierarchy you have this far might change so read it trough. After you followed that site you should get something like Input file:….. Output directory:…. No config file for…. Hit a key to continue from each compilation. Now you should go to the output directory, and in another window go to the GCFFiles  folder. Copy the 8 needed VMT files in the CGFFiles, to the folder with all the VTF files. Now open, one by one, each vmt file and change in $basetexture the following: “models/Humans/Male/pupil_r” becomes “kevin/pupil_r”. Do the same method for the other 7 files. Note that $iris should undergo the same procedure. It might work without this change but you won’t get any custom textures, I haven’t tested it.
Now we’ll go compile the model. Open Source SDK, Engine Orange Box, game HL2 EP2 or one of those. Open a command prompt, and drag and drop either the shortcut or the exe itself from the StudioMDL compiler found in the same folder as the VTEX program. then, in CMD, make a space and then drag and drop the QC file into it. Hit enter. The model should compile! In the end you should see “Completed Kevin.qc” or something like that. Close CMD. The model should be placed into the models folder of the game you compiled with (see the VTEX link) Now you should be able to load it into Faceposer! Problem is that it has alien eyes… Open the QC file again, and under eye data, replace 0.63 with 1.34. Now open the pupil TGA files and, somehow (the tutorial I got suggests GIMP), make them instead of 64*64, 128*128. The canvas size, not the texture. Then recompile them with VTEX. Now in faceposer the alien should be gone 😉 The include command from the QC file lets you use other animations from other models. Inheritance so to speak. This can give quite funny results. Kleiner having alyx’ moves 😀

Using the animation in Choreography:

Now you should be able to use the model and animate it via choreography and via my other tutorials getting it into Gmod! 😀

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s