I recently had a discussion about storing relationships in Maya, and hadn’t realized the role of the message attribute wasn’t this universally cherished thing. In previous posts entitled ‘Don’t use string paths‘, or ‘Why Storing String Refs is Dangerous and Irresponsible‘ I outlined why this is the devil’s work, but in those posts I talked about the API, PyMel and Message Attrs. I didn’t really focus on why message attrs were so important: they serialize node relationships.

For quite some time I have advocated storing relationships with message attrs. At the Maya SIGGRAPH User Event, when they asked me to speak about our modular rigging system, I kind of detailed how we leveraged those at Crytek in CryPed.

I am not quite sure when I started using message attrs to convey relationships, I’m no brainiac, it could have been after seeing this 2003 post from Jason Schleifer on CGTalk:

Or maybe I read it in the Maya docs (unlikely):

“Message attributes only exist to formally declare relationships between nodes. By connecting two nodes via message attributes, a relationship between those nodes is expressed.”

So why does Maya use this, and why should I?

As you read in the docs above, when Maya wants to declare a relationship between a camera and image plane, they do so with a message attribute that connects them. This is important because this bond won’t be broken if the plane or it’s parent is renamed. As soon as you store the string path to a node in the DAG, that data is already stale.  It’s no longer valid.  When you query a message attribute, Maya returns the item, it’s DAG path will be valid, regardless of hierarchy or name changes.

Jason’s example above is maybe the most simple, in my image (a decade later) you can see the messages declaring many relationships abstracting the character at three main levels of interface, Character, ChatacterPart and RigPart. I talked about the basic ideas here in a 2013 post about object oriented python in Maya.

Though Rob vigorously disagreed in the comments there, I am still doing this today.  Here’s an example from the facial code we released in EPIC’s ARTv1 rigging tools some time ago. The face is abstracted on two levels, the ‘face’ and the ‘mask’, here I am only displaying the message connecting them:

By using properties as described in that previous blog post, below I am accessing the system, creating a face instance, walking down the message connection to the mask node, and then asking it for the attach locations. It’s giving me these transforms, by querying the DAG, live:

So, that property looks like this:

    @property
    def attachLocations(self):
        return cmds.listConnections(self.node + '.attachLocations')
    @attachLocations.setter
    def attachLocations(self, locs):
        for loc in locs:
            utils.msgConnect(self.node + '.attachLocations', loc + '.maskNode')

Setting the attach locations through python would look like this, and it would rebuild the message attrs:

face.mask.attachLoactions = ['myLoc1', 'myLoc2']

Working like this, you have to think hard about what a rigger would want to access at what level and expose what’s needed. But in the end, as you see, through python, you have access to everything you need, and none of the data is stale.

How and when to use strings

There are times when the only way you can store a relationship is by using a string in some fashion. Here are some situations and how I have handled them in the past, feel free to leave a comment with your experiences.

• Maya can’t store a relationship to something that doesn’t exist (has been deleted). It can’t store a relationship when it’s not open. In these situations, instead of storing the name in an attr, I stamp the two nodes with a string attr to store the relationship, then you query the world for another node with a matching stamped attr.
• Many times you need to feed your class an initial interface node to build/wrap. Instead of feeding it a string name, you can query the world for node type, in the Ryse example above, the rigging and animation tools could query cmds.ls(type=’CryCharacter’), this would return all characters in the scene. This means all rigging and animation tools needed a common ‘working character’ combobox at the top to define the character the tool is operating on. If you don’t have a node type, you can use a special string attr to query for.
• Sometimes you’re like saving joint names to serialize skinning data or something. You can use message attrs to play it safe here as well. Some pseudocode: For character in characters, if character identifier matches file on disk, for mesh in character.meshes if mesh in file skin it. For joint in character.joints if in file, add them to the skincluster, etc. Here you’re validating all your serialized string data against your class which is traversing the DAG live.
• Message attrs can get SLOW if you’re tracking thousands of items, you should only be tracking important things you would want later. In CryPed, when we wanted to track all nodes that were created when a module was built, we would stamp them all with a string attr that was the function name that built the module. To track this kind of data HarryZ at Crytek had the pragmatic idea of just doing a global ls of the world when a buildout started and then one at the end and boolean them out, this caught all the intermediate and utility nodes and everything generated by the rigging code.

DISCLAIMER/WARNING: Trying to implement this in production I have found other items on top of the massive list that do not work. The ignore names flag ‘ig’ causes a hard crash on file load, the weight precision flag ‘wp’ isn’t implemented though it’s documented, the weight tolerance flag ‘wt’ causes files to hang indefinitely on load. When it does load weights properly, it often does so in a way that crashes the Paint Skin Weights tool. I have reported this in the Maya Beta forums.

Previously I discussed the promise in Maya command ‘deformerWeights‘. The tool that ships with Maya was not very useful, but the code it called was 125 times faster than python if you used it correctly..

Let’s make a python class that can save and load skin weights. You hand it a few hundred skinned meshes (avg Paragon character) and it saves the weights and then you delete history on the meshes, and it loads the weights back on.  What I just described is the process riggers go through when updating a rig or a mesh _every day_.

Below we begin the class, we import a python module to parse XML, and we say “if the user passed in a path, let’s parse it.”

#we import an xml parser that ships with python
import xml.etree.ElementTree
 
#this will be our class, which can take the path to a file on disk
class SkinDeformerWeights(object):
    def __init__(self, path=None):
        self.path = path
 
        if self.path:
            self.parseFile(self.path)

Next, let’s make this parseFile function. Why is parsing the file important? In the last post we found out that there’s a bug that doesn’t appropriately apply saved weights unless you have a skinCluster with the *exact* same joints as were exported. We’re going to read the file and make a skinCluster that works.

#the function takes a path to the file we want to parse
def parseFile(self, path):
    root = xml.etree.ElementTree.parse(path).getroot()
 
    #set the header info
    for atype in root.findall('headerInfo'):
        self.fileName = atype.get('fileName')
 
    for atype in root.findall('weights'):
        jnt = atype.get('source')
        shape = atype.get('shape')
        clusterName = atype.get('deformer')

Now we’re getting some data here, we know that the format can save deformers for multiple shapes, let’s make a shape class and store these.

class SkinnedShape(object):
    def __init__(self, joints=None, shape=None, skin=None, verts=None):
        self.joints = joints
        self.shape = shape
        self.skin = skin
        self.verts = verts

Let’s use that when we parse the file, let’s then store the data we paresed in our new shape class:

#the function takes a path to the file we want to parse
def parseFile(self, path):
    root = xml.etree.ElementTree.parse(path).getroot()
 
    #set the header info
    for atype in root.findall('headerInfo'):
        self.fileName = atype.get('fileName')
 
    for atype in root.findall('weights'):
        jnt = atype.get('source')
        shape = atype.get('shape')
        clusterName = atype.get('deformer')
 
        if shape not in self.shapes.keys():
            self.shapes[shape] = self.skinnedShape(shape=shape, skin=clusterName, joints=[jnt])
        else:
            s = self.shapes[shape]
            s.joints.append(jnt)

So now we have a dictionary of our shape classes, and each knows the shape, cluster name, and all influences. This is important because, if you read the previous post, the weights will only load onto a skinCluster with the exact same number and names of joints.
Now we write a method to apply the weight info we parsed:

def applyWeightInfo(self):
    for shape in self.shapes:
        #make a skincluster using the joints
        if cmds.objExists(shape):
            ss = self.shapes[shape]
            skinList = ss.joints
            skinList.append(shape)
            cmds.select(cl=1)
            cmds.select(skinList)
            cluster = cmds.skinCluster(name=ss.skin, tsb=1)
            fname = self.path.split('\\')[-1]
            dir = self.path.replace(fname,'')
            cmds.deformerWeights(fname , path = dir, deformer=ss.skin, im=1)

And there you go. Let’s also write a method to export/save the skinWeights from a list of meshes so we never have to use the Export DeformerWeights tool:

def saveWeightInfo(self, fpath, meshes, all=True):
    t1 = time.time()
 
    #get skin clusters
    meshDict = {}
    for mesh in meshes:
        sc = mel.eval('findRelatedSkinCluster '+mesh)
        #not using shape atm, mesh instead
        msh =  cmds.listRelatives(mesh, shapes=1)
        if sc != '':
            meshDict[sc] = mesh
        else:
            cmds.warning('>>>saveWeightInfo: ' + mesh + ' is not connected to a skinCluster!')
    fname = fpath.split('\\')[-1]
    dir = fpath.replace(fname,'')
 
    for skin in meshDict:
        cmds.deformerWeights(meshDict[skin] + '.skinWeights', path=dir, ex=1, deformer=skin)
 
    elapsed = time.time()-t1
    print 'Exported skinWeights for', len(meshes), 'meshes in', elapsed, 'seconds.'

You give this a folder and it’ll dump one file per skinCluster into that folder.
Here is the final class we’ve created [deformerWeights.py], and let’s give it a test run.

sdw = skinDeformerWeights()
sdw.saveWeightInfo('e:\\gadget\\', cmds.ls(sl=1))
>>>Exported skinWeights for 214 meshes in 2.433 seconds.

Let’s now load them back, we will iterate through the files in the directory and parse each, applying the weights:

import os
t1=time.time()
path = "e:\\gadget\\"
files = 0
for file in os.listdir(path):
    if file.endswith(".skinWeights"):
        fpath = path + file
        sdw = skinDeformerWeights(path=fpath)
        sdw.applyWeightInfo()
        files += 1
elapsed = time.time() - t1
print 'Loaded skinWeights for', files, 'meshes in', elapsed, 'seconds.'
>>> Loaded skinWeights for 214 meshes in 8.432 seconds.

“>>> Loaded skinWeights for 214 meshes in 8.432 seconds.”

So that’s a simple 50 line wrapper to save and load skinWeights using the deformerWeights command. No longer do we need to write C++ API plugins to save/load weights quickly.

This post was originally going to be entitled ‘Why Everyone Writes their Own Skin Exporter’. Maya’s smooth skin weight export tool hasn’t changed since Maya 4.0 when it was introduced 15 years ago. It saves out greyscale weightmaps in UV space, one image per joint influence per mesh. The only update they have done in 15 years is change the slider to go from a max of 1024 pixels to 4096, and now 8192!

Autodesk understands the need and importance of a skin weight exporter, they even ship it as a C++ example in their Maya Developer Kit / SDK. So why are they still shipping this abomination above?

Like blind data discussed before, in pythonland we cannot set skinweights in one go, we must iterate through the entire mesh setting skin weights one vertex at a time. This means a Maya feature or C++ plugin can save and load skin weights in seconds that take python 15 minutes.

I have written lots of skin weight save/load crap in my time, and as I sat down and started to do that very thing in an instructional blog post one night, we had an interesting discussion in the office the next day. ADSK added ‘Export Deformer Weights‘ in 2011, but it has never really worked. I don’t know a single person who uses it. But it does save and load ‘deformer’ weights via the C++ API –so there’s real promise here!

Save/load skin weights fast without writing a custom Maya plugin? This is kinda like the holy grail, which is ridiculous, but you should see the lengths people go to eek out a little more performance! My personal favorite was MacaroniKazoo back in 2010 reaching in and setting the skinCluster weight list directly by hand using an unholy conglomeration of python API and MEL commands. Tyler Thornock has a post that builds on this here.

So I asked the guys if anyone had looked at Export Deformer Weights recently, everyone either hadn’t heard of it, heard it was shit, or had some real first-hand experience of it being “A bit shit.” But still –the promise was there!

Export Deformer Weights: Broken and Backwards

So, first thing’s first, I made an awesome test case. I am going to go over all the gotchas and things that are broken, but if you don’t want to take this voyage of discovery, skip this section.

I open the deformer weight export tool, and just wow.. I mean the UI team really likes it’s space:

I save my skin weights to an XML file, delete history on the mesh, and open the import UI:

Gotcha 1) It requires a deformer to load weights onto. You need to re-skin your mesh.

I re-skinned my mesh and loaded the XML using Index.. drumroll..

Well, this is definitely not applying the weights back by vertex index. I decided to try ‘Nearest’:

Gotcha 2) Of the options [Index, Nearest, Over], ‘Index’ is somehow lossy, and anything other than ‘Index’ seems to crash often, ‘Nearest’ seems totally borked. (above)

So this was when I just began to think this was a complete waste of my time. I was pretty annoyed that they even shipped a tool like this, something that is so needed and so important, yet crashes frequently and completely trashes your data when it does work.

Not Taking ‘Broken and Backwards’ for an Answer

I am already invested and, not understanding how loading weights by point index could be lossy and broken, I decided to look at the XML file. The tool writes out one XML file per skinCluster, here’s a rundown of the file format:

Mesh Info (Shape) – the vertices of the shape are stored local space x,y,z and corresponding index

<?xml version="1.0"?>
<deformerWeight>
  <headerInfo fileName="C:/Users/chris.evans/Desktop/test_sphere.xml" worldMatrix="1.000000 0.000000 0.000000 0.000000 0.000000 1.000000 0.000000 0.000000 0.000000 0.000000 1.000000 0.000000 0.000000 0.000000 0.000000 1.000000 "/>
  <shape name="pSphereShape1" group="7" stride="3" size="382" max="382">
    <point index="0" value=" 0.148778 -0.987688 -0.048341"/>
    <point index="1" value=" 0.126558 -0.987688 -0.091950"/>
    <point index="2" value=" 0.091950 -0.987688 -0.126558"/>
    ...

Joints (Weights) – There is one block per joint that calls out each vertex that it have influences for on the shape

  <weights deformer="skinCluster1" source="root" shape="pSphereShape1" layer="0" defaultValue="0.000" size="201" max="380">
    <point index="0" value="0.503"/>
    <point index="1" value="0.503"/>
    <point index="2" value="0.503"/>
    ...

And that’s it, not a lot of data, nothing about the skinCluster attributes or options, no support for spaces like UV or world. (odd, since it’s had UV support for 15 years)

Next I decided to run the tool again and see what command it was calling, I then looked up the command documentation and here’s where it gets interesting, go ahead, take a look!

So now I am hooked, someone is putting some thought into this –at least on some level.

I don’t at all understand why the UI has none of these options, but I need to get this working. If you read through the docs, the command also supports:

• Exporting multiple skinClusters/shapes/deformers per XML file
• Exporting skinCluster/deformer attributes like ‘skinningMethod’ and ‘envelope’
• Local and world space positions with a positional tolerance

“Someone is putting some thought into this”

So I started trying to figure out why a file format that explicitly knows every influence of every vertex by index and inf name, doesn’t load weights properly. After some trials I hit gotcha #3:

Gotcha 3) Of the options the weights only load properly if the skinCluster has the *exact* same influences it was saved with. Which really makes no sense, because the file format has the name of every joint in the old skinCluster.

So now I had it working, time to wrap it and make it useful.

The Documentation is a Lie.

So, first thing’s first, I did a speed test.

Importing with deformerWeights was about 125 times faster: Gold mine.

But I just couldn’t get some of the flags to work, I thought I was just a moron, until I finally tried the code example in the ADSK Maya documentation, which FAILS. Let’s first look at the -vc flag, which is required to load using ‘bilinear’ or ‘barycentric’ mapping/extrapolation:

cmds.deformerWeights ("testWeights.xml", ex=True, vc=True, deformer="skinCluster1")
 
# Error: Invalid flag 'vc'
# Traceback (most recent call last):
#   File "<maya console>", line 1, in <module>
# TypeError: Invalid flag 'vc' #

Gotcha 4) The python examples do not work. -vertexConnections flag doesn’t work, -attribute flag doesn’t work, so no saving skincluster metadata like ‘skinningMethod’, etc. Because of that, ‘barycentric’ and other methods that need vertex connection info do not work. The ‘deformer’ flag shows that it takes a list of deformers and writes them all to one file, but this is not true, it takes a single string name of a deformer.

I now know why the UI doesn’t have all these cool options! –they don’t work!

Gotcha 5) It doesn’t take a file path, to save a file to a path you need to specify the filename, and then the path separate.

cmds.deformerWeights ("testWeights.xml", path='d:\\myWeight\\export\\folder\\', ex=True, deformer="skinCluster1")

Perhaps someone fixed this stuff, documented it, and then reverted the fix, but this has been around since 2011.. I tried the above in maya 2016 latest service pack and all my links above are to that version of the documentation.

I wasn’t really intending to write this much, so now that we know this can import weights 125 times faster, we’ll make a tool to utilize it. Stay tuned!

As technically-inclined artists, we often like to create polished UIs, but we have to balance this with not wanting to complicate the user experience (fewer files the better). I tend to not use too many icons with my tools, and my Maya tools often just steal existing icons from Maya: because I know they will be there.

However, you can embed bitmap icons into your PySide/PyQt apps by using the XPM file format, and storing it as a string array. Often I will just place images at the bottom of the file, this keeps icons inside your actual tool, and you don’t need to distribute multiple files or link to external resources.

Here’s an example XPM file:

/* XPM */
static char *heart_xpm[] = {
/* width height num_colors chars_per_pixel */
"7 6 2 1",
/* colors */
"` c None",
". c #e2385a",
/* pixels */
"`..`..`",
".......",
".......",
"`.....`",
"``...``",
"```.```"
};

This is a small heart, you can actually see it, in the header you se the ‘.’ maps to pink, you can see the ‘.’ pattern of a heart. The XPM format is like C, the comments tell you what each block does.
Here’s an example in PySide that generates the above button with heart icon:

import sys
from PySide import QtGui, QtCore
 
def main():
    app = QtGui.QApplication(sys.argv)
    heartXPM = ['7 6 2 1','N c None','. c #e2385a','N..N..N',\
    '.......','.......','N.....N','NN...NN','NNN.NNN']
    w = QtGui.QWidget()
    w.setWindowTitle('XPM Test')
    w.button = QtGui.QPushButton('XPM Bitmaps', w)
    w.button.setIcon(QtGui.QIcon(QtGui.QPixmap(heartXPM)))
    w.button.setIconSize(QtCore.QSize(24,24))
    w.show()
 
    sys.exit(app.exec_())
 
if __name__ == '__main__':
    main()

You need to feed QT a string array, and strip everything out. Gimp can save XPM, but you can also load an image into xnView and save as XPM.

Addendum: Robert pointed out in the comments that pyrcc4, a tool that ships with PyQt4, can compile .qrc files into python files that can be imported. I haven’t tried, but if they can be imported, and are .py files, I guess they can be embedded as well. He also mentioned base64 encoding bitmap images into strings and parsing them. Both these solutions could really make your files much larger than XPM though.

Before I get into the collosal mess that is setting driven keys in Maya, let me start off by saying, when I first made an ‘SDK’ in college, back in 1999, never did I think I would still be rigging like this 15 years later. (Nor did I know that I was setting a ‘driven key’, or a ‘DK’ which sounds less glamorous)

How The Mess is Made

Grab this sample scene [driven_test]. In the file, a single locator with two attributes drives the translation and rotation of two other locators. Can’t get much ismpler than that, but look at this spaghetti mess above! This simple driven relationship created 24 curves, 12 blendWeighted nodes, and 18 unitConversion nodes. So let’s start to take apart this mess. When you set a driven-key relationship, it uses an input and a curve to drive another attribute:

When you have multiple attributes driving a node, maya creates ‘blendWeighted’ nodes, this blends the driven inputs to one scalar output, as you see with the translateX below:

Blending scalars is fairly straight forward, but for rotations, get ready for this craziness: A blendWeighted node cannot take the output of an animCurveUA (angle output), the value must first be converted to radians, then blended. But the final node cannot take radians, so the result must be converted back to an euler angle. This happens for each channel.

If you think this is retarded; welcome to the club. It is a very cool usage of general purpose nodes in Maya, but you wouldn’t think so much of rigging was built on that, would you? That when you create a driven-key it basically makes a bunch of nodes and doesn’t track an actual relationship, because of this, you can’t even reload a relationship into the SDK tool later to edit! (unless you traverse the spaghetti or takes notes or something)

I am in love with Node Edtor, but by default hypergraph did hide some of the spaghetti, it used to hide unitConversions as ‘auxiliary nodes’:

Node Editor shows unitConversions regardless of whether aux nodes are enabled, I would rather see too much and know what’s going on than have things hidden, but maybe that’s just me. You can actually define what nodes are considered aux nodes and whether ‘editors’ show them, but I am way off on a tangent here.

So just go in there and delete the unit conversion yourself and wire the euler angle output, which you would think is a float.. into the blendWeighted input, which takes floats. Maya won’t let you, it creates the unitConversion for you because animCurveUA outputs angles, not floats.

This is why our very simple example file generated over 50 nodes. On Ryse, Maruis’ face has about 73,000 nodes of driven-key spaghetti. (47,382 curves,  1,420 blendWeighted, 24,074 unitConversion)

Finding and traversing

So how do we find this stuff and maybe query and deal with it? Does Maya have some built in way of querying a driven relationship? No. Not that I have ever found. You must become one with the spaghetti! setDrivenKeyframe has a query mode, but that only tells you what ‘driver’ or ‘driven’ nodes/attrs are in the SDK window if it’s open, they don’t query driver or driven relationships!

We know that these intermediate nodes that store the keys are curves, but they’re a special kind of curve, one that doesn’t take time as an input. Here’s how to search for those:

print len(cmds.ls(type=("animCurveUL","animCurveUU","animCurveUA","animCurveUT")))

So what are these nodes? I mentioned above that animCurveUA puts out an angle:

• animCurveUU – curve that takes a double precision float and has a double output
• animCurveUA – takes a double and outputs an angle
• animCurveUL – takes a double and outputs a distance (length)
• animCurveUT – takes a double and outputs a time

When working with lots of driven-key relationships you frequently want to know what is driving what, and this can be very difficult because of all the intermediate node-spaghetti. Here’s what you often want to know:

• What attribute is driving what – for instance, select all nodes an attr drives, so that you can add them back to the SDK tool. ugh.
• What is being driven by an attribute

I wrote a small script/hack to query these relationships, you can grab it here [drivenKeyVisualizer]. Seriously, this is just a UI front end to a 100 line script snippet, don’t let the awesomeness of QT fool you.

The way I decided to go about it was:

1. Find the driven-key curves
2. Create a tiny curve class to store little ‘sdk’ objects
3. List incoming connections (listConnections) to get the driving attr
4. Get the future DG history as a list and reverse it (listHistory(future=1).reverse())
5. Walk the reversed history until I hit a unitConversion or blendWeighted node
6. Get it’s outgoing connection (listConnections) to find the plug that it’s driving
7. Store all this as my sdk objects
8. Loop across all my objects and generate the QTreeWidget

Here’s how I traversed that future history (in the file above):

 #search down the dag for all future nodes
 futureNodes = [node for node in cmds.listHistory(c, future=1, ac=1)]
 #reverse the list order so that you get farthest first
 futureNodes.reverse()
 drivenAttr = None
 
 #walk the list until you find either a unitConversion, blendWeighted, or nothing
 for node in futureNodes:
     if cmds.nodeType(node) == 'unitConversion':
         try:
             drivenAttr = cmds.listConnections(node + '.output', p=1)[0]
             break
         except:
             cmds.warning('blendWeighted node with no output: ' + node)
             break
     elif cmds.nodeType(node) == 'blendWeighted':
         try:
             drivenAttr = cmds.listConnections(node + '.output', p=1)[0]
             break
         except:
             cmds.warning('blendWeighted node with no output: ' + node)
             break
 if not drivenAttr:
     drivenAttr = cmds.listConnections(c + '.output', p=1)[0]
 if drivenAttr:
     dk.drivenAttr = drivenAttr
 else:
     cmds.warning('No driven attr found for ' + c)

This of course won’t work if you have anything like ‘contextual rigging’ that checks the value of an attr and then uses it to blend between two banks of driven-keys, but because this is all general DG nodes, as soon as you enter nodes by hand, it’s no longer really a vanilla driven-key relationship that’s been set.

If you have a better idea, let me know, this above is just a way I have done it that has been robust, but again, I mainly drive transforms.

 What can you do?

Prune Unused Pasta

Pruned version of the driven_test.ma DAG

By definition, when rigging something with many driven transforms like a face, you are creating driven-key relationships based on what you MIGHT need. This goes for when making the initial relationship, or in the future when you maybe want to add detail. WILL I NEED to maybe translate an eyelid xform to get a driven pose I want.. MAYBE.. so you find yourself keying rot/trans on *EVERYTHING*. That’s what I did in my example, and the way the Maya SDK tool works, you can’t really choose which attrs per driven node you want to drive, so best to ‘go hunting with a shotgun’ as we say. (shoot all the trees and hope something falls out)

Ok so let’s write some code to identify and prune driven relationships we didn’t use.

CAUTION: I would only ever do this in a ‘publish’ step, where you take your final rig and delete crap to make it faster (or break it) for animators. Also, I have never used this exact code below in production, I just created it while making this blog post as an example. I have run it on some of our production rigs and haven’t noticed anything terrible, but I also haven’t really looked. 😀

def deleteUnusedDriverCurves():
    for driverCurve in cmds.ls(type=("animCurveUL","animCurveUU","animCurveUA","animCurveUT")):
        #delete unused driven curves
        if not [item for item in cmds.keyframe(driverCurve, valueChange=1, q=1) if abs(item) > 0.0]:
            cmds.delete(driverCurve)
 
deleteUnusedDriverCurves()

This deletes any curves that do not have a change in value. You could have multiple keys, but if there’s no curve, let’s get rid of it. Now that we have deleted some curves, we have some blendWeighted nodes that now aren’t blending anything and unitConversions that are worthless creatures. Let’s take care of them:

def deleteUnusedBlendNodes():
    #rewire blend nodes that aren't blending anything
    for blendNode in cmds.ls(type='blendWeighted'):
        if len(cmds.listConnections(blendNode, destination=0)) == 1:
            curveOutput = None
            drivenInput = None
 
            #leap over the unitConversion if it exists and find the driving curve
            curveOutput = cmds.listConnections(blendNode, destination=0, plugs=1, skipConversionNodes=1)[0]
            #leap over the downstream unitConversion if it exists
            conns = cmds.listConnections(blendNode, source=0, plugs=1, skipConversionNodes=1)
            for conn in conns:
                if cmds.nodeType(conn.split('.')[0]) == 'hyperLayout': conns.pop(conns.index(conn))
            if len(conns) == 1:
                drivenInput = conns[0]
            else:
                cmds.warning('BlendWeighted had more than two outputs? Node: ' + blendNode)
 
            #connect the values, delete the blendWeighted
            cmds.connectAttr(curveOutput, drivenInput, force=1)
            cmds.delete(blendNode)
            print 'Removed', blendNode, 'and wired', curveOutput, 'to', drivenInput
 
deleteUnusedBlendNodes()

We find all blendWeighted nodes with only one input, then traverse out from them and directly connect whatever it was the node was blending, then we delete it. This is a bit tricky and I still think I may have missed something because I wrote this example at 2am, but I ran it on some rigs and haven’t seen an issue.

Here are the results running this code on an example rig:

Create a Tool To Track / Mirror / Select Driven Relationships

This is a prototype I was working on at home but shelved, I would like to pick it up again when I have some time, or would be open to tossing it on github if people would like to help. It’s not rocket science, it’s besically what Maya should have by default. You just want to track the relationships you make, and also select all nodes driven by an attr. Also mirror their transforms across an axis (more geared toward driven transforms).

Write Your Own Driver Node

Many places create their own ‘driven node’ that just stores driven relationships. Judd Simantov showed a Maya node that was used on Uncharted2 to store all facial poses/driven relationships:

The benefits of making your own node are not just in DAG readability, check out the time spent evaluating all these unitConversion and blendWeighted nodes in a complex facial rig (using the new Maya 2015 EXT 1 Profiler tool) –that’s over 760ms! (click to enlarge)

Though it’s not enough to make a node like this, you need to make a front end to manage it, here’s the front end for this node:

Give Autodesk Feedback!

As the PSD request is ‘under review’, maybe when they make the driver, they can make a more general framework to drive things in Maya.

Conclusion

As you can see, there are many ways to try to manage and contain the mess generated by creating driven-key relationships.  I would like to update this based on feedback, it’s definitely not an overview, and speaks more to people who have been creating driven-key relationships for quite some time.  Also, if you would find a tool like my Maya SDK replacement useful, let me know, especially if you would like to help iron it out and/or test it.

Yes! Definitely a clickbait title if there ever was one; but this is important! Some of you contacted me regarding my earlier post (Don’t Use String Paths), where I said you should never use string names to keep track of anything even vaguely important in Maya.

This problem is so fundamental in Maya that the initial Python code test I came up with for the Crytek Technical Art Dept used to be a simple maya node renamer. No joke.

THE PROBLEM

From time to time I see code that attempts reach out into the DAG and grab nodes by concatenating tokens like:

character_side + '_' + character_name + '_arm_' + UID + '_' + type

This is SUPER dangerous. You are basically guessing that an object exists, and betting your whole tool or codebase on this, it’s super fragile.

Let’s use the example of making a turtle, let’s call him ‘Red’, he’s red, and his shell is soft:

cmds.joint(name = 'red' + '_' + shell_hardness + '_' + animal_type + '_hand')
cmds.select('red' + '_' + shell_hardness + '_' + animal_type + '_hand')
# Error: No object matches name: red_soft_turtle_hand
# Traceback (most recent call last):
#   File "", line 1, in 
# ValueError: No object matches name: red_soft_turtle_hand #

First up, you should *never* do this, when you are creating a node with a name you are making from scratch, you *must* first check if it exists. If it exists, Maya will alter the name and you will not be able to find it, your code will immediately break either with the above error, or this one:

cmds.select('red_soft_turtle_hand')
# Error: More than one object matches name: red_soft_turtle_hand
# Traceback (most recent call last):
#   File "", line 1, in 
# ValueError: More than one object matches name: red_soft_turtle_hand #

SLIGHTLY BETTER

So here’s something a bit more safe, but still not recommended in situations where this is important:

hand_jnt = cmds.joint(name = 'red' + '_' + shell_hardness + '_' + animal_type + '_hand')
print hand_jnt
# >>: red_soft_turtle_hand1

Here, we’re storing the node created in a variable, if the name is not the name we expected, we still know how to find the node. This is a lot safer, your code will continue to run, but it’s also not a great way of working. Maya created our joint, but called it ‘red_soft_turtle_hand1’, but, because we stored the node returned from the command creating it into a variable, we can print hand_jnt, and it will return ‘red_soft_turtle_hand1’.

To be super clear, this is more safe because:

• Maya is resolving any name clashes for you automatically
• You get the *real* name returned to you from Maya upon node creation
• If you create a node name that exists elsewhere, it returns you a full path automatically

This is somewhat acceptable in code that’s building a rig. Wham! Bam! You created a node and then did something with it seconds later and never looked back!

STILL A PROBLEM

So let’s say you’re doing the slightly safer way, and you’re storing long paths of nodes which Maya gives you the name of. As soon as you store a long path, it’s stale.  This is what I meant above, the longer you store this information, the less reliable it is.

Let’s look at our joint, its long path is:

|root|pelvis|shell|some_other_joint|arm1|arm2|red_soft_turtle_hand

If *any* of the six parent nodes names change, just a single character! You’re dead in the water. If the hierarchy changes: you’re dead in the water.

If you’re working with someone who defends the above by saying the following

• “Oh, the hierarchy will never change”
• “Oh, there will never be a node with the same name”
• “Oh, no one will ever rename any of these”

The Maya DAG is the wild west.
You WILL have duplicate node names.
You WILL have hierarchy changes.
Professional tools don’t only work with a bunch of caveats,
if it’s a valid Maya scene, your code should work with it:
BE PREPARED.

THE SOLUTIONS

There are clear ways to overcome the challenge above, they’re not secrets, they are actually the professional way Autodesk tells you to do it in the docs, code examples, and their own implementations.

You don’t *have* to work 100% in the API or PyMel, but like we said before, as soon as possible get a pointer to your object. Think of any string path as having a short half-life.

Using The Maya Python API

In C++ when you pass around an object, you have a pointer to it’s location in memory. The name and DAG path can change, but you can at any time get the current name, and path. Fresh, not stale!

import maya.OpenMaya as om
m_dg = om.MFnDagNode()
m_obj = m_dg.create("joint")
m_dagPath = om.MDagPath()
 
if m_obj.hasFn(om.MFn.kDagNode):
    om.MDagPath.getAPathTo( m_obj, m_dagPath )
    print m_dagPath.fullPathName()
# >>: |joint1

Using PyMel

PyMel wraps the API, a PyMel object is a *real* python object that stores the actual MObject, but you don’t have to use the API.

myJoint = joint(name='doesnt_even_matter')
print myJoint.fullPath()
# >>: |root|pelvis|shell|some_other_joint|arm1|arm2|red_soft_turtle_hand|doesnt_even_matter
#In case you never used PyMel, here's some examples of convenience functions:
myJoint.transformationMatrix()
myJoint.inputs()

Notice that PyMel is handing you back python objects, you can ask the object for it’s full path at any time, and it will be fresh data.

Using Message Attrs

Message attributes are how Alias decided users will serialize relationships between nodes. I have removed this section because future-me wrote an entire post dedicated to this (The Mighty Message Attribute).

MAKING A PLAN

The important thing here is that you decide a consistent way of using the above. At Crytek we tried to have all core functions that accepted or passed DG nodes, do the handoff as MObjects. We serialized all important relationships with message attrs, and when there was too much data for that to be efficient, we stamped nodes with metadata.

RETROFITTING

If you have an existing pipeline where you build a lot of string references by concatenating tokens like the example above, you can make a convenience function that will validate your assumptions and deal gracefully with issues. Something like my_utils.get_node(‘my’ + ‘_’ + ‘thing’)

I have written many tools at different companies, I taught myself, and don’t have a CS degree. I enjoy shot-sculpting, skinning, and have been known to tweak parameters of on-screen visuals for hours; I don’t consider myself a ‘coder’; still can’t allocate my own memory.  I feel I haven’t really used OOP from an architecture standpoint. So I bought an OOP book, and set out on a journey of self improvement.

‘OOP’ In Maya

In Maya, you often use DG nodes as ‘objects’. At Crytek we have our own modular nodes that create meta-frameworks encapsulating the character pipeline at multiple levels (characters, characterParts, and rigParts). Without knowing it, we were using Object Oriented Analysis when designing our frameworks, and even had some charts that look quite a bit like UML. DG node networks are connected often with message nodes, this is akin to a pointer to the object in memory, whereas with a python ‘object’ I felt it could always easily lose it’s mapping to the scene.

It is possible now with the OpenMaya C++ API to store a pointer to the DG node in memory and just request the full dag path any time you want it, also PyMel objects are Python classes and link to the DG node even when the string name changes.

“John is 47 Years Old and 6 Feet Tall”

Classes always seemed great for times when I had a bunch of data objects, the classic uses are books in a library, or customers: John is 47 years old and likes the color purple. Awesome. However, in Maya, all our data is in nodes already, and those nodes have attributes, those attributes serialize into a Maya file when I save: so I never really felt the need to use classes.

Although, all this ‘getting’, ‘setting’ and ‘listing’ really grows tiresome, even when you have custom methods to do it fairly easily.

It was difficult to find any really useful examples of OOP with classes in Maya. Most of our code is for ‘constructing’: building a rig, building a window, etc. Code that runs in a linear fashion and does ‘stuff’. There’s no huge architecture, the architecture is Maya itself.

Class Warfare

I wanted to package my information in classes and pass that back and forth in a more elegant way –at all times, not just while constructing things. So for classes to be useful to me, I needed them to synchronously exist with DG nodes.

I also didn’t want to have to get and set the information when syncing the classes with DG nodes, that kind of defeats the purpose of Python classes IMO.

Any time I opened a tool I would ‘wrap’ DG nodes in classes that harnessed the power of Python and OOP. To do this meant diving into more of the deep end, but since that was what was useful to me, that’s what I want to talk about here.

To demonstrate, let’s construct this example:

#the setup
loc = cmds.spaceLocator()
cons = [cmds.circle()[0], cmds.circle()[0]]
meshes = [cmds.sphere()[0], cmds.sphere()[0], cmds.sphere()[0]]
cmds.addAttr(loc, sn='controllers', at='message')
cmds.addAttr(cons, sn='rigging', at='message')
for con in cons: cmds.connectAttr(loc[0] + '.controllers', con + '.rigging')
cmds.addAttr(loc, sn='rendermeshes', at='message')
cmds.addAttr(meshes, sn='rendermesh', at='message')
for mesh in meshes: cmds.connectAttr(loc[0] + '.rendermeshes', mesh + '.rendermesh')

So now we have this little node network:

Now if I wanted to wrap this network in a class. We are going to use @property to give us the functionality of an attribute, but really a method that runs to return us a value (from the DG node) when the ‘attribute’ is queried. I believe using properties is key to harnessing the power of classes in Maya.

class GameThing(object):
	def __init__(self, node):
		self.node = node
 
	#controllers
	@property
	def controllers(self):
		return cmds.listConnections(self.node + ".controllers")

So now we can query the ‘controllers’ attribute/property, and it returns our controllers:

test = GameThing(loc)
print test.controllers
##>>[u'nurbsCircle2', u'nurbsCircle1']

Next up, we add a setter, which runs code when you set a property ‘attribute’:

class GameThing(object):
	def __init__(self, node):
		self.node = node
 
	#controllers
	@property
	def controllers(self):
		return cmds.listConnections(self.node + ".controllers")
 
	@controllers.setter
	def controllers(self, cons):
		#disconnect existing controller connections
		for con in cmds.listConnections(self.node + '.controllers'):
			cmds.disconnectAttr(self.node + '.controllers', con + '.rigging')
 
		for con in cons:
			if cmds.objExists(con):
				if not cmds.attributeQuery('rigging', n=con, ex=1):
					cmds.addAttr(con, longName='rigging', attributeType='message', s=1)
				cmds.connectAttr((self.node + '.controllers'), (con + '.rigging'), f=1)
			else:
				cmds.error(con + ' does not exist!')

So now when we set the ‘controllers’ attribute/property, it runs a method that blows away all current message connections and adds new ones connecting your cons:

test = GameThing(loc)
print test.controllers
##>>[u'nurbsCircle2', u'nurbsCircle1']
test.controllers = [cmds.nurbsSquare()[0]]
print test.controllers
##>>[u'nurbsSquare1']

To me, something like properties makes classes infinitely more useful in Maya. For a short time we tried to engineer a DG node at Crytek that when an attr changed, could eval a string with a similar name on the node. This is essentially what a property can do, and it’s pretty powerful. Take a moment to look through code of some of the real ‘heavy lifters’ in the field, like zooToolBox, and you’ll see @property all over the place.

I hope you found this as useful as I would have.

I am still finding my feet in Maya, on my project, some files have grown to 800mb in size. Things get corrupt, hand editing MAs is common; I am really learning some of the internals.

In the past week I have had to do a lot of timeline walking to switch coord spaces and get baked animations into and out of hierarchies. In 3dsMax you can do a loop and evaluate a node ‘at time i’, and there is no redraw or anything. I didn’t know how to do this in Maya.

I previously did this with looping cmds.currentTime(i) and ‘walking the timeline’, however, you can set the time node directly like so: cmds.setAttr(“time1.outTime”, int(i))

Unparenting a child with keyed compensation (1200 frames)
10.0299999714 sec – currentTime
2.02 sec – setAttr

There are some caveats, whereas in a currentTime loop you can just cmds.setKeyframe(node), I now have to cmds.setKeyframe(node, time=i). But when grabbing a matrix, I don’t need to pass time and it works, I don’t think you can anyway.. I guess it gets time from the time node.

Here’s a sample loop that makes a locator and copies a nodes animation to world space:

#function feeds in start, end, node
	if not start: start = cmds.playbackOptions(minTime=1, q=1)
	if not end: end = cmds.playbackOptions(maxTime=1, q=1)
	loc = cmds.spaceLocator(name='parentAlignHelper')
	for i in range(start, (end+1)):
		cmds.setAttr("time1.outTime", int(i))
		matrix = cmds.xform(node, q=1, ws=1, m=1)
		cmds.xform(loc, ws=1, m=matrix)
		cmds.setKeyframe(loc, time=i)

Maybe it’s me, but I often find myself parsing weird ascii text files from others. Sometimes the authors knew what the data was and there’s no real markup. Take this joint list for example:

143 # bones
root ground
-1
0 0 0
root hips
0
0 0.9512207 6E-08
spine 1
1
4E-08 0.9522207 1.4E-07
spine 2
2
3E-07 1.0324 8.3E-07
spine 3
3
5.6E-07 1.11357 1.53E-06
spine 4
4
8.2E-07 1.194749 2.22E-06
head neck lower

So the first line is the number of joints then it begins in three line intervals stating from the root outwards: joint name, parent integer, position. I used to make a pretty obtuse loop using a modulus operator. Basically, modulus is the remainder left over after division. So X%Y gives you the remainder of X divided by Y; here’s an example:

for i in range(0,20+1):
	if i%2 == 0: print i
#>> 0
#>> 2
#>> 4
#>> 6
#>> 8
#>> 10

The smart guys out there see where this is goin.. so I never knew range had a ‘step’ argument. (Or I believe I did, I think I actually had this epiphany maybe two years ago, but my memory is that bad.) So parsing the above is as simple as this:

jnts = []
for i in range(1,numJnts*3+1,3):
	jnt = lines[i].strip()
	parent = int(lines[i+1].strip())
	posSplit = lines[i+2].strip().split(' ')
	pos = (float(posSplit[0])*jointScale, \
	float(posSplit[1])*jointScale, float(posSplit[2])*jointScale)
	jnts.append([jnt, parent, pos])

Thanks to phuuchai on #python (efnet) for nudging me to RTFM!

I gave a workshop/talk at SIGGRAPH geared toward introducing people to Python. There were ~25 people on PCs following along, and awkwardly enough, many more than that standing and watching. I prefaced my talk with the fact that I am self-taught and by no means an expert. That said, I have created many python tools people use every day at industry-leading companies.

Starting from zero, in the next hour I aimed to not only introduce them to Python, but get them doing cool, usable things like:

• Iterating through batches/lists
• Reading / writing data to excel files
• Wrangling data from one format to another in order to create a ‘tag cloud’

Many people have asked for the notes, and I only had rough notes. I love Python, and I work with this stuff every day, so I have had to really go back and flesh out some of what I talked about. This tutorial has a lot less of the general chit-chat and information. I apologize for that.

Installation / Environment Check

Let’s check to see that you have the tools properly installed. If you open the command prompt and type ‘python’ you should see this:

So Python is correctly installed, for the following you can either follow along in the cmd window (more difficult) or in IDLE, the IDE that python ships with (easier). This can be found by typing IDLE into the start menu:

Variables

Variables are pieces of information you store in memory, I will talk a bit about different types of variables.

Strings

Strings are pieces of text. I assume you know that, so let’s just go over some quick things:

string = 'this is a string'
print string
#>>this is a string
num = '3.1415'
print num
#>>3.1415

One thing to keep in mind, the above is a string, not a number. You can see this by:

print num + 2
#>>Traceback (most recent call last):
#>>  File "basics_variables.py", line 5, in
#>>    print num + 2
#>>TypeError: cannot concatenate 'str' and 'int' objects

Python is telling you that you cannot add a number to a string of text. It does not know that ‘3.1415’ is a number. So let’s convert it to a number, this is called ‘casting’, we will ‘cast’ the string into a float and back:

print float(num) + 2
#>>5.1415
print str(float(num) + 2) + ' addme'
#>>5.1415 addme

Lists

Lists are the simplest ways to store pieces of data. Let’s make one by breaking up a string:

txt = 'jan tony senta michael brendon phillip jonathon mark'
names = txt.split(' ')
print names
#>>['jan', 'tony', 'senta', 'michael', 'brendon', 'phillip', 'jonathon', 'mark']
for item in names: print item
#>>jan
#>>tony
#>>senta
#>>michael
...

Split breaks up a string into pieces. You tell it what to break on, above, I told it to break on spaces txt.split(‘ ‘). So all the people are stored in a List, which is like an Array or Collection in some other languages.
You can call up the item by it’s number starting with zero:

print names[0], names[5]
#>>jan phillip

TIP: [-1] index will return the last item in an array, here’s a quick way to get a file from a path:

path = 'D:\\data\\dx11_PC_(110)_05_09\\Tools\\CryMaxInstaller.exe'
print path.split('\\')[-1]
#>>CryMaxInstaller.exe

Dictionaries

These store keys, and the keys reference different values. Let’s make one:

dict = {'sascha':'tech artist', 'harry': 142.1, 'sean':False}
print dict['sean']
#>>False

So this is good, but these are just the keys, we need to know the values. Here’s another way to do this, using .keys()

dict = {'sascha':'tech artist', 'harry': 142.1, 'sean':False}
for key in dict.keys(): print key, 'is', dict[key]
#>>sean is False
#>>sascha is tech artist
#>>harry is 142.1

So, dictionaries are a good way to store simple relationships of key and value pairs. In case you hadn’t notices, I used some ‘floats’ and ‘ints’ above. A float is a number with a decimal, like 3.1415, and an ‘int’ is a whole number like 10.

Creating Methods (Functions)

A method or function is like a little tool that you make. These building blocks work together to make your program.

Let’s say that you have to do something many times, you want to re-use this code and not copy/paste it all over. Let’s use the example above of names, let’s make a function that takes a big string of names and returns an ordered list:

def myFunc(input):
	people = input.split(' ')
	people = sorted(people)
	return people
txt = 'jan tony senta michael brendon phillip jonathon mark'
orderedList = myFunc(txt)
print orderedList
#>>['brendon', 'jan', 'jonathon', 'mark', 'michael', 'phillip', 'senta', 'tony']

Basic Example: Create A Tag Cloud From an Excel Document

So we have an excel sheet, and we want to turn it into a hip ‘tag cloud’ to get people’s attention.
If we go to http://www.wordle.net/ you will see that in order to create a tag cloud, we need to feed it the sentences multiple times, and we need to put a tilde in between the words of the sentence. We can automate this with Python!

First, download the excel sheet from me here: [info.csv] The CSV filetype is a great way to read/write docs easily that you can give to others, they load in excel easily.

file = 'C:\\Users\\chris\\Desktop\\intro_to_python\\info.csv'
f = open(file, 'r')
lines = f.readlines()
f.close()
print lines
#>> ['always late to work,13\n', 'does not respect others,1\n', 'does not check work properly,5\n', 'does not plan properly,4\n', 'ignores standards/conventions,3\n']

‘\n’ is a line break character, it means ‘new line’, we want to get rid of that, we also want to just store the items, and how many times they were listed.

file = 'C:\\Users\\chris\\Desktop\\intro_to_python\\info.csv'
f = open(file, 'r')
lines = f.readlines()
f.close()
dict = {}
for line in lines:
	split = line.strip().replace(' ','~').split(',')
	dict[split[0]] = int(split[1])
print dict
#>>{'ignores~standards/conventions': 3, 'does~not~respect~others': 1, 'does~not~plan~properly': 4, 'does~not~check~work~properly': 5, 'always~late~to~work': 13}

Now we have the data in memory in an easily readable way, let’s write it out to disk.

output = ''
for key in dict.keys():
	for i in range(0,dict[key]): output += (key + '\n')
f = open('C:\\Users\\chris\\Desktop\\intro_to_python\\test.txt', 'w')
f.write(output)
f.close()

There we go. In one hour you have learned to:

• Read and write excel files
• Iterate over data
• Convert data sets into new formats
• Write, read and alter ascii files

If you have any questions, or I left out any parts of the presentation you liked, reply here and I will get back to you.

I recently wrote a custom tool to diff CryEngine layer files in P4, and was surprised how simple it was. What follows is a quick tutorial on adding custom python tools to Perforce.

Start by heading over to Tools>Manage Custom Tools… Then click ‘New’:

You can pass a lot of information to an external tool, here is a detailed rundown. As you see above, we pass the client spec (local) file name (%f) to a python script, let’s create a new script called ‘custom_tool.py’:

import sys
from PyQt4 import QtGui    
 
class custom_tool(QtGui.QMessageBox):
	def __init__(self, parent=None):
		QtGui.QMessageBox.__init__(self)
		self.setDetailedText(str(sys.argv))
		self.show()
 
if __name__ == "__main__":
	app = QtGui.QApplication(sys.argv)
	theTool = custom_tool()
	theTool.show()
	sys.exit(app.exec_())

What this does is simply spits out the sys.argv in a way you can see it. So now you can feed any file you right click in Perforce into a python script:

If you would like to actually do something with a file or revision on the server and are passing the %F flag to get the depot file path, you then need to use p4 print to redirect the file contents (non-binary) to a local file:

p4.run_print('-q', '-o', depotFile, localFile)

Perforce is a wily beast. A lot of companies use it, but I feel few outside of the IT department really have to deal with it much. As I work myself deeper and deeper into the damp hole that is asset validation, I have really been writing a lot of python to deal with certain issues; but always scripts that work from the outside.

Perforce has a system that allows you to write scripts that are run, server side, when any number of events are triggered. You can use many scripting languages, but I will only touch on Python.

Test Environment

To follow along here, you should set up a test environment. Perforce is freely downloadable, and free to use with 2 users. Of course you are going to need python, and p4python. So get your server running and add two users, a user and an administrator.

Your First Trigger

Let’s create the simplest python script. It will be a submit trigger that says ‘Hello World’ then passes or fails. If it passes, the item will be checked in to perforce, if it fails, it will not. exiting while returning a ‘1’ is considered a fail, ‘0’ a pass.

print 'Hello World!'
print 'No checkin for you!'
sys.exit(1)

Ok, so save this file as hello_trigger.py. Now go to a command line and enter ‘p4 triggers’ this will open a text document, edit that document to point to your trigger, like so (but point to the location of your script on disk):

Triggers:
	hello_trigger change-submit //depot/... "python X:/projects/2010/p4/hello_trigger.py"

Close/save the trigger TMP file, you should see ‘Triggers saved.’ echo’d at the prompt. Now, when we try to submit a file to the depot, we will get this:

So: awesome, you just DENIED your first check-in!

Connecting to Perforce from Inside a Trigger

So we are now denying check-ins, but let’s try to do some other things, let’s connect to perforce from inside a trigger.

from P4 import P4, P4Exception
 
p4 = P4()
 
try:
	#use whatever your admin l/p was
	#this isn't the safest, but it works at this beginner level
	p4.user = "admin"
	p4.password = "admin"
	p4.port = "1666"
	p4.connect()
	info = p4.run("info")
	print info
	sys.exit(1)
 
#this will return any errors
except P4Exception:
	for e in p4.errors: print e
	sys.exit(1)

So now when you try to submit a file to depot you will get this:

Passing Info to the Trigger

Now we are running triggers, accepting or denying checkins, but we really don’t know much about them. Let’s try to get enough info to where we could make a decision about whether or not we want the file to pass validation. Let’s make another python trigger, trigger_test.py, and let’s query something from the perforce server in the submit trigger. To do this we need to edit our trigger file like so:

Triggers:
	test change-submit //depot/... "python X:/projects/2010/p4/test_trigger.py %user% %changelist%"

This will pass the user and changelist number into the python script as an arg, the same way dragging/dropping passed args to python in my previous example. So let’s set that up, save the script from before as ‘test_trigger.py’ as shown above, and add the following:

import sys
from P4 import P4, P4Exception
 
p4 = P4()
describe = []
 
try:
	p4.user = "admin"
	p4.password = "admin"
	p4.port = "1666"
	p4.connect()
 
except P4Exception:
	for e in p4.errors: print e
	sys.exit(1)
 
print str(sys.argv)
describe = p4.run('describe',sys.argv[2])
print str(describe)
 
p4.disconnect()
sys.exit(1)

So, as you can see, it has returned the user and changelist number:

However, for this changelist to be useful, we query p4, asking the server to describe the changelist. This returns a lot of information about the changelist.

Where to Go From here

The few simple things shown here really give you the tools to do many more things. Here are some examples of triggers that can be  created with the know-how above:

• Deny check-ins of a certain filetype (like deny compiled source files/assets)
• Deny check-ins whose hash digest matches an existing file on the server
• Deny/allow a certain type of file check-in from a user in a certain group
• Email a lead any time a file in a certain folder is updated

Did you find this helpful? What creative triggers have you written?

In Python, a Decorator is a type of macro that allows you to inject or modify code in functions or classes. I was turned onto this by my friend Matt Chapman at ILM, but never fully grasped the importance.

class myDecorator(object):
	def __init__(self, f):
		self.f = f
	def __call__(self):
		print "Entering", self.f.__name__
		self.f()
		print "Exited", self.f.__name__
 
@myDecorator
def aFunction():
	print "aFunction running"
 
aFunction()

When you run the code above you will see the following:

>>Entering aFunction
>>aFunction running
>>Exited aFunction

So when we call a decorated function, we get a completely different behavior. You can wrap any existing functions, here is an example of wrapping functions for error reporting:

class catchAll:
	def __init__(self, function):
		self.function = function
 
	def __call__(self, *args):
		try:
			return self.function(*args)
		except Exception, e:
			print "Error: %s" % (e)
 
@catchAll
def unsafe(x):
  return 1 / x
 
print "unsafe(1): ", unsafe(1)
print "unsafe(0): ", unsafe(0)

So when we run this and divide by zero we get:

unsafe(1):  1
unsafe(0):  Error: integer division or modulo by zero

Using decorators you can make sweeping changes to existing code with minimal effort, like the error reporting function above, you could go back and just sprinkle these in older code.

I have really been trying to learn some Python fundamentals lately, reading some books and taking an online class. So: wow. I can’t believe that I have written so many tools, some used by really competent people at large companies, without really understanding polymorphism and other basic Python concepts.

Here’s an example of my sequence method from before, but making it a class using special class methods:

class imSequence:
	def __init__(self, file):
		dir = os.path.dirname(file)
		file = os.path.basename(file)
		segNum = re.findall(r'\d+', file)[-1]
		self.numPad = len(segNum)
		self.baseName = file.split(segNum)[0]
		self.fileType = file.split('.')[-1]
		globString = self.baseName
		for i in range(0,self.numPad): globString += '?'
		self.images = glob.glob(dir+'\\'+globString+file.split(segNum)[1])
 
	def __len__(self):
		return len(self.images)
 
	def __iter__(self):
		return iter(self.images)

Here’s an example of use:

seq = imSequence('seq\\test_00087.tga')
print len(seq)
>>94
print 'BaseName: %s  FileType: %s  Padding: %s' % (seq.baseName, seq.fileType, seq.numPad)
>>BaseName: test_  FileType: tga  Padding: 5
for image in seq: print image
>>seq\test_00000.tga
>>seq\test_00001.tga
>>seq\test_000002.tga
...

[More info and examples: Dive Into Python: Special Class Methods]

I have been parsing through the files of other people a lot lately, and finally took the time to make a little function to give me general information about a sequence of files. It uses regex to yank the numeric parts out of a filename, figure out the padding, and glob to tell you how many files in the sequence. Here’s the code and an example usage:

#returns [base name, padding, filetype, number of files, first file, last file]
def getSeqInfo(file):
	dir = os.path.dirname(file)
	file = os.path.basename(file)
	segNum = re.findall(r'\d+', file)[-1]
	numPad = len(segNum)
	baseName = file.split(segNum)[0]
	fileType = file.split('.')[-1]
	globString = baseName
	for i in range(0,numPad): globString += '?'
	theGlob = glob.glob(dir+'\\'+globString+file.split(segNum)[1])
	numFrames = len(theGlob)
	firstFrame = theGlob[0]
	lastFrame = theGlob[-1]
	return [baseName, numPad, fileType, numFrames, firstFrame, lastFrame]

Here is an example of usage:

print getSeqInfo('E:\\data\\data\\Games\\Project\\CaptureOutput\\Frame000547.jpg')
>>['Frame', 6, 'jpg', 994, 'E:\\data\\data\\Games\\Project\\CaptureOutput\\Frame000000.jpg', 'E:\\data\\data\\Games\\Project\\CaptureOutput\\Frame000993.jpg']

I know this is pretty simple, but I looked around a bit online and didn’t see anything readily available showing how to deal with different numbered file sets. I have needed something like this for a while that will work with anything from OBJs sent from external contractors, to images from After Effects…

So I have always been wondering how you can create almost like a ‘droplet’ to steal the photoshop lingo, from a python script. A while ago I came across some sites showing how to edit shellex in regedit to allow for files to be dropped on any python script and fed to it as args (Windows).

It’s really simple, you grab this reg file [py_drag_n_drop.reg] and install it.

Now when you drop files onto a python script, their filenames will be passed as args, here’s a simple script to test.

import sys
 
f = open('c:\\tmp.txt', 'w')
for arg in sys.argv:
    f.write(arg + '\n')
f.close()

When you save this, and drop files onto its icon, it will create tmp.txt, which will look like this:

X:\projects\2010\python\drag_and_drop\drag_n_drop.py
X:\photos\2010.04 - easter weekend\fuji\DSCF9048.MPO
X:\photos\2010.04 - easter weekend\fuji\DSCF9049.MPO
X:\photos\2010.04 - easter weekend\fuji\DSCF9050.MPO
X:\photos\2010.04 - easter weekend\fuji\DSCF9051.MPO
X:\photos\2010.04 - easter weekend\fuji\DSCF9052.MPO

The script itself is the first arg, then all the files. This way you can easily create scripts that accept drops to do things like convert files, upload files, etc..

I have been really amazing myself at how much knowledge I have forgotten in the past five or six months… Most of the work I did in the past year utilized the UIC module to load UI files directly, but I can find very little information about this online. I was surprised to see that even the trusty old Rapid GUI Programming with Python and Qt book doesn’t cover loading UI files with the UIC module.

So, here is a tiny script with UI file [download] that will generate a pyqt example window that does ‘stuff’:

import sys
from PyQt4 import QtGui, QtCore, uic
 
class TestApp(QtGui.QMainWindow):
	def __init__(self):
		QtGui.QMainWindow.__init__(self)
 
		self.ui = uic.loadUi('X:/projects/2010/python/pyqt_tutorial/pyqt_tutorial.ui')
		self.ui.show()
 
		self.connect(self.ui.doubleSpinBox, QtCore.SIGNAL("valueChanged(double)"), spinFn)
		self.connect(self.ui.comboBox, QtCore.SIGNAL("currentIndexChanged(QString)"), comboFn)
		self.connect(self.ui.pushButton, QtCore.SIGNAL("clicked()"), buttonFn)
 
def spinFn(value):
	win.ui.doubleSpinBoxLabel.setText('doubleSpinBox is set to ' + str(value))
def buttonFn():
	win.ui.setWindowTitle(win.ui.lineEdit.text())
def comboFn(value):
	win.ui.comboBoxLabel.setText(str(value) + ' is selected')
 
if __name__ == "__main__":
	app = QtGui.QApplication(sys.argv)
	win = TestApp()
	sys.exit(app.exec_())

Change the path to reflect where you have saved the UI file, and when you run the script you should get this:

EDIT: A few people have asked me to update this for other situations

PySide Inside Maya:

import sys
from PySide.QtUiTools import *
from PySide.QtCore import *
from PySide.QtGui import *
 
class TestApp(QMainWindow):
	def __init__(self):
		QMainWindow.__init__(self)
 
		loader = QUiLoader()
		self.ui = loader.load('c:/pyqt_tutorial.ui')
		self.ui.show()
 
		self.connect(self.ui.doubleSpinBox, SIGNAL("valueChanged(double)"), spinFn)
		self.connect(self.ui.comboBox, SIGNAL("currentIndexChanged(QString)"), comboFn)
		self.connect(self.ui.pushButton, SIGNAL("clicked()"), buttonFn)
 
def spinFn(value):
	win.ui.doubleSpinBoxLabel.setText('doubleSpinBox is set to ' + str(value))
def buttonFn():
	win.ui.setWindowTitle(win.ui.lineEdit.text())
def comboFn(value):
	win.ui.comboBoxLabel.setText(str(value) + ' is selected')
 
win = TestApp()

PyQT Inside Maya:

import sys
from PyQt4 import QtGui, QtCore, uic
 
class TestApp(QtGui.QMainWindow):
	def __init__(self):
		QtGui.QMainWindow.__init__(self)
 
		self.ui = uic.loadUi('c:/pyqt_tutorial.ui')
		self.ui.show()
 
		self.connect(self.ui.doubleSpinBox, QtCore.SIGNAL("valueChanged(double)"), spinFn)
		self.connect(self.ui.comboBox, QtCore.SIGNAL("currentIndexChanged(QString)"), comboFn)
		self.connect(self.ui.pushButton, QtCore.SIGNAL("clicked()"), buttonFn)
 
def spinFn(value):
	win.ui.doubleSpinBoxLabel.setText('doubleSpinBox is set to ' + str(value))
def buttonFn():
	win.ui.setWindowTitle(win.ui.lineEdit.text())
def comboFn(value):
	win.ui.comboBoxLabel.setText(str(value) + ' is selected')
 
win = TestApp()

I have gotten back into some pyqt in my spare time, just because it’s what I used on a daily basis at the last place I worked at. However, I had trouble getting it to run in my text editor of choice. (SciTE)

I couldn’t find a solution with like 45 minutes of googling. When trying to import PyQt4 it would give me a dll error, but I could paste the code into IDLE and it would execute fine. I found a solution by editing the python preferences of SciTE. I noticed that it wasn’t running python scripts the way IDLE was, but compiling them (?). I edited the last line to just run the script, and viola! It worked.

Find this line (usually the last):

command.1.*.py=python -c "import py_compile; py_compile.compile(r'$(FilePath)')"

And change it to:

command.1.*.py=python "(r'$(FilePath)')"

I don’t really know if this messes anything else up, but it does allow the PyQt4 libs to load and do their thing.

The Vatican recently put up an interactive Sistine Chapel flash application. You can pan around the entire room and zoom in and out in great detail.

The Vatican is not very open with it’s art, the reason they scream ‘NO PHOTO’ when you pull a camera out in the chapel is that they sold the ability to take photos of it to a Japanese TV Station (Nippon TV) for 4.2 million dollars. Because the ceiling has long been in the public domain, the only way they can sell ‘the right to photograph’ the ceiling is by screwing over us tourists who visit. If you take a photo, they have no control over that image –because they don’t own the copyright of the work.

Many of you who know me, know I am a huge fan of Michelangelo’s work, this data was just too awesomely tempting and when I saw it posted publicly online, I really wanted to get my hands on the original assets.

Here is a python script to grab all of the image tiles that the flash app reads, and then generate the 8k faces of the cubemap. In the end you will have a 32,000 pixel cubemap.

First we copy the swatches from the website:

def getSistineCubemap(saveLoc):
	import urllib
	#define the faces of the cubemap, using their own lettering scheme
	faces = ['f','b','u','d','l','r']
	#location of the images
	url = 'http://www.vatican.va/various/cappelle/sistina_vr/Sistine-Chapel.tiles/l3_'
	#copy all the swatches to your local drive
	for face in faces:
		for x in range(1,9):
			for y in range(1,9):
				file = (face + '_' + str(y) + '_' + str(x) + '.jpg')
				urllib.urlretrieve((url + face + '_' + str(y) + '_' + str(x) + '.jpg'), (saveLoc + file))
				urllib.urlcleanup()
				print "saved " + file

Next we use PIL to stitch them together:

def stitchCubeMapFace(theImage, x, y, show):
	from PIL import Image, ImageDraw
	from os import path
 
	file = theImage.split('/')[-1]
	fileSplit = file.split('_')
	im = Image.open(theImage)
	#create an 8k face from the first swatch
	im = im.resize((8000, 8000), Image.NEAREST)
	thePath = path.split(theImage)[0]
 
	xPixel = 0
	yPixel = 0
	#loop through the swatches, stitching them together
	for y_ in range(1, x+1):
		for x_ in range(1,y+1):
			if yPixel == 8000:
				yPixel = 0
			nextImage = (thePath + '/' + fileSplit[0] + '_' + str(x_) + '_' + str(y_) + '.jpg')
			print ('Merging ' + nextImage + ' @' + str(xPixel) + ',' + str(yPixel))
			loadImage = Image.open(nextImage)
			im.paste(loadImage, (xPixel, yPixel))
			yPixel += 1000
		xPixel += 1000
	saveImageFile = (thePath + '/' + fileSplit[0] + '_face.jpg')
	print ('Saving face: ' + saveImageFile)
	#save the image
	im.save(saveImageFile, 'JPEG')
	#load the image in default image viewer for checking
	if show == True:
		import webbrowser
		webbrowser.open(saveImageFile)

Here is an example of the input params:

getSistineCubemap('D:/sistineCubeMap/')
stitchCubeMapFace('D:/sistineCubeMap/r_1_1.jpg', 8, 8, True)

So, before we looked at just outputting a list of the files that were on device1 and not device2, now I will copy the files to a folder on the main device.

The tricky thing about this is I want the directory structure intact. After looking as os.path, and pywin32, I didn’t see anything like ‘mkdir’ where it would make all the folders deep needed to recreate the branch that a file was in. I did however find a function online:

def mkdir(newdir):
    if os.path.isdir(newdir):
        pass
    elif os.path.isfile(newdir):
        raise OSError("a file with the same name as the desired " \
                      "dir, '%s', already exists." % newdir)
    else:
        head, tail = os.path.split(newdir)
        if head and not os.path.isdir(head):
            mkdir(head)
        if tail:
            os.mkdir(newdir)

To copy the files and create the directories, I altered the previous script a bit:

for (path, dirs, files) in os.walk(path2):
	for file in files:
		if os.path.basename(file) not in filenames:
			newPath = os.path.abspath(os.path.join(path,file)).replace(path2,(path1 + 'isItThere//'))
			fileFull = os.path.abspath(os.path.join(path,file))
			print fileFull + " not found in " + path1 + " file cloud"
			print "Copying " + fileFull + " >>> " + newPath
			if os.path.isdir(os.path.dirname(newPath)) == False:
				mkdir(os.path.dirname(newPath))
			win32file.CopyFile (fileFull, newPath, 0))

The results printed should look like below, the files should have been copied accordingly and the directories created.

U:\photos\Crystal\Orlando - Lauras wedding\P0003270.jpg not found in D:\photos\Crystal\ file cloud
Copying U:\photos\Crystal\Orlando - Lauras wedding\P0003270.jpg >>> D:\photos\Crystal\isItThere\Orlando - Lauras wedding\P0003270.jpg
U:\photos\Crystal\Orlando - Lauras wedding\P0003271.jpg not found in D:\photos\Crystal\ file cloud
Copying U:\photos\Crystal\Orlando - Lauras wedding\P0003271.jpg >>> D:\photos\Crystal\isItThere\Orlando - Lauras wedding\P0003271.jpg
U:\photos\Crystal\Orlando - Lauras wedding\P0003272.jpg not found in D:\photos\Crystal\ file cloud

If I had time, or perhaps when I have time, I’ll add MD5 checks.

I am anal-retentive about data retention. There, I said it. There are many times when I find myself in the situation of having two storage devices, that may or may not have duplicate files. I then want to erase one, but do I have all those files backed up?

I use two existing programs to aid me in my anal-retentivity: TerraCopy and WinMerge. Terracopy replaces the windows default copy with something much better (can hash check files when they are copied, etc). With WinMerge, I can right click a folder and say ‘Compare To…’ then right click another and say ‘Compare’. This tells me any differences between the two file/folder trees.

However, here’s an example I have not yet found a good solution for:

I want to erase a camera card I have, I am pretty certain I copied the images off –but how can I be sure! I took those images and sorted them into folders by location or date taken.

So I wrote a small and I am sure inefficient python script to help:

import os
 
filenames = []
 
i = 1
 
path1 = 'D://photos//south america//'
path2 = 'N://DCIM//100ND300//'
 
if os.path.isdir(path1):
	if os.path.isdir(path2):
		print "creating index.."
 
for (path, dirs, files) in os.walk(path1):
	for file in files:
		filenames.append(os.path.basename(file))
 
for (path, dirs, files) in os.walk(path2):
	for file in files:
		if os.path.basename(file) not in filenames:
			print os.path.abspath(os.path.join(path,file)) + ' not found in ' + path1 + ' file cloud'

This will print something like this:

N:/DCIM/100ND300/image.NEF not found in D:/photos/south america/ file cloud

I don’t use python that often at all, please lemme know if there’s a better way to be doing this.

This is a simple proof-of-concept showing how to implement a perforce animation browser via python for MotionBuilder. Clicking an FBX animation syncs it and loads it.

The script can be found here: [p4ui.py], it requires the [wx] and [p4] libraries.

Clicking directories goes down into them, clicking fbx files syncs them and loads them in MotionBuilder. This is just a test, the ‘[..]’ doesn’t even go up directories. Opening an animation does not check it out, there is good documentation for the p4 python lib, you can start there; it’s pretty straight forward and easy: sure beats screen scraping p4 terminal stuff.

You will see the following, you should replace this with the p4 location of your animations, this will act as the starting directory.

	path1 = 'PUT YOUR PERFORCE ANIMATION PATH HERE (EXAMPLE: //DEPOT/ANIMATION)'
	info = p4i.run("info")
	print info[0]['clientRoot']

That should about do it, there are plenty of P4 tutorials out there, my code is pretty straight forward. The only problem was where I instanced it, be sure to instance it with something other than ‘p4’, I did this and it did not work, using ‘p4i’ it did without incident:

p4i = P4.P4()
p4i.connect()

Facial motion capture stabilization is basically where you isolate the movement of the face from the movement of the head. This sounds pretty simple, but it is actually a really difficult problem. In this post I will talk about the general process and give you an example facial stabilization python script.

Disclaimer: The script I have written here is loosely adapted from a MEL script in the book Mocap for Artists, and not something proprietary to Crytek. This is a great book for people of all experience levels, and has a chapter dedicated to facial mocap. Lastly, this script is not padded out or optimized.

To follow this you will need some facial mocap data, there is some freely downloadable here at www.mocap.lt. Grab the FBX file.

Stabilization markers

Get at least 3 markers on the actor that do not move when they move their face. These are called ’stabilization markers’ (STAB markers). You will use these markers to create a coordinate space for the head, so it is important that they not move. STAB markers are commonly found on the left and right temple, and nose bridge. Using a headband and creating virtual markers from multiple solid left/right markers works even better. Headbands move, it’s good to keep this in mind, above you see a special headrig used on Kong to create stable markers.

It is a good idea to write some tools to help you out here. At work I have written tools to parse a performance and tell me the most stable markers at any given time, if you have this data, you can also blend between them.

Load up the facial mocap file you have downloaded, it should look something like this:

In the data we have, you can delete the root, the headband markers, as well as 1-RTMPL, 1-LTMPL, and 1-MNOSE could all be considered STAB markers.

General Pipeline

As you can see, mocap data is just a bunch of translating points. So what we want to do is create a new coordinate system that has the motion of the head, and then use this to isolate the facial movement.

This will take some processing, and also an interactive user interface. You may have seen my tutorial on Creating Interactive MotionBuilder User Interface Tools. You should familiarize yourself with that because this will build on it. Below is the basic idea:

You create a library ‘myLib’ that you load into motionbuilder’s python environment. This is what does the heavy lifting, I say this because you don’t want to do things like send the position of every marker, every frame to your external app via telnet. I also load pyEuclid, a great vector library, because I didn’t feel like writing my own vector class. (MBuilder has no vector class)

Creating ‘myLib’

So we will now create our own library that sits inside MBuilder, this will essentially be a ‘toolkit’ that we communicate with from the outside. Your ‘myLib’ can be called anything, but this should be the place you store functions that do the real processing jobs, you will feed into to them from the outside UI later. The first thing you will need inside the MB python environment is something to cast FBVector3D types into pyEuclid. This is fairly simple:

#casts point3 strings to pyEuclid vectors
def vec3(point3):
	return Vector3(point3[0], point3[1], point3[2])
 
#casts a pyEuclid vector to FBVector3d
def fbv(point3):
	return FBVector3d(point3.x, point3.y, point3.z)

Next is something that will return an FBModelList of models from an array of names, this is important later when we want to feed in model lists from our external app:

#returns an array of models when given an array of model names
#useful with external apps/telnetlib ui
def modelsFromStrings(modelNames):
	output = []
	for name in modelNames:
		output.append(FBFindModelByName(name))
	return output

Now, if you were to take these snippets and save them as a file called myLib.py in your MBuilder directory tree (MotionBuilder75 Ext2\bin\x64\python\lib), you can load them into the MBuilder environment. (You should have also placed pyEuclid here)

In the image above, I get the position of a model in MBuilder, it returns as a FBVector3D, I then import myLib and pyEuclid and use our function above to ‘cast’ the FBVector3d to a pyEuclid vector. It can now be added, subtracted, multiplied, and more; all things that are not possible with the default MBuilder python tools. Our other function ‘fbv()‘ casts pyEuclid vectors back to FBVector3d, so that MBuilder can read them.

So we can now do vector math in motionbuilder! Next we will add some code to our ‘myLib’ that stabilizes the face.

Adding Stabilization-Specific Code to ‘myLib’

One thing we will need to do a lot is generate ‘virtual markers’ from the existing markers. To do this, we need a function that returns the average position of however many vectors (marker positions) it is fed.

#returns average position of an FBModelList as FBVector3d
def avgPos(models):
	mLen = len(models)
	if mLen == 1:
		return models[0].Translation
	total = vec3(models[0].Translation)
	for i in range (1, mLen):
		total += vec3(models[i].Translation)
	avgTranslation = total/mLen
	return fbv(avgTranslation)

Here is an example of avgPos() in use:

Now onto the stabilization code:

#stabilizes face markers, input 4 FBModelList arrays, leaveOrig  for leaving original markers
def stab(right,left,center,markers,leaveOrig):
 
	pMatrix = FBMatrix()
	lSystem=FBSystem()
	lScene = lSystem.Scene
	newMarkers = []
 
	def faceOrient():
		lScene.Evaluate()
 
		Rpos = vec3(avgPos(right))
		Lpos = vec3(avgPos(left))
		Cpos = vec3(avgPos(center))
 
		#build the coordinate system of the head
		faceAttach.GetMatrix(pMatrix)
		xVec = (Cpos - Rpos)
		xVec = xVec.normalize()
		zVec = ((Cpos - vec3(faceAttach.Translation)).normalize()).cross(xVec)
		zVec = zVec.normalize()
		yVec = xVec.cross(zVec)
		yVec = yVec.normalize()
		facePos = (Rpos + Lpos)/2
 
		pMatrix[0] = xVec.x
		pMatrix[1] = xVec.y
		pMatrix[2] = xVec.z
 
		pMatrix[4] = yVec.x
		pMatrix[5] = yVec.y
		pMatrix[6] = yVec.z
 
		pMatrix[8] = zVec.x
		pMatrix[9] = zVec.y
		pMatrix[10] = zVec.z
 
		pMatrix[12] = facePos.x
		pMatrix[13] = facePos.y
		pMatrix[14] = facePos.z
 
		faceAttach.SetMatrix(pMatrix,FBModelTransformationMatrix.kModelTransformation,True)
		lScene.Evaluate()
 
	#keys the translation and rotation of an animNodeList
	def keyTransRot(animNodeList):
		for lNode in animNodeList:
			if (lNode.Name == 'Lcl Translation'):
				lNode.KeyCandidate()
			if (lNode.Name == 'Lcl Rotation'):
				lNode.KeyCandidate()
 
	Rpos = vec3(avgPos(right))
	Lpos = vec3(avgPos(left))
	Cpos = vec3(avgPos(center))
 
	#create a null that will visualize the head coordsys, then position and orient it
	faceAttach = FBModelNull("faceAttach")
	faceAttach.Show = True
	faceAttach.Translation = fbv((Rpos + Lpos)/2)
	faceOrient()
 
	#create new set of stabilized nulls, non-destructive, this should be tied to 'leaveOrig' later
	for obj in markers:
		new = FBModelNull(obj.Name + '_stab')
		newTran = vec3(obj.Translation)
		new.Translation = fbv(newTran)
		new.Show = True
		new.Size = 20
		new.Parent = faceAttach
		newMarkers.append(new)
 
	lPlayerControl = FBPlayerControl()
	lPlayerControl.GotoStart()
	FStart = int(lPlayerControl.ZoomWindowStart.GetFrame(True))
	FStop = int(lPlayerControl.ZoomWindowStop.GetFrame(True))
 
	animNodes = faceAttach.AnimationNode.Nodes
 
	for frame in range(FStart,FStop):
 
		#build proper head coordsys
		faceOrient()
 
		#update stabilized markers and key them
		for m in range (0,len(newMarkers)):
			markerAnimNodes = newMarkers[m].AnimationNode.Nodes
			newMarkers[m].SetVector(markers[m].Translation.Data)
			lScene.Evaluate()
			keyTransRot(markerAnimNodes)
 
		keyTransRot(animNodes)
 
		lPlayerControl.StepForward()

We feed our ‘stab‘ function FBModelLists of right, left, and center stabilization markers, it creates virtual markers from these groups. Then ‘markers’ is all the markers to be stabilized. ‘leavrOrig’ is an option I usually add, this allows for non-destructive use, I have just made the fn leave original in this example, as I favor this, so this option does nothing, but you could add it. With the original markers left, you can immediately see if there was an error in your script. (new motion should match orig)

Creating an External UI that Uses ‘myLib’

Earlier I mentioned Creating Interactive MotionBuilder User Interface Tools, where I explain how to screenscrape/use the telnet Python Remote Server to create an interactive external UI that floats as a window in MotionBuilder itself. I also use the libraries mentioned in the above article.

The code for the facial stabilization UI I have created is here: [stab_ui.py]

I will now step through code snippets pertaining to our facial STAB tool:

def getSelection():
	selectedItems = []
	mbPipe("selectedModels = FBModelList()")
	mbPipe("FBGetSelectedModels(selectedModels,None,True)")
	for item in (mbPipe("for item in selectedModels: print item.Name")):
		selectedItems.append(item)
	return selectedItems

This returns a list of strings that are the currently selected models in MBuilder. This is the main thing that our external UI does. The person needs to interactively choose the right, left, and center markers, then all the markers that will be stabilized.

At the left here you see what the UI looks like. To add some feedback to the buttons, you can make them change to reflect that the user has selected markers. We do so by changing the button text.

Example:

def rStabClick(self,event):
	self.rStabMarkers = getSelection()
	print str(self.rStabMarkers)
	self.rStab.Label = (str(len(self.rStabMarkers)) + " Right Markers")

This also stores all the markers the user has chosen into the variable ‘rStabMarkers‘. Once we have all the markers the user has chosen, we need to send them to ‘myLib‘ in MBuilder so that it can run our ‘stab‘ function on them. This will happen when they click ‘Stabilize Markerset‘.

def stabilizeClick(self,event):
	mbPipe('from euclid import *')
	mbPipe('from myLib import *')
	mbPipe('rStab = modelsFromStrings(' + str(self.rStabMarkers) + ')')
	mbPipe('lStab = modelsFromStrings(' + str(self.lStabMarkers) + ')')
	mbPipe('cStab = modelsFromStrings(' + str(self.cStabMarkers) + ')')
	mbPipe('markerset = modelsFromStrings(' + str(self.mSetMarkers) + ')')
	mbPipe('stab(rStab,lStab,cStab,markerset,False)')

Above we now use ‘modelsFromStrings‘ to feed ‘myLib’ the names of selected models. When you run this on thousands of frames, it will actually hang for up to a minute or two while it does all the processing. I discuss optimizations below. Here is a video of what you should have when stabilization is complete:

Conclusion: Debugging/Optimization

Remember: Your stabilization will only be as good as your STAB markers. It really pays off to create tools to check marker stability.

Sometimes the terminal/screen scraping runs into issues. The mbPipe function can be padded out a lot and made more robust, this here was just an example. If you look at the external python console, you can see exactly what mbPipe is sending to MBuilder, and what it is receiving back through the terminal:

Sending>>> selectedModels = FBModelList()
Sending>>> FBGetSelectedModels(selectedModels,None,True)
Sending>>> for item in selectedModels: print item.Name
['Subject 1-RH1', 'Subject 1-RTMPL']

All of the above can be padded out and optimized. For instance, you could try to do everything without a single lPlayerControl.StepForward() or lScene.Evaluate(), but this takes a lot of MotionBuilder/programming knowhow; it involves only using the keyframe data to generate your matrices, positions etc, and never querying a model.

I was talking to my friend Marco the other day.  As he is a real programmer, he is somewhat equipped with the needed skills required to decode MotionBuilder’s procedurally-generated Python documentation.  We were both frustrated, fighting with the ‘Python Console Tool’, when I showed him the telnet interface he was like “why don’t you just use that?”

And this is what I started doing. I now do much of my tests and work in the telnet console, because, unlike the built in console that Motion Builder offers, the telnet window at least offers copy/paste, and you can press the up arrow to cycle through previous arguments that you have entered. I would suggest using this until Autodesk adds usable features to their ‘Python Console Tool’.

Here’s an example:

This is another post in my short quest to profile per-frame functions in Motion Builder. This script below uses hotshot, the built in python profiler, to profile not only a function I write, but built in MBuilder c++ modules (FBPlayerControl().StepForward()).

8
9
10

#wrap motionbuilder c module functions you want to profile in python functions
def stepForward():
	lPlayerControl.StepForward()

Above is a snippet of the script below. We are wrapping the StepForward() MBuilder call in a function called stepForward(). Hotshot can profile any python function, so we trick it by wrapping a function around the c module.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38

from pyfbsdk import *
import hotshot
import hotshot.stats
import sys
 
lPlayerControl = FBPlayerControl()
 
#wrap motionbuilder c module functions you want to profile in python functions
def stepForward():
	lPlayerControl.StepForward()
 
#this is our function that we want to profile
def myFn():
	FStart = int(lPlayerControl.ZoomWindowStart.GetFrame(True))
	FStop = int(lPlayerControl.ZoomWindowStop.GetFrame(True))
 
	FBPlayerControl().GotoStart()
	for frame in range(FStart,FStop):
		stepForward()
 
#here we set up the hotshot profiler to create a profile file (binary)
#by running our function (myFn)
prof = hotshot.Profile("c:\\myFn.prof")
prof.runcall(myFn)
prof.close()
 
#now we load the profile stats
stats = hotshot.stats.load("c:\\myFn.prof")
stats.strip_dirs()
stats.sort_stats('time', 'calls')
 
#and finally, we print the profile stats to the disk in a file 'myFn.log'
saveout = sys.stdout
fsock = open('c:\\myFn.log', 'w')
sys.stdout = fsock
stats.print_stats(20)
sys.stdout = saveout
fsock.close()

This is what the file we generated (myFn.log) looks like:

         1305 function calls in 15411.020 CPU seconds
 
   Ordered by: internal time, call count
 
   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
     1304 15398.358   11.809 15398.358   11.809 speedtest.py:9(stepForward)
        1   12.662   12.662 15411.020 15411.020 speedtest.py:13(myFn)
        0    0.000             0.000          profile:0(profiler)

As you can see, FBPlayerControl().StepForward() is very resource intensive, and should rarely be used when crunching a lot of keyframe data. I hope this helps anyone in finding resource bottlenecks in large scripts, here’s an example of output from a larger script where I am trying to see how long scene.evaluate() and stepforward() are taking, and get rid of them by querying the keyframe data the way KxL showed before:

         58745 function calls in 26273.622 CPU seconds
 
   Ordered by: internal time, call count
 
   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
     1304 14243.406   10.923 14243.406   10.923 stabtest.py:57(stepForward)
     5217 8448.493    1.619 8448.493    1.619 stabtest.py:52(evalScene)
     5216 1347.949    0.258 1347.949    0.258 stabtest.py:99(keyTransRot)
     5223 1028.827    0.197 1028.827    0.197 stabtest.py:23(getPosition)
        1  672.045  672.045 26273.394 26273.394 stabtest.py:62(stab)
     1305  268.026    0.205 3963.387    3.037 stabtest.py:67(faceOrient)
     5226   53.200    0.010   68.283    0.013 stabtest.py:16(vec3)
     5220   40.443    0.008   40.443    0.008 euclid.py:485(__abs__)
     2610   31.510    0.012   36.667    0.014 euclid.py:384(__sub__)
     5220   30.552    0.006   70.996    0.014 euclid.py:497(normalize)
    13058   30.549    0.002   30.549    0.002 euclid.py:285(__init__)
     2610   27.759    0.011   32.958    0.013 euclid.py:519(cross)
     3918   22.071    0.006  796.673    0.203 stabtest.py:34(avgPos)
     1306   16.727    0.013   19.413    0.015 euclid.py:439(__div__)
     1306   11.738    0.009   14.161    0.011 euclid.py:354(__add__)
        1    0.228    0.228 26273.622 26273.622 stabtest.py:152(stabCall)
        4    0.100    0.025    0.100    0.025 stabtest.py:20(fbv)
        0    0.000             0.000          profile:0(profiler)

As you can see above, most of the time is spent on scene evaluation and stepforward.