Stumbling Toward 'Awesomeness'

I have been working on Ryse for almost two years now, it’s one of the most amazing projects I have had the chance to work on. The team we have assembled is just amazing, and it’s great to be in the position to show people what games can look like on next-gen hardware..  Autodesk asked us to come out to Anaheim and talk about some of the pipeline work we have been doing, and it’s great to finally be able to share some of the this stuff.

A lot of people have been asking about the fidelity, like ‘where are all those polygons?’, if you look at the video, you will see that the regular Romans, they actually have leather ties modeled that deform with the movement of the plates, and something that might never be noticed: deforming leather straps underneath the plates modeled/rigged holding together every piece of Lorica Segmata armor, and underneath that: a red tunic! Ryse is a labor of love!

We’re all working pretty hard right now, but it’s the kind of ‘pixel fucking’ that makes great art -we’re really polishing, and having a blast. We hope the characters and world we have created knock your socks off in November.

I have written many tools at different companies, I taught myself, and don’t have a CS degree. I enjoy shot-sculpting, skinning, and have been known to tweak parameters of on-screen visuals for hours; I don’t consider myself a ‘coder’; still can’t allocate my own memory.  I feel I haven’t really used OOP from an architecture standpoint. So I bought an OOP book, and set out on a journey of self improvement.

‘OOP’ In Maya

In Maya, you often use DG nodes as ‘objects’. At Crytek we have our own modular nodes that create meta-frameworks encapsulating the character pipeline at multiple levels (characters, characterParts, and rigParts). Without knowing it, we were using Object Oriented Analysis when designing our frameworks, and even had some charts that look quite a bit like UML. DG node networks are connected often with message nodes, this is akin to a pointer to the object in memory, whereas with a python ‘object’ I felt it could always easily lose it’s mapping to the scene.

It is possible now with the OpenMaya C++ API to store a pointer to the DG node in memory and just request the full dag path any time you want it, also PyMel objects are Python classes and link to the DG node even when the string name changes.

“John is 47 Years Old and 6 Feet Tall”

Classes always seemed great for times when I had a bunch of data objects, the classic uses are books in a library, or customers: John is 47 years old and likes the color purple. Awesome. However, in Maya, all our data is in nodes already, and those nodes have attributes, those attributes serialize into a Maya file when I save: so I never really felt the need to use classes.

Although, all this ‘getting’, ‘setting’ and ‘listing’ really grows tiresome, even when you have custom methods to do it fairly easily.

It was difficult to find any really useful examples of OOP with classes in Maya. Most of our code is for ‘constructing’: building a rig, building a window, etc. Code that runs in a linear fashion and does ‘stuff’. There’s no huge architecture, the architecture is Maya itself.

Class Warfare

I wanted to package my information in classes and pass that back and forth in a more elegant way –at all times, not just while constructing things. So for classes to be useful to me, I needed them to synchronously exist with DG nodes.

I also didn’t want to have to get and set the information when syncing the classes with DG nodes, that kind of defeats the purpose of Python classes IMO.

Any time I opened a tool I would ‘wrap’ DG nodes in classes that harnessed the power of Python and OOP. To do this meant diving into more of the deep end, but since that was what was useful to me, that’s what I want to talk about here.

To demonstrate, let’s construct this example:

#the setup
loc = cmds.spaceLocator()
cons = [cmds.circle()[0], cmds.circle()[0]]
meshes = [cmds.sphere()[0], cmds.sphere()[0], cmds.sphere()[0]]
cmds.addAttr(loc, sn='controllers', at='message')
cmds.addAttr(cons, sn='rigging', at='message')
for con in cons: cmds.connectAttr(loc[0] + '.controllers', con + '.rigging')
cmds.addAttr(loc, sn='rendermeshes', at='message')
cmds.addAttr(meshes, sn='rendermesh', at='message')
for mesh in meshes: cmds.connectAttr(loc[0] + '.rendermeshes', mesh + '.rendermesh')

So now we have this little node network:

Now if I wanted to wrap this network in a class. We are going to use @property to give us the functionality of an attribute, but really a method that runs to return us a value (from the DG node) when the ‘attribute’ is queried. I believe using properties is key to harnessing the power of classes in Maya.

class GameThing(object):
	def __init__(self, node):
		self.node = node
 
	#controllers
	@property
	def controllers(self):
		return cmds.listConnections(self.node + ".controllers")

So now we can query the ‘controllers’ attribute/property, and it returns our controllers:

test = GameThing(loc)
print test.controllers
##>>[u'nurbsCircle2', u'nurbsCircle1']

Next up, we add a setter, which runs code when you set a property ‘attribute’:

class GameThing(object):
	def __init__(self, node):
		self.node = node
 
	#controllers
	@property
	def controllers(self):
		return cmds.listConnections(self.node + ".controllers")
 
	@controllers.setter
	def controllers(self, cons):
		#disconnect existing controller connections
		for con in cmds.listConnections(self.node + '.controllers'):
			cmds.disconnectAttr(self.node + '.controllers', con + '.rigging')
 
		for con in cons:
			if cmds.objExists(con):
				if not cmds.attributeQuery('rigging', n=con, ex=1):
					cmds.addAttr(con, longName='rigging', attributeType='message', s=1)
				cmds.connectAttr((self.node + '.controllers'), (con + '.rigging'), f=1)
			else:
				cmds.error(con + ' does not exist!')

So now when we set the ‘controllers’ attribute/property, it runs a method that blows away all current message connections and adds new ones connecting your cons:

test = GameThing(loc)
print test.controllers
##>>[u'nurbsCircle2', u'nurbsCircle1']
test.controllers = [cmds.nurbsSquare()[0]]
print test.controllers
##>>[u'nurbsSquare1']

To me, something like properties makes classes infinitely more useful in Maya. For a short time we tried to engineer a DG node at Crytek that when an attr changed, could eval a string with a similar name on the node. This is essentially what a property can do, and it’s pretty powerful. Take a moment to look through code of some of the real ‘heavy lifters’ in the field, like zooToolBox, and you’ll see @property all over the place.

I hope you found this as useful as I would have.

‘Blind data’ is custom data that you can store on any object or its components (vertex, edge, polygon, etc). The documentation says ‘Blind data is information stored with polygons which is not used by Maya in any way..’ I believe it is used when importing meshes from other apps that have properties that do not map to Maya, so that when you take them back to those apps, those properties remain.

Anyway, the important point here is that blind data is metadata (int, float, double, boolean, string, binary) that you can attach to any component. It matters not what happens to said component, you can extract a polygon from a mesh, it’s index will have changed, it’s object will have changed, but its blind data will remain with it. The only drawback can be that it can be painfully slow to write this data, but we will get to that later.

Simple Example

First let’s create a blind data template, this is required to store the data later. We use the command ‘blindDataType’ to create a template for a string type called ‘skinningInfo’ or ‘skin’ for short, giving it an ID 12344. Then we query the ID and it returns the blind data attribs we have created.

cmds.blindDataType(id=12344, dt='string', ldn='skinningInfo', sdn='skin')
print cmds.blindDataType(id=12344, tn=1, q=1)
>>['skinningInfo', 'skin', 'string']

So now we have our template, let’s try using it, this is more focused on getting the idea across than speed:

#query vertex # of mesh
v = cmds.polyEvaluate(node, v=1)
#loop through vertices
for vtx in range(0, v):
    #get influences
    infs = cmds.skinCluster(sc, inf=1, q=1)
    #get weights
    objVtx = node + ".vtx[" + str(vtx) + "]"
    vals = cmds.skinPercent(sc, objVtx, q=1, v=1)
    #build dict of influence:weight
    for i in range(0, len(infs)):
        weightDict[infs[i]] = vals[i]
    #write value to blind data
 cmds.polyBlindData(objVtx, id=12344, at='vertex', ldn='skinningInfo', sd=str(weightDict))

So here you have saved a dictionary per vertex that has key/value pairs of influence/weight. You can query like so:

#I have a vertex selected in component mode
print cmds.polyQueryBlindData(cmds.ls(sl=1), id=12344, showComp=1)
['polySurface2.vtx[64].skin', "{u'joint2': 0.49755714634259796, 'joint3': 0.49755714634259784, 'joint1': 0.0048857073148042395}"]

Now On To Something More Useful

So let’s create a function to store skinning data per-vertex, as you may have seen with the above, that was painfully slow. If you have written any skinning tools, you know that the solution to this (other than learning C++) is to apply your change to all vertices at once. Below we build two lists, one of vertices and one of weights, then we

def storeBlindSkinning(mesh, sc):
	'''
	mesh is a skinned mesh, and sc is the skincluster affecting the mesh
	'''
	vtxList = []
	v = cmds.polyEvaluate(mesh, v=1)
	infs = cmds.skinCluster(sc, inf=1, q=1)
	vtxWeights = []
	for vtx in range(0, v):
	    objVtx = mesh + ".vtx[" + str(vtx) + "]"
	    vtxList.append(objVtx)
	    vals = cmds.skinPercent(sc, objVtx, q=1, v=1)
	    for i in range(0, len(infs)):
        	weightDict[infs[i]] = vals[i]
            vtxWeights.append(str(weightDict))
        cmds.polyBlindData(vtxList, id=12344, at='vertex', ldn='skinningInfo', sd=vtxWeights)

Setting all the data at once is 1/3 faster, however, setting this data takes quite some time, and you may want to take a hit for a progress bar. (break it up into groups) On ~60,000 vertices this took 10min (15min doing it inside the loop). I don’t mind that hit if it means that I can now detach/alter/slice my mesh without losing skinning data. You can even extract faces and the new vertices created will get the same blind data as their original. (one becomes two)

As always, the C++ API is much faster, my colleague, Bogdan speed tested the above function and 50,000 vertices took only a few milliseconds, compared to 10 minutes in pythonland.

Remember, there are other ways to store skinning data, using UVs, position, vertex color channels, etc. I just wanted to introduce people to blind data in Maya and show a potential use.

CRYENGINE CINEBOX

I am giving a talk at SIGGRAPH 2012 entitled ‘Film/Game Convergence: What’s Taking So Long?‘ where I discuss the inherent differences between games and film and go over a few case studies of projects that attempted to use a game engine for film previs. I also talk a bit about the development of our CINEBOX application, the decisions we had to make, and how we dealt with many of the issues previous attempts have run into.

STUDIO WORKSHOPS

I will be giving two more Studio Workshops this year, the first is a followup to last year’s Introduction to Python, entitled ‘Python Scripting in Maya‘. The other workshop is ‘Building a Game Level‘, which is the same basic workshop I gave last year where I show people how to make a playable game level in CryEngine in an hour. Studio Workshops are hands-on sessions where each attendee has a computer and follows along with the instructor. It’s a great chance for people of all ages to learn new things.

I am still finding my feet in Maya, on my project, some files have grown to 800mb in size. Things get corrupt, hand editing MAs is common; I am really learning some of the internals.

In the past week I have had to do a lot of timeline walking to switch coord spaces and get baked animations into and out of hierarchies. In 3dsMax you can do a loop and evaluate a node ‘at time i’, and there is no redraw or anything. I didn’t know how to do this in Maya.

I previously did this with looping cmds.currentTime(i) and ‘walking the timeline’, however, you can set the time node directly like so: cmds.setAttr(“time1.outTime”, int(i))

Unparenting a child with keyed compensation (1200 frames)
10.0299999714 sec – currentTime
2.02 sec – setAttr

There are some caveats, whereas in a currentTime loop you can just cmds.setKeyframe(node), I now have to cmds.setKeyframe(node, time=i). But when grabbing a matrix, I don’t need to pass time and it works, I don’t think you can anyway.. I guess it gets time from the time node.

Here’s a sample loop that makes a locator and copies a nodes animation to world space:

#function feeds in start, end, node
	if not start: start = cmds.playbackOptions(minTime=1, q=1)
	if not end: end = cmds.playbackOptions(maxTime=1, q=1)
	loc = cmds.spaceLocator(name='parentAlignHelper')
	for i in range(start, (end+1)):
		cmds.setAttr("time1.outTime", int(i))
		matrix = cmds.xform(node, q=1, ws=1, m=1)
		cmds.xform(loc, ws=1, m=matrix)
		cmds.setKeyframe(loc, time=i)

As many of you know, I feel the whole ‘autorigging’ schtick is a bit overrated. Though Bungie gave a great talk at GDC09 (Modular Procedural Rigging), Dice was to give one this year at SIGGRAPH (Modular Rigging in Battlefield 3), but never showed up for the talk.

At Crytek we are switching our animation dept from 3dsMax to Maya. This forces us to build a pipeline there from scratch; in 3dsMax we had 7 years of script development focused on animation and rigging tools. So I am looking at quite a bit o Maya work. The past two weeks focusing on a ‘rigging system’ that I guess could be thought of as ‘procedural’ but is not really an ‘autorigger’. My past experience was always regenerating rigs with mel cmds.

Things I would like to solve:

• Use one set of animator tools for many rigs – common interfaces, rig block encapsulation (oh god i said ‘block’)
• Abstract things away, thinking of rigging ‘units’ and character ‘parts’ instead of individual rig elements, break reliance on naming, version out different parts
• Be fluid enough to regenerate the ‘rigging’ at any time

First Weekend: Skeleton ‘Tagging’

I created a wrapper around the common rigging tools that I used, this way, when I rigged, it would automagically markup the skeleton/elements as I went. This looked like so:

The foundation of this was marking up the skeleton or cons with message nodes that pointed to things or held metadata.  This was cool, and I still like how simple it was, however, it didn’t really create the layer of abstraction I was after. There wasn’t the idea of a limb that I could tell to switch from FK to IK.

Second Weekend: Custom Nodes

That Bungie talk got a lot of us all excited, Roman went and created a really cool custom node plugin that does way more than we spec’d it out to do. I rewrote the rigging tools to create ‘rigPart’ nodes, which could be like an IK chain, set of twist joints, expression, or constraint. These together could form a ‘charPart’ like an arm or leg. All these nodes sat under a main ‘character’ node. I realize that many companies abstract their characters into ‘blocks’ or ‘parts’, but I had never seen a system that had another layer underneath that. Roman also whipped up a way that when an attr on a customNode changes, you could evaluate a script. So whether it’s a human arm or alien tentacle arm, the ‘charPart’ node can have one FK/IK enum. I am still not sure if this is a better idea, because of the sheer legwork involved..

Third Weekend: A Mix of Both?

So a class like ‘charParts.gruntLeg()’ not only knew how to build the leg rigParts, but also only the leg ‘rigging’ if needed. This works pretty well, but the above was pretty hard to read. I took some of my favorite things about the tree-view-based system and I created a ‘character’ outliner of sorts. This made it much easier to visualize the rigParts that made up individual ‘systems’ of the character, like leg, spine, arm, etc. I did it as a test, but in a way that I easily swap it out with the treeWidget in the rigging tools dialog.

So how do you guys solve some of these issues?

My friends Judd and Rich gave a talk on some of the Character Tech behind Uncharted 2. Here are the slides.

Many, many people are having weird, buggy camera issues where you rotate a view and it snaps back to the pre-tumbled state (view does not update properly). There are posts all over, and Autodesk’s official response is “Consumer gaming videocards are not supported”. Really? That’s basically saying: All consumer video cards, gaming or not, are unsupported. I have had this issue on my laptop, which is surely not a ‘gaming’ machine. Autodesk says the ‘fix’ is to upgrade to an expensive pro-level video card. But what they maybe would tell you if they weren’t partnered with nVidia is: It’s an easy fix!

Find your Maya ENV file:

C:\Documents and Settings\Administrator\My Documents\maya\2009-x64\Maya.env

And add this environment variable to it:

MAYA_GEFORCE_SKIP_OVERLAY=1

Autodesk buried this information in their Maya 2009 Late Breaking Release Notes, and it fixes the issue completely! However, even on their official forum, Autodesk employees and moderators reply to these draw errors as follows:

Maya 2009 was tested with a finite number of graphics cards from ATI and Nvidia, with drivers from each vendor that provided the best performance, with the least amount of issues. (at the time of product launch).  A list of officially qualified hardware can be found here: http://www.autodesk.com/maya-hardware. Maya is not qualified/supported on consumer gaming cards.  Geforce card users can expect to have issues.  This is clearly stated in the official qualification charts mentioned above.

I saw this presentation about a year ago, talking about the pipeline Epic uses on their games. Maybe some interesting stuff for others here. The images are larger, you can right click -> view image to see a larger version.

45 days or more to create a single character… wow.

They don’t use polycruncher to generate LODs, they do this by hand. They just use it to import the mesh into max in a usable form from mudbox/zbrush.

They don’t care so much about intersecting meshes when making the high res, as it’s just used to derive the nMap, not RP a statue or anything.

They said they only use DeepUV for it’s ‘relax’ feature. They make extensive use of the 3DS Max ‘render to texture’ as well.

Their UT07 characters are highly customizable. Individual armor parts can be added or removed, or even modded. Their UV maps are broken down into set sections that can be generated on the fly. So there are still 2×2048 maps but all the maps can be very different. This is something I have also seen in WoW and other games.

They mentioned many times how they use COLLADA heavily to go between DCC apps.

They share a lot of common components accross characters

Here are some screens of animation rigs from Kung Fu Panda:

In a shot: