Stumbling Toward 'Awesomeness'

One aspect that most people immediately want to do, when they hear that UE4 has python exposure, is import assets. This isn’t the most straight forward, so I will walk you through a character import. Yes, all the code below imports a single skelmesh/skeleton.

#first we need to know where we're importing the character into, let's make a new path
asset_path = '/Game/Characters/MyTest'
unreal.EditorAssetLibrary.make_directory(asset_path)
 
#now we make an import task and access it's options, which are similar to the import UI
task = unreal.AssetImportTask()
 
#now let's set some import options on the task class
task.filename = PATH_TO_FBX_FILE
task.destination_path = asset_path
task.destination_name = 'my_asset'
task.replace_existing = True
task.automated = True
#save the file when it is imported, that's right!
task.save = True

Now let’s instance an ‘options’ class on the task class and set some skeletal mesh specific options

task.options = unreal.FbxImportUI()
 
task.options.import_as_skeletal = True
task.options.override_full_name = True
task.options.mesh_type_to_import = unreal.FBXImportType.FBXIT_SKELETAL_MESH
 
task.options.skeletal_mesh_import_data.set_editor_property('update_skeleton_reference_pose', False)
task.options.skeletal_mesh_import_data.set_editor_property('use_t0_as_ref_pose', True)
task.options.skeletal_mesh_import_data.set_editor_property('preserve_smoothing_groups', 1)
task.options.skeletal_mesh_import_data.set_editor_property('import_meshes_in_bone_hierarchy', False)
task.options.skeletal_mesh_import_data.set_editor_property('import_morph_targets', True)
task.options.skeletal_mesh_import_data.set_editor_property('threshold_position',  0.00002)
task.options.skeletal_mesh_import_data.set_editor_property('threshold_tangent_normal', 0.00002)
task.options.skeletal_mesh_import_data.set_editor_property('threshold_uv', 0.000977)
task.options.create_physics_asset = False
task.options.import_animations = False
task.options.skeletal_mesh_import_data.set_editor_property('convert_scene', True)
task.options.skeletal_mesh_import_data.set_editor_property('force_front_x_axis', False)
task.options.skeletal_mesh_import_data.set_editor_property('convert_scene_unit', False)

Now let’s set the normal/tangent import method, and the normal generation method. This a bit odd, you need to set it as an existing python object, let’s set it to normals and tangents, but there are also (unreal.FBXNormalImportMethod.FBXNIM_IMPORT_NORMALS, unreal.FBXNormalImportMethod.FBXNIM_COMPUTE_NORMALS)

normal_import_method = unreal.FBXNormalImportMethod.FBXNIM_IMPORT_NORMALS_AND_TANGENTS
normal_generation_method = unreal.FBXNormalGenerationMethod.MIKK_T_SPACE
 
task.options.skeletal_mesh_import_data.set_editor_property('normal_generation_method', normal_generation_method)
task.options.skeletal_mesh_import_data.set_editor_property('normal_import_method', normal_import_method)

The task class and its options look to be prepared, let’s now do the deed!

imported_asset = unreal.AssetToolsHelpers.get_asset_tools().import_asset_tasks([task])
#let's check what files were imported/created:
imported_skelmesh = task.imported_object_paths

One thing that you will want to do right away is iterate through a bank of existing assets or find assets in a build. In UE4, your main window into the ‘content browser’ is the ‘asset registry’. You can use it to find all kinds of assets, iterate through assets in a folder, etc.

Let’s go ahead and instance it to take a look, now would be a good time to open the unreal.AssetRegistryHelpers UE4 Python API docs in another tab! Also, I am running this in UE4 4.21 release, with the free Paragon asset Marketplace assets.

Walking Assets In A Directory

Let’s ask it for all the assets in a certain path.

asset_reg = unreal.AssetRegistryHelpers.get_asset_registry()
assets = asset_reg.get_assets_by_path('/Game/ParagonMorigesh/Characters/Heroes/Morigesh/Meshes')

The method to get the asset registry has returned an unreal.AssetRegistry class. If you look at this class, you can see some really useful calls, like get_assets_by_path, that I used on the next line.

Let’s take a look at the assets:

for asset in assets: print asset

This yields:

LogPython: <Struct 'AssetData' (0x000001ADF8564560) {object_path: "/Game/ParagonMorigesh/Characters/Heroes/Morigesh/Meshes/Morigesh_Skeleton.Morigesh_Skeleton", package_name: "/Game/ParagonMorigesh/Characters/Heroes/Morigesh/Meshes/Morigesh_Skeleton", package_path: "/Game/ParagonMorigesh/Characters/Heroes/Morigesh/Meshes", asset_name: "Morigesh_Skeleton", as
set_class: "Skeleton"}>
LogPython: <Struct 'AssetData' (0x000001ADF8566A90) {object_path: "/Game/ParagonMorigesh/Characters/Heroes/Morigesh/Meshes/Orion_Proto_Retarget.Orion_Proto_Retarget", package_name: "/Game/ParagonMorigesh/Characters/Heroes/Morigesh/Meshes/Orion_Proto_Retarget", package_path: "/Game/ParagonMorigesh/Characters/Heroes/Morigesh/Meshes", asset_name: "Orion_Proto_R
etarget", asset_class: "Rig"}>
LogPython: <Struct 'AssetData' (0x000001ADF8564560) {object_path: "/Game/ParagonMorigesh/Characters/Heroes/Morigesh/Meshes/Morigesh_Cyl_Shadows.Morigesh_Cyl_Shadows", package_name: "/Game/ParagonMorigesh/Characters/Heroes/Morigesh/Meshes/Morigesh_Cyl_Shadows", package_path: "/Game/ParagonMorigesh/Characters/Heroes/Morigesh/Meshes", asset_name: "Morigesh_Cyl_
Shadows", asset_class: "PhysicsAsset"}>
LogPython: <Struct 'AssetData' (0x000001ADF8566A90) {object_path: "/Game/ParagonMorigesh/Characters/Heroes/Morigesh/Meshes/Morigesh_Physics.Morigesh_Physics", package_name: "/Game/ParagonMorigesh/Characters/Heroes/Morigesh/Meshes/Morigesh_Physics", package_path: "/Game/ParagonMorigesh/Characters/Heroes/Morigesh/Meshes", asset_name: "Morigesh_Physics", asset_
class: "PhysicsAsset"}>
LogPython: <Struct 'AssetData' (0x000001ADF85654B0) {object_path: "/Game/ParagonMorigesh/Characters/Heroes/Morigesh/Meshes/Morigesh.Morigesh", package_name: "/Game/ParagonMorigesh/Characters/Heroes/Morigesh/Meshes/Morigesh", package_path: "/Game/ParagonMorigesh/Characters/Heroes/Morigesh/Meshes", asset_name: "Morigesh", asset_class: "SkeletalMesh"}></code>

It has returned Python Objects of ‘unreal.AssetData‘ type, this class has a lot of things we can query, like class type, name, full path, etc. Let’s print the class name for each:

for asset in assets:
    print asset.class

Let’s only look at skeletal meshes and then let’s do something to them. In order to manipulate them, we need to load them, look at what the get_full_name function returns:

for asset in assets:
    #you could use isinstance unreal.SkeletalMesh, but let's build on what we learned
    if asset.asset_class == 'SkeletalMesh':
        print asset.get_full_name()
#>SkeletalMesh'/Game/ParagonMorigesh/Characters/Heroes/Morigesh/Meshes/Morigesh.Morigesh'

We need to split that output, then load the asset:

for asset in assets:
    if asset.asset_class == 'SkeletalMesh':
        full_name = asset.get_full_name()
        path = full_name.split(' ')[-1]
        skelmesh = unreal.load_asset(path)

Now this returned an unreal.SkeletalMesh class and we can ask it for it’s skeleton:

skeleton = skelmesh.skeleton

Finding Assets

Let’s say someone gives you a list of problematic assets, but they’re not long paths, just asset names! You want to be able to find the long path for all assets in the list so that you can do something with them. The AssetRegistry can help!

Let’s build a dictionary of all asets in the build, the keys will be the short names, and the values will be the long paths:

def get_asset_dict(asset_type=None):
    asset_list = None
    if asset_type:
        asset_list = unreal.AssetRegistryHelpers.get_asset_registry().get_assets_by_class(asset_type)
    else:
        asset_list = unreal.AssetRegistryHelpers.get_asset_registry().get_all_assets()
    asset_dict = {}
    for asset in asset_list:
        asset_name = str(asset.asset_name)
        obj_path = asset.object_path
        if asset_name not in asset_dict:
            asset_dict[asset_name] = [str(obj_path)]
        else:
            asset_dict[asset_name].append(str(obj_path))
 
    return asset_dict

This takes a second or two to build, but you now have an index of all assets by package name, that you can query their full path. It’s a bit faster if you query all assets of a certain type you know you’re looking for. You also will know when there is more than one asset with a name, because it’s list will have multiple entries. (That’s why we store what we find in a list, there could be multiple assets with the same name)

The team has worked really hard to get this building and automagically updated as we expose more useful stuff:

https://api.unrealengine.com/INT/PythonAPI/

Working in cinematics and Sequencer, I often would like a global audio slider to lower audio levels while I am working on a sequence. I thought this might be a good simple tool example. This is completely bare bones, just a single slider in a dialog, for something more complex, you can check out scriptWrangler.

import unreal
from PySide import QtCore, QtGui, QtUiTools
 
class VolumeSlider(QtGui.QDialog):
 
    def __init__(self, parent=None):
        super(VolumeSlider, self).__init__(parent)
        self.setWindowTitle("Volume Slider")
        self.slider = QtGui.QSlider()
        self.slider.sliderMoved.connect(self.slider_fn)
        self.slider.setRange(0,100)
        self.slider.setValue(100)
 
        layout = QtGui.QVBoxLayout()
        layout.addWidget(self.slider)
        self.setLayout(layout)
 
        #load the master SoundClass
        self.master_sound_class = unreal.load_asset(None, name='/Engine/EngineSounds/Master')
 
    def slider_fn(self):
        volume_float = self.slider.value()/100.00
        #set the volume to the desired value
        self.master_sound_class.properties.set_editor_property(name='volume', value=volume_float)
 
APP = None
if not QtGui.QApplication.instance():
    APP = QtGui.QApplication(sys.argv)
 
main_window = VolumeSlider()
main_window.show()

One of the most visited/indexed posts on this site was the brief Maya vector math post, now that we are releasing Python in UE4, I thought I would port those examples to UE4 Python.

Creating Vectors

Let’s query the bounding box extents of a skeletal mesh actor. This will return two points in world space as vectors:

bbox = mesh.get_actor_bounds(False)
v1, v2 = bbox[0], bbox[1]

If we print the first vector, we see it’s a struct of type Vector

print v1
#<Struct 'Vector' (0x000001EACECEFE78) {x: 540.073303, y: 32.021194, z: 124.710869}>;

If you want the vector as a tuple or something to export to elsewhere, you just make a tuple of its components:

print (v1.x, v1.y, v1.z)
#(540.0733032226562, 32.02119445800781, 124.71086883544922)

If you want to create your own vector, you simply do:

my_vec3 = unreal.Vector(1,2,3)
Print my_vec3
#<Struct 'Vector' (0x000001EACECECA40) {x: 1.000000, y: 2.000000, z: 3.000000}>

Length / Distance / Magnitude

The Vector struct supports many mathematical operations, let’s say we want to get the distance from v1 to v2, we would subtract them, which returns a new vector, then we would get the length (‘size’ in this case) of the vector:

new_vec = v2-v1
Print new_vec.size()
#478.868011475

The vector struct has a lot of convenience functions built in (check out the docs here) , for instance, let’s get the distance from v1, to v2 (the diagonal across our bounding box), without doing the above by calling the dist() function:

print v1.dist(v2)
#478.868011475

There is also a distSquared function it you just want to quickly compare which are greater and not calc real distance.

USE CASE: FIND SMALL ACTORS

Using what we got, let’s play in the default scene:

We can iterate through the scene and find actors under a certain size/volume:

for actor in static_meshes:
    mesh_component = actor.get_component_by_class(unreal.StaticMeshComponent)
    v1, v2 = mesh_component.get_local_bounds()
    if v1.dist(v2) < 150:
        print actor.get_name()
#Statue

I am querying the bounds of the static mesh components because the get_bounds() on the actor class returns the unscaled bounds, you may notice that many things in the scene, including the statue and floor are scaled. This is reflected in the get_local_bounds() of the mesh component.

DOT PRODUCT / ANGLE BETWEEN TWO VECTORS

If we wanted to know the angle between two vectors, we would use the dot product of those, let’s create two new vectors v1, and v2, because our previous were not really vectors per se, but point locations Just like in UE4, you use the ‘|’ pipe to do a vector dot product.

v1 = unreal.Vector(0,0,1)
v2 = unreal.Vector(0,1,1)
v1.normalize()
v2.normalize()
print v1 | v2
print v1 * v2

Notice that asterisk will multiply the vectors, pipe will return the sum of the components, or the dot product / scalar product as a float.

For the angle between, let’s import python’s math library:

import math
dot = v1|v2
print math.acos(dot) #returns 0.785398180512
print math.acos(dot) * 180 / math.pi
#returns 45.0000009806

Above I used the Python math library, but there’s also an Unreal math library available unreal.MathLibrary, you should check that out, it has vector math functions that don’t exist in the vector class:

dot = v1|v2
print unreal.MathLibrary.acos(dot) #returns 0.785398180512
print unreal.MathLibrary.acos(dot) * 180 / math.pi #returns 45.0000009806
print unreal.MathLibrary.radians_to_degrees(unreal.MathLibrary.acos(dot)) #returns: 45.0

USE CASE: CHECK ACTOR COLINEARITY

Let’s use the dot product to check if chairs are colinear, or facing the same direction. Let’s dupe one chair and call it ‘Chair_dupe’:

fwd_vectors = {}
for actor in static_meshes:
    if 'Chair' in  actor.get_name():
        actor_vec = actor.get_actor_forward_vector()
        for stored_actor in fwd_vectors:
            if actor_vec | fwd_vectors[stored_actor] == 1.0:
                print actor.get_name(), 'is colinear to', stored_actor
        fwd_vectors[actor.get_name()] = actor_vec
#returns: Chair_dupe is colinear to Chair

I hope this was helpful, I can post some other examples later.

Out of the gate, it’s important to understand that the Python exposure wraps BluePrint, not the C++ SDK, if something isn’t exposed to blueprint, you cannot access it with Python.

A Note on Naming

There are notable differences that should be pointed out regarding the python exposure and the UE4 programming documentation. UE4 class names have dropped the prefix, an FRotator would be unreal.Rotator, FVector, unreal.Vector, etc. Functions on those classes have been converted to PEP8 style names that differ from their C++ counterparts, for example Actor.GetActorBounds is Actor.get_actor_bounds, just FYI.

Detective Work

If you have ever used an initial alpha or beta embedded Python implementation in an application, like Maya or MotionBuilder, (if you’re old like me!) –you know the drill. There’s no documentation yet, but you can use existing blueprint/C++ documentation and native python built-in functions like dir, help, and type to try and navigate.

DIR()
Python’s dir command returns a ‘directory’ of all attributes of an object. This is extremely useful. Let’s query the attributes of a static mesh actor:

print dir(actor)

This returns a massive list, if you print each item, you’ll see something like this:

...
get_remote_role
get_squared_distance_to
get_tickable_when_paused
get_typed_outer
get_velocity
get_vertical_distance_to
get_world
has_authority
hidden
initial_life_span
instigator
is_actor_being_destroyed
...

HELP()
Jamie Dale has put a lot of work into making help work as expected, through dir we saw the actor has a get_velocity property, let’s ask help abut it:

help(actor.get_velocity)

Help returns not only info about get_velocity, but tells us what a function takes and returns:

Help on built-in function get_velocity:
get_velocity(...)
    x.get_velocity() -> Vector -- Returns velocity (in cm/s (Unreal Units/second) of the rootcomponent if it is either using physics or has an associated MovementComponent
    param: return_value (Vector)

TYPE()
Type just returns the type of object, but it’s useful when you’re not sure exactly what something is. Let’s ask what type our actor is:

print type(actor)
#<type 'StaticMeshActor'>

Now let’s ask it about the get_velocity above, we know it’s a function because I just used it, but as an example:

print type(actor.get_velocity)
#<type 'builtin_function_or_method_with_closure'>

With these core concepts, you can really begin stumbling around and making useful scripts and tools!

In Practice

Let’s query all StaticMeshes in a level (the default UE4 map). First thing is to load the level:

import unreal
 
level_path = '/Game/StarterContent/Maps/Minimal_Default'
level = unreal.find_asset(None, name=level_path)

Next we use get_all_actors_of_class to query all the static meshes:

static_meshes = unreal.GameplayStatics.get_all_actors_of_class(level, unreal.StaticMeshActor)
print static_meshes
#returns: [StaticMeshActor'"/Game/StarterContent/Maps/Minimal_Default.Minimal_Default:PersistentLevel.Table"', StaticMeshActor'"/Game/StarterContent/Maps/Minimal_Default.Minimal_Default:PersistentLevel.Statue"', StaticMeshActor'"/Game/StarterContent/Maps/Minimal_Default.Minimal_Default:PersistentLevel.Floor_14"', StaticMeshActor'"/Game/StarterContent/Maps/Minimal_Default.Minimal_Default:PersistentLevel.Floor"', StaticMeshActor'"/Game/StarterContent/Maps/Minimal_Default.Minimal_Default:PersistentLevel.Chair_15"', StaticMeshActor'"/Game/StarterContent/Maps/Minimal_Default.Minimal_Default:PersistentLevel.Chair"']

That’s returned a long list of class objects, let’s make that more readable by calling the get_name function of each class with a simple list comprehensions:

print [mesh.get_name() for mesh in static_meshes]
#returns: ['Table', 'Statue', 'Floor_14', 'Floor', 'Chair_15', 'Chair']

How did I know there was a get_name functions? I saw it when I ran directory on the static mesh actor: dir(actor).

Let’s move the table:

if mesh.get_name() == 'Table':
    xform = actor.get_actor_transform()
    location = actor.get_actor_location()
    print xform
    print location
#returns:
#<Struct 'Transform' (0x0000000095E5D3C0) {rotation: {x: 0.000000, y: 0.000000, z: 0.000000, w: 1.000000}, translation: {x: -180.000000, y: 0.000000, z: 32.000000}, scale3d: {x: 1.000000, y: 1.000000, z: 1.000000}}>
#<Struct 'Vector' (0x00000000B8ACE5C0) {x: -180.000000, y: 0.000000, z: 32.000000}>

Let’s change the location and use set_actor_location to set a new location:

location.z += 28 #location = 60
actor.set_actor_location(location)
#returns:
#Traceback (most recent call last):
#TypeError: Required argument 'sweep' (pos 2) not found

Wow, so that fails, we need to find out why, let’s ask help:

print help(actor.set_actor_location)

Help on built-in function set_actor_location:
 
set_actor_location(...)
    x.set_actor_location(new_location, sweep, teleport) -> HitResult or None -- Move the Actor to the specified location.
    param: new_location (Vector) -- The new location to move the Actor to.
    param: sweep (bool) -- Whether we sweep to the destination location, triggering overlaps along the way and stopping short of the target if blocked by something. Only the root component is swept and checked for blocking collision, child components move without sweeping. If collision is off, this has no effect.
    param: teleport (bool) -- Whether we teleport the physics state (if physics collision is enabled for this object). If true, physics velocity for this object is unchanged (so ragdoll parts are not affected by change in location). If false, physics velocity is updated based on the change in position (affecting ragdoll parts). If CCD is on and not teleporting, this will affect objects along the entire swept volume.
    param: sweep_hit_result (HitResult) -- The hit result from the move if swept.
    param: return_value (bool)
    return: Whether the location was successfully set (if not swept), or whether movement occurred at all (if swept).

Ok, awesome, so I’ll tell it I don’t want it to sweep, and I do want it to teleport:

location.z += 28 #location = 60
actor.set_actor_location(location. False, True)

You should see the table move!

NOTE: Careful, in order for your Python changes to be added to the undo stack, you need to use unreal.ScopedEditorTransaction.

I hope this was helpful, I can post some other examples later.

As a UE4 Python tool example, I wrote a pretty bare bones script editor for UE4 and tossed it up on github. While my goal was making a tool to allow me to play around with UE4 Python, this is actually a lightweight script editor for any python environment that is lacking one.

Epic will make some kind of script editor in the future, but until then, feel free to use (and improve!) this one:
https://github.com/chrisevans3d/scriptWrangler

This requires PySide, you can pip install that, I gave an example in my previous post, here.

To run the script, you simply enter the path to it, example:

ADDING THE PLUGIN

For anyone with version 4.19 of the engine or later, you now have access to Python. It’s marked ‘experimental’, enable it under plugins:

You then can enter python through the command line:

As the Python implementation wraps blueprint and blutilities, you only have access to things exposed through blueprint.

INSTALLING THIRD PARTY LIBRARIES

The team has made it a lot easier to install 3rd party libraries by shipping with pip.exe, it’s in this folder: <build>\Engine\Source\ThirdParty\Python\Win64\Scripts

Here’s an example of installing PySide into the build:

>>pip install --target=Y:\Build\UE_4.19\Engine\Source\ThirdParty\Python\Win64\Lib\site-packages Pyside
Collecting Pyside
  Using cached PySide-1.2.4-cp27-none-win_amd64.whl
Installing collected packages: Pyside
Successfully installed Pyside-1.2.4

Hey guys, I have multiple tools on github and have never converted them to Maya 2017 / Pyside.  I see many of you have forked them to add that support.

What’s the best way to deal with this? Should I just update the tools to only work with PySide2? Is there an option through github to branch my own tool, or let people know that for older Maya support they need to grab up to a certain changelist?

I posted the skeleton of this a while ago in the post “Why does everyone write their own FBX exporter?“, many people asked me for the exporter over the years and I sent it to them. Since that post, uExport has become the way we get all characters into UE4 at Epic. It’s been crazy to see this little fledgling tool, developed largely in spare time to allow us to export non-A.R.T.v1 rigs (like the kite and deer in the first demo I worked on at Epic) become something we rely on so much.

It’s here on GitHub, there’s a lot more info in the readme.

I wanted to touch on something that comes up every once in a while: The PEP8 Style Guide. When you’re first getting into Python, you are messy as hell, and as you gain more experience, you start understanding the need to adhere to basic standards for code style. However, just at that moment, I see people completely miss the point of PEP8, and go down this path of assuming that anyone not adhering to PEP8 is doing it wrong.

That’s not the case, and without really understanding PEP8, you could easily fall into this trap.

PEP8 Is About One Thing

PEP8, as defined by itself is about one thing over all CONSISTENCY. Literally the FIRST THING PEP8 says after the Introduction this:

But people don’t really want to pay attention to that, I mean you’re looking at the PEP8 docs to be told how to write code, not that you might be in a situation where you should ignore their standards. But the most important message they impart is to be CONSISTENT.

Python Application APIs

In the VFX and games industry, most of the Python we write is inside existing applications through exposed C++ APIs that have been wrapped in Python. Often people wonder why these professional applications don’t adhere to the PEP8 Style Guide. The answer is simple, they are wrapping existing C++ functions, for CONSISTENCY they use the same camelCase function name as in C++, not lowercase_underscore following PEP8. But why?

• It allows someone who knows the C++ function to immediately know the Python function
• It allows a user who is accessing a new method added to an existing class to not have to wonder about it’s naming
• In documentation, the often human-written explanations of functionality are often completely usable for Python docs

At Crytek, when we implemented Python exposure in the engine, we had to think about this and I noticed that almost all applications with C++ Python APIs were consistent with the C++ style/naming. Here are some examples:

• Maya: cmds.setAttr()
• 3dsMax: GetLengthSquared()
• CryEngine: Alembic.playSequence()
• PySide: QtGui.QMainWindow.setCentralWidget()
• Modo: scene.ItemLookup()
• Houdini: point.setAttribValue()
• MotionBuilder: actor.setDefinitionScaleVector()
• Substance: sbsexporter.getExportedDependencies()

Are all these developers stupid for ignoring the ‘Jesus book’ of Python Coding Standards? No, they are actually following it! Remember: CONSISTENCY!

Interpretation of The Good Book

Let’s take a look at some times when you should or shouldn’t follow the good book. These are just my take on things, I would love to hear your take in the comments. Also, this post mainly centers on naming standards, it’s usually best to decide as a team which PEP8 rules you want to follow and which don’t make sense.

I am generating a python API that wraps my C++ code

Be consistent with your naming standards that are already set. Even Python itself has multiple libs that don’t follow PEP naming just to be consistent with existing work. This one was covered above, so I’ll move on.

I am writing code that fits within an existing Python API

Be consistent with that API. If you’re in Maya or using PySide, you’re forced to use their function names. Please don’t have half your code with their function names and then half the code with your own PEP8 function names, this is really inconsistent. Not to mention when you override or reimpelent existing functions,  you’ll find yourself typing something like “on_close”. Another annoyance will be style enforcement in IDEs, if you use PEP8 you are probably enabling style enforcement, and you’ll a ton of false warnings from the API you’re using.

I am writing a standalone Python application or package

Use the PEP8 Styleguide if it makes sense. But don’t feel the need to refactor all your code, again, just take a look at Python’s own threading or logging modules, they don’t adhere to their own standards because it didn’t make sense.

ADDITIONAL NOTE:
I was pointed to the blog of Eric Husler, a veteran VFX and game pipeline programmer who covered this very topic in his post PyQT Coding Style Guidelines, his sentiments also follow the above:

“While you may be programming in Python, you’re creating Qt classes – inheriting their conventions along the way. When adding a new method, you should keep the consistency of Qt – that way someone working with your widgets doesn’t have to think about whether a method came from Python, and so use underscores, or came from Qt itself, and so use camel humps.”

He offered good examples of what we are discussing above, here is a widget with custom functions consistent with QT:

widget = MyWidget()
widget.setWindowTitle('Testng')
widget.loadItems()
widget.setMaximumWidth(100)
widget.refreshResults()

Here is a widget that has custom functions that adhere to PEP8 style but are inconsistent with QT:

widget = MyWidget()
widget.setWindowTitle('Testing')
widget.load_items()
widget.setMaximumWidth(100)
widget.refresh_results()

I recently had a discussion about storing relationships in Maya, and hadn’t realized the role of the message attribute wasn’t this universally cherished thing. In previous posts entitled ‘Don’t use string paths‘, or ‘Why Storing String Refs is Dangerous and Irresponsible‘ I outlined why this is the devil’s work, but in those posts I talked about the API, PyMel and Message Attrs. I didn’t really focus on why message attrs were so important: they serialize node relationships.

For quite some time I have advocated storing relationships with message attrs. At the Maya SIGGRAPH User Event, when they asked me to speak about our modular rigging system, I kind of detailed how we leveraged those at Crytek in CryPed.

I am not quite sure when I started using message attrs to convey relationships, I’m no brainiac, it could have been after seeing this 2003 post from Jason Schleifer on CGTalk:

Or maybe I read it in the Maya docs (unlikely):

“Message attributes only exist to formally declare relationships between nodes. By connecting two nodes via message attributes, a relationship between those nodes is expressed.”

So why does Maya use this, and why should I?

As you read in the docs above, when Maya wants to declare a relationship between a camera and image plane, they do so with a message attribute that connects them. This is important because this bond won’t be broken if the plane or it’s parent is renamed. As soon as you store the string path to a node in the DAG, that data is already stale.  It’s no longer valid.  When you query a message attribute, Maya returns the item, it’s DAG path will be valid, regardless of hierarchy or name changes.

Jason’s example above is maybe the most simple, in my image (a decade later) you can see the messages declaring many relationships abstracting the character at three main levels of interface, Character, ChatacterPart and RigPart. I talked about the basic ideas here in a 2013 post about object oriented python in Maya.

Though Rob vigorously disagreed in the comments there, I am still doing this today.  Here’s an example from the facial code we released in EPIC’s ARTv1 rigging tools some time ago. The face is abstracted on two levels, the ‘face’ and the ‘mask’, here I am only displaying the message connecting them:

By using properties as described in that previous blog post, below I am accessing the system, creating a face instance, walking down the message connection to the mask node, and then asking it for the attach locations. It’s giving me these transforms, by querying the DAG, live:

So, that property looks like this:

    @property
    def attachLocations(self):
        return cmds.listConnections(self.node + '.attachLocations')
    @attachLocations.setter
    def attachLocations(self, locs):
        for loc in locs:
            utils.msgConnect(self.node + '.attachLocations', loc + '.maskNode')

Setting the attach locations through python would look like this, and it would rebuild the message attrs:

face.mask.attachLoactions = ['myLoc1', 'myLoc2']

Working like this, you have to think hard about what a rigger would want to access at what level and expose what’s needed. But in the end, as you see, through python, you have access to everything you need, and none of the data is stale.

How and when to use strings

There are times when the only way you can store a relationship is by using a string in some fashion. Here are some situations and how I have handled them in the past, feel free to leave a comment with your experiences.

• Maya can’t store a relationship to something that doesn’t exist (has been deleted). It can’t store a relationship when it’s not open. In these situations, instead of storing the name in an attr, I stamp the two nodes with a string attr to store the relationship, then you query the world for another node with a matching stamped attr.
• Many times you need to feed your class an initial interface node to build/wrap. Instead of feeding it a string name, you can query the world for node type, in the Ryse example above, the rigging and animation tools could query cmds.ls(type=’CryCharacter’), this would return all characters in the scene. This means all rigging and animation tools needed a common ‘working character’ combobox at the top to define the character the tool is operating on. If you don’t have a node type, you can use a special string attr to query for.
• Sometimes you’re like saving joint names to serialize skinning data or something. You can use message attrs to play it safe here as well. Some pseudocode: For character in characters, if character identifier matches file on disk, for mesh in character.meshes if mesh in file skin it. For joint in character.joints if in file, add them to the skincluster, etc. Here you’re validating all your serialized string data against your class which is traversing the DAG live.
• Message attrs can get SLOW if you’re tracking thousands of items, you should only be tracking important things you would want later. In CryPed, when we wanted to track all nodes that were created when a module was built, we would stamp them all with a string attr that was the function name that built the module. To track this kind of data HarryZ at Crytek had the pragmatic idea of just doing a global ls of the world when a buildout started and then one at the end and boolean them out, this caught all the intermediate and utility nodes and everything generated by the rigging code.

Lots of people were interested in Deformer Weights and ways to save/load skin weights faster. Trowbridge pointed out that the API now allows for setting in one go, no loops needed, like the C++ call. Aaron Carlisle on our team here at Epic had noticed the same thing. Aaron took the time to write up a post going over it here:

Using GetWeights and SetWeights in the Maya Python API

Also, it looks like you can get/set blind data in one go now… 😮

In 2006, the team at Crytek was hard at work trying to come up with ways to ship Crysis –we had definitely bitten off more than we could chew. I found some videos that are now a decade old when cleaning out my HD, but it’s interesting to see –that’s for sure!

Facial Editor

We didn’t have a clue how we would animate all the lines of dialog that were required for the game.  3D Studio Max, which we used to animate at the time had nothing as a means of animating faces, now a decade later, Max and Maya still have zero offering to help with facial rigging or animation.  So.. we decided to write our own. Stephen Bender (Animation Lead) and I worked with Timur Davidenko and Michael Smith (Programmers) on this tool. Marco Corbetta wrote the 2.5D head/facial tracker. Here’s a video:

The user would feed into the system a text file, an audio file, and a webcam video of themselves. It would generate the mouth phonemes from the text/audio, and upper 2/3rds of the face from the video. The system would generate this animation on the same interface the animators used to animate so it was easily editable. It shipped with the MS speech DLL, but you could swap that for Annosoft if you licensed it. Crysis shipped with all characters having 98 blendshapes, driven by Facial Editor curves/animation using non-linear expressions. Imagine shipping a game today without having animators touch a face in a DCC app!

click to enlarge

PhotoBump

Many people know that Crytek released the first commercially available normal map generator: PolyBump, but rarely has anyone heard of it’s companion: PhotoBump. This was Created by Marco Corbetta around the same time, but released only to CryEngine licensees in 2005. It was probably one of the first commercial photogrammetry apps, and definitely one of the first uses of photogrammetry in games. Much of the rocky terrain in Crysis was created with the help of  PhotoBump! Marco also stamped/derived high frequency details from the diffuse, which I hadn’t seen others do until sometime after.

SIGGRAPH Best Realtime Graphics 2007

Here’s the SIGGRAPH ET reel from the year we released Crysis. I still can’t believe some of this stuff, like the guy pathfinding across the bridge of constrained boards and pieces of rope! I actually cut and edited this video myself back then, rendering it all out from the engine as well!

Windows to the Soul

In CG, the eyes are unforgiving.  For thousands of generations we have had to decipher true intent of other human beings from their eyes, because of this, we have evolved to notice the most minute, 1mm shift in eye shape. If there was one part of a digital character that was one of the most difficult to simulate and render, it would be this. Let’s talk about it.

Talking About the Eyes

First, I am going to go over some eye terminology. There’s a lot out there, but I am only going to go over what is required to give meaningful feedback in dailies. :D  You don’t need to be an ophthalmologist to discuss why a character’s eyes don’t look right.

The Iris – This is the round, colored circle in the eye, it encompasses the pupil, but it is not the pupil.
The Pupil – The black dot, or the hole in the iris.
The Sclera – The ‘whites of your eyes’
The Cornea – This is the bulge over the iris

Vergence – The eyes converge or turn inward to aim at the object a person is gazing at. They diverge, or turn outward when tracking something that is receding, if they diverge more than parallel, you can say the person is ‘walleyed’ (see: strabismus).

KEY TAKEAWAY – When viewing and reviewing eyes, it’s important to notice how the lids break across the iris and the shape of the negative space the lids and the iris create in the sclera. When we look at someone we are mainly seeing this shape. For more advanced readers, I picked the image above because you can really see the characteristics of the wetness meniscus, and eyelash refraction, but we’ll talk about that later.

Eye Placement

Initial eye placement is very important. Eyes too large or placed improperly will not rotate accurately when set in the face. There’s a lot you can learn by just looking at yourself in a mirror, looking at a friend, checking or scouring youtube. There’s even more you can learn from reading forensic facial reconstruction textbooks!

There are a few books for forensic artists, and most have information about placing eyes for facial re-construction. In my previous post about the jaw, I talked about forensic facial reconstruction a bit, these are my go-to books when it comes to eye (and teeth) placement:

Face It: A Visual Reference for Multi-ethnic Facial Modeling
Forensic Art and Illustration
Forensic Analysis of the Skull
Facial Geometry

I use a Mary Kay Travel Mirror and have bought them for all riggers/animators on my teams, at six bucks you can’t go wrong.

If you were to draw a line from the upper and lower orbits of the eye socket, it would be in line with the back of the cornea. I don’t usually like to point to skeletal reference, but in this case the orbits are relatively bony parts of the face that can be seen in surface anatomy.

Here are some good anatomical images for centering the eye in the socket (click to enlarge):

       

And in practice; here’s eye placement from Marius, the hero Character in Ryse the video game. Notice that the eyeball doesn’t even cover the entire ocular opening when viewed front on in wireframe, as the with anatomical images above

Abdenour Bachir, Ryse 2011, Crytek (click to enlarge)

Lastly, here is a GIF I made from a video tutorial called ‘How to Paint the Human Eye‘ by Cat Reyto. It shows eye placement rather well.

So this has all been discussing eye placement in the socket, but what about eye socket placement in the face?  This is where proportions come into play. The books above, “A Visual Reference to Multi-Ethnic Facial Modeling” and “Facial Geometry” are very good at discussing facial proportions and showing you how those proportions can change based on ethnicity. Here is a page from “Facial Geometry” that shows the basic facial proportions, most importantly pupil and eye placement relative to the rest of the face:

Here’s a page from the other book, which actually discusses 3D modeling. This is the chapter on eye placement, anyone interested in facial modeling should pick this up, it’s a great full color reference:

In my post about the jaw, I discussed jaw placement relative to the eyes and pupils, you can do the inverse: check the eye placement relative to teeth that you feel happy with. Notice that there’s a correlation between the molars or width of the upper teeth and the pupils.

Sometimes art direction would like ‘larger’ eyes, this is sometimes attempted by making the eyeball larger, but then it can feel weird when the eye is too large for the socket. This causes issues with the rotations of the eye. For reference, take a look at people who have been handed down a specific neanderthal gene for large eyes (or Axenfeld-Rieger syndrome), like the Ukrainian model Masha Tyelna. She has large eyeballs, but also the facial physiology to accept them:

Eye Movement

Range of Movement

Making digital humans, often from fiction, I find that these numbers are all relative, but the book Three Dimensional Rotations of the Eye, has a some good information, as does this chapter of another book: Physiology of the Ocular Movements. From default pose I usually fond that each eye can rotate on the horizontal plane 40 degrees in each direction (right and left) and 30 degrees in each direction for the vertical plane (up and down).

Eye ‘Accommodation’ and Convergence
For the purpose of rigging, the eyes are at maximum divergence (or parallel) at about 1.5m or 6 feet. I have not come across the eye gaze distance that results in maximum divergence, if you happen to have that information; let me know!

Pupils dilate to focus on a near object, this is known as accommodation. A standard young person’s eye can gaze/focus on objects from infinity to 6.5cm from the eyes. In action, eyes cannot converge/diverge faster than about 25 degrees a second.

“Why Does My Character Look Cross-Eyed”?

So, take a look at the MRI at the top of this page. Do you notice that the eyes are not looking forward or parallel? It’s because the eyes aren’t parallel when looking parallel/at an infinite distance. Human eyes bow out a few degrees, the amount of degrees varies between 4 and 6, per eye. The amount ‘off’ an eyeball is bowed outward is called the ‘kappa angle’, check this diagram below:

Let’s go over some more terminology:

Pupillary Axis – A line drawn straight out the pupil.
Visual Axis – A line from the fovea to the fixation target or the item being gazed upon.
Angle Kappa – the angle between the pupillary and visual axis.

The Hirschberg Test

The way ophthalmologists determine if eyes are converging properly is with the Hirschberg test. It’s a simple test where they ask a child to look at a teddy bear, and they shine a pen light into their eye, they look at his this light source reflects off the cornea, and they can tell if the child has a lazy eye/issue converging his eyes on a point.

Doing a ‘CG Hirschberg’ test requires a renderer with accurate reflections. You place a point light directly behind the camera and have the rig fixate/gaze directly onto the camera.

For more information on Angle Kappa, as well as data on the average angle across groups of people, check out Pablo Artal’s blog posts.

KEY TAKEAWAY

Even though the eyes bow out a little bit, for all intents and purposes, as you see in the Hirschberg, the pupil still seems like it is looking at the person. So yes, the pupillary axis is off by a few degrees, but with the refraction it’s not too noticeable (<1mm reflection offset from the center of the cornea). An Angle Kappa offset should be built into your rigs, you should not have the pupillary axis converging on the fixation target, this is what causes CG characters to look cross-eyed when viewed with scrutiny.

“Why Don’t My Eyes Feel Real?”

Hanno Hagedorn, Crysis 2005, Crytek

A difficult issue with eyes in computer graphics is making them feel like they are really set in the face.

The eye has a very interesting soft transition where the sclera meets the lids.  Some of this is ambient occlusion and shadows from the brow and lids, and some of it is from reflection of the upper eyelashes.

In 2005, Hanno Hagedorn and I were working on Crysis, we were having an issue with eyes that I called ‘game eye’. It’s where the sclera, has a sharp contrast with the skin.  In 2005, there was no obvious solution to do this in realtime.

We (I still credit Hanno) solved this in a very pragmatic way, the following is from one of our slides at GDC 2007:

We created an ‘eye overlay’, a thin film that sat on the eye and deformed with the fleshy eye deformation. A lot of games today, including Paragon, my current project at Epic, use this technique. Many years later, a famous VFX company actually tried to patent the technique.

Here are those same meshes, six years later on Ryse: Son of Rome:

Wetness Meniscus

One thing added since Crysis is the tearline or wetness meniscus. This is very important, it’s purpose of this is to kick up small spec highlights and fake the area where the lid meets the sclera. You can see this line in photo of the female eye where I have outlined initial anatomical terms.

Eye Rendering

All of these eye parts are not easy to shade correctly, Nicolas Schulz describes how Crytek implemented a forward pass in their deferred renderer to deal with eyes in his 2014 paper The Rendering Technology of Ryse. Nicholas has two slides dedicated to eye shading:

The eye shader used on Ryse is documented in depth here [docs.cryengine.com], the modified HLSL CFX shader code is here [github].   Some important features of the shader include:

• Cornea refraction and scattering – this is important, it simulates the refraction of the liquid in the cornea
• Iris color, depth, self shadowing, and SSS – Since there is no physical iris, we need to fake that there is a physical form there using a displacement map (POM)
• Eye occlusion overlay depth bias – this is very important, it allows you to push/pull the depth of the overlay film, so that the eye doesn’t penetrate through it
• Sclera SSS – not to be overlooked, or else the eye looks like a golfball.

These were the images that fed into the shader features above, and below are the overlay spec and AO masks, and the spec texture for the wetness/tearline.

This is an example of the iris displacement map ingame seen with a debug render view (Xbox One, 2011)

Abdenour Bachir, Ryse, Crytek 2011

Does the eye really need a corneal bulge?

On Ryse and Crysis, we had spherical eyes, and faked the corneal bulge with a shader and fleshy eye deformation. A corneal bulge means that you really have to have your shit together when it comes to the eye deformation. All those layers mentioned above have to deform with the bulge and it can be a lot to manage across all the deformation contexts of the eye (blink directionals, squint directionals, etc) On Paragon, we have non-spherical eyes, and it’s very challenging to deform the eyes properly in the context of a 60hz e-sports title with only one joint per lid. It’s actually pretty much impossible.

Altogether

Below, you see the eyes in Ryse: Son of Rome, they feel set in the face, and consistent with the world and hyper-real style of the game. (in-game mesh and rig, click to enlarge)

Abdenour Bachir, Ryse 2012, Crytek

Scanning/Acquiring Eyes

When scanning a character’s head it’s important to get their eyes fixed at a gaze distance you have recorded. I also use the scan to see how the iris breaks across the lid and infer information about how the eye will be set in the face and skull.

• If you want to take your own eye texture reference, shoot through a ring or attach a light co-axial with the camera lens, try to get the highlight in the pupil as you will discard this part of the image anyway.
• Because the eye is shiny and it’s characteristics change as you move around it, photogrammetry often falls flat.
• Because the eye is a complex translucent lens, cutting spec with cross-polarization often also falls flat. Light loses it’s polarization when bouncing around in there.

When it comes to scanning the eyes themselves, Disney Research has published an interesting paper on the High Quality Capture of Eyes.

How You Review Eyes

First off you should check eyeball depth and placement, as shown above.

• You should be reviewing eyes with at least the FOV of a portrait lens: 80mm or a 25 degree FOV. When getting ‘all up in there’ I often use a 10 degree FOV.
• Try to review them at the distance you will see them in your shipping product.
• If you are embodying the person that is the fixation point of the digital human, you really need runtime look IK (for the eyes, but preferably feathered torso>head>neck>eyes). From some distance, you can tell is someone is looking at your eyes or your ear. Think about that. The slightest anim compression or issue in any joint from root to eyes can cause the gaze to be off a few degrees and that’s all it takes.
• If you’re doing a lot of work, build a debug view into your software that draws the pupillary axis and visual axis, all the way to the fixation/gaze point

Some had asked me to package the previous code together into a tool. I was reluctant because there were many situations where it just didn’t work. I made something to aid a discussion on the beta forum.

However, accidentally, I seem to have the code working. It’s odd, but the solution to skinClusters not being paintable after deformerWeights was used to load them was to try and make another skinCluster (which errors, but fixes the Artisan issue).

I have tossed this on GitHub here:
https://github.com/chrisevans3d/deformerWeightsPlus/

DISCLAIMER/WARNING: Trying to implement this in production I have found other items on top of the massive list that do not work. The ignore names flag ‘ig’ causes a hard crash on file load, the weight precision flag ‘wp’ isn’t implemented though it’s documented, the weight tolerance flag ‘wt’ causes files to hang indefinitely on load. When it does load weights properly, it often does so in a way that crashes the Paint Skin Weights tool. I have reported this in the Maya Beta forums.

Previously I discussed the promise in Maya command ‘deformerWeights‘. The tool that ships with Maya was not very useful, but the code it called was 125 times faster than python if you used it correctly..

Let’s make a python class that can save and load skin weights. You hand it a few hundred skinned meshes (avg Paragon character) and it saves the weights and then you delete history on the meshes, and it loads the weights back on.  What I just described is the process riggers go through when updating a rig or a mesh _every day_.

Below we begin the class, we import a python module to parse XML, and we say “if the user passed in a path, let’s parse it.”

#we import an xml parser that ships with python
import xml.etree.ElementTree
 
#this will be our class, which can take the path to a file on disk
class SkinDeformerWeights(object):
    def __init__(self, path=None):
        self.path = path
 
        if self.path:
            self.parseFile(self.path)

Next, let’s make this parseFile function. Why is parsing the file important? In the last post we found out that there’s a bug that doesn’t appropriately apply saved weights unless you have a skinCluster with the *exact* same joints as were exported. We’re going to read the file and make a skinCluster that works.

#the function takes a path to the file we want to parse
def parseFile(self, path):
    root = xml.etree.ElementTree.parse(path).getroot()
 
    #set the header info
    for atype in root.findall('headerInfo'):
        self.fileName = atype.get('fileName')
 
    for atype in root.findall('weights'):
        jnt = atype.get('source')
        shape = atype.get('shape')
        clusterName = atype.get('deformer')

Now we’re getting some data here, we know that the format can save deformers for multiple shapes, let’s make a shape class and store these.

class SkinnedShape(object):
    def __init__(self, joints=None, shape=None, skin=None, verts=None):
        self.joints = joints
        self.shape = shape
        self.skin = skin
        self.verts = verts

Let’s use that when we parse the file, let’s then store the data we paresed in our new shape class:

#the function takes a path to the file we want to parse
def parseFile(self, path):
    root = xml.etree.ElementTree.parse(path).getroot()
 
    #set the header info
    for atype in root.findall('headerInfo'):
        self.fileName = atype.get('fileName')
 
    for atype in root.findall('weights'):
        jnt = atype.get('source')
        shape = atype.get('shape')
        clusterName = atype.get('deformer')
 
        if shape not in self.shapes.keys():
            self.shapes[shape] = self.skinnedShape(shape=shape, skin=clusterName, joints=[jnt])
        else:
            s = self.shapes[shape]
            s.joints.append(jnt)

So now we have a dictionary of our shape classes, and each knows the shape, cluster name, and all influences. This is important because, if you read the previous post, the weights will only load onto a skinCluster with the exact same number and names of joints.
Now we write a method to apply the weight info we parsed:

def applyWeightInfo(self):
    for shape in self.shapes:
        #make a skincluster using the joints
        if cmds.objExists(shape):
            ss = self.shapes[shape]
            skinList = ss.joints
            skinList.append(shape)
            cmds.select(cl=1)
            cmds.select(skinList)
            cluster = cmds.skinCluster(name=ss.skin, tsb=1)
            fname = self.path.split('\\')[-1]
            dir = self.path.replace(fname,'')
            cmds.deformerWeights(fname , path = dir, deformer=ss.skin, im=1)

And there you go. Let’s also write a method to export/save the skinWeights from a list of meshes so we never have to use the Export DeformerWeights tool:

def saveWeightInfo(self, fpath, meshes, all=True):
    t1 = time.time()
 
    #get skin clusters
    meshDict = {}
    for mesh in meshes:
        sc = mel.eval('findRelatedSkinCluster '+mesh)
        #not using shape atm, mesh instead
        msh =  cmds.listRelatives(mesh, shapes=1)
        if sc != '':
            meshDict[sc] = mesh
        else:
            cmds.warning('>>>saveWeightInfo: ' + mesh + ' is not connected to a skinCluster!')
    fname = fpath.split('\\')[-1]
    dir = fpath.replace(fname,'')
 
    for skin in meshDict:
        cmds.deformerWeights(meshDict[skin] + '.skinWeights', path=dir, ex=1, deformer=skin)
 
    elapsed = time.time()-t1
    print 'Exported skinWeights for', len(meshes), 'meshes in', elapsed, 'seconds.'

You give this a folder and it’ll dump one file per skinCluster into that folder.
Here is the final class we’ve created [deformerWeights.py], and let’s give it a test run.

sdw = skinDeformerWeights()
sdw.saveWeightInfo('e:\\gadget\\', cmds.ls(sl=1))
>>>Exported skinWeights for 214 meshes in 2.433 seconds.

Let’s now load them back, we will iterate through the files in the directory and parse each, applying the weights:

import os
t1=time.time()
path = "e:\\gadget\\"
files = 0
for file in os.listdir(path):
    if file.endswith(".skinWeights"):
        fpath = path + file
        sdw = skinDeformerWeights(path=fpath)
        sdw.applyWeightInfo()
        files += 1
elapsed = time.time() - t1
print 'Loaded skinWeights for', files, 'meshes in', elapsed, 'seconds.'
>>> Loaded skinWeights for 214 meshes in 8.432 seconds.

“>>> Loaded skinWeights for 214 meshes in 8.432 seconds.”

So that’s a simple 50 line wrapper to save and load skinWeights using the deformerWeights command. No longer do we need to write C++ API plugins to save/load weights quickly.

This post was originally going to be entitled ‘Why Everyone Writes their Own Skin Exporter’. Maya’s smooth skin weight export tool hasn’t changed since Maya 4.0 when it was introduced 15 years ago. It saves out greyscale weightmaps in UV space, one image per joint influence per mesh. The only update they have done in 15 years is change the slider to go from a max of 1024 pixels to 4096, and now 8192!

Autodesk understands the need and importance of a skin weight exporter, they even ship it as a C++ example in their Maya Developer Kit / SDK. So why are they still shipping this abomination above?

Like blind data discussed before, in pythonland we cannot set skinweights in one go, we must iterate through the entire mesh setting skin weights one vertex at a time. This means a Maya feature or C++ plugin can save and load skin weights in seconds that take python 15 minutes.

I have written lots of skin weight save/load crap in my time, and as I sat down and started to do that very thing in an instructional blog post one night, we had an interesting discussion in the office the next day. ADSK added ‘Export Deformer Weights‘ in 2011, but it has never really worked. I don’t know a single person who uses it. But it does save and load ‘deformer’ weights via the C++ API –so there’s real promise here!

Save/load skin weights fast without writing a custom Maya plugin? This is kinda like the holy grail, which is ridiculous, but you should see the lengths people go to eek out a little more performance! My personal favorite was MacaroniKazoo back in 2010 reaching in and setting the skinCluster weight list directly by hand using an unholy conglomeration of python API and MEL commands. Tyler Thornock has a post that builds on this here.

So I asked the guys if anyone had looked at Export Deformer Weights recently, everyone either hadn’t heard of it, heard it was shit, or had some real first-hand experience of it being “A bit shit.” But still –the promise was there!

Export Deformer Weights: Broken and Backwards

So, first thing’s first, I made an awesome test case. I am going to go over all the gotchas and things that are broken, but if you don’t want to take this voyage of discovery, skip this section.

I open the deformer weight export tool, and just wow.. I mean the UI team really likes it’s space:

I save my skin weights to an XML file, delete history on the mesh, and open the import UI:

Gotcha 1) It requires a deformer to load weights onto. You need to re-skin your mesh.

I re-skinned my mesh and loaded the XML using Index.. drumroll..

Well, this is definitely not applying the weights back by vertex index. I decided to try ‘Nearest’:

Gotcha 2) Of the options [Index, Nearest, Over], ‘Index’ is somehow lossy, and anything other than ‘Index’ seems to crash often, ‘Nearest’ seems totally borked. (above)

So this was when I just began to think this was a complete waste of my time. I was pretty annoyed that they even shipped a tool like this, something that is so needed and so important, yet crashes frequently and completely trashes your data when it does work.

Not Taking ‘Broken and Backwards’ for an Answer

I am already invested and, not understanding how loading weights by point index could be lossy and broken, I decided to look at the XML file. The tool writes out one XML file per skinCluster, here’s a rundown of the file format:

Mesh Info (Shape) – the vertices of the shape are stored local space x,y,z and corresponding index

<?xml version="1.0"?>
<deformerWeight>
  <headerInfo fileName="C:/Users/chris.evans/Desktop/test_sphere.xml" worldMatrix="1.000000 0.000000 0.000000 0.000000 0.000000 1.000000 0.000000 0.000000 0.000000 0.000000 1.000000 0.000000 0.000000 0.000000 0.000000 1.000000 "/>
  <shape name="pSphereShape1" group="7" stride="3" size="382" max="382">
    <point index="0" value=" 0.148778 -0.987688 -0.048341"/>
    <point index="1" value=" 0.126558 -0.987688 -0.091950"/>
    <point index="2" value=" 0.091950 -0.987688 -0.126558"/>
    ...

Joints (Weights) – There is one block per joint that calls out each vertex that it have influences for on the shape

  <weights deformer="skinCluster1" source="root" shape="pSphereShape1" layer="0" defaultValue="0.000" size="201" max="380">
    <point index="0" value="0.503"/>
    <point index="1" value="0.503"/>
    <point index="2" value="0.503"/>
    ...

And that’s it, not a lot of data, nothing about the skinCluster attributes or options, no support for spaces like UV or world. (odd, since it’s had UV support for 15 years)

Next I decided to run the tool again and see what command it was calling, I then looked up the command documentation and here’s where it gets interesting, go ahead, take a look!

So now I am hooked, someone is putting some thought into this –at least on some level.

I don’t at all understand why the UI has none of these options, but I need to get this working. If you read through the docs, the command also supports:

• Exporting multiple skinClusters/shapes/deformers per XML file
• Exporting skinCluster/deformer attributes like ‘skinningMethod’ and ‘envelope’
• Local and world space positions with a positional tolerance

“Someone is putting some thought into this”

So I started trying to figure out why a file format that explicitly knows every influence of every vertex by index and inf name, doesn’t load weights properly. After some trials I hit gotcha #3:

Gotcha 3) Of the options the weights only load properly if the skinCluster has the *exact* same influences it was saved with. Which really makes no sense, because the file format has the name of every joint in the old skinCluster.

So now I had it working, time to wrap it and make it useful.

The Documentation is a Lie.

So, first thing’s first, I did a speed test.

Importing with deformerWeights was about 125 times faster: Gold mine.

But I just couldn’t get some of the flags to work, I thought I was just a moron, until I finally tried the code example in the ADSK Maya documentation, which FAILS. Let’s first look at the -vc flag, which is required to load using ‘bilinear’ or ‘barycentric’ mapping/extrapolation:

cmds.deformerWeights ("testWeights.xml", ex=True, vc=True, deformer="skinCluster1")
 
# Error: Invalid flag 'vc'
# Traceback (most recent call last):
#   File "<maya console>", line 1, in <module>
# TypeError: Invalid flag 'vc' #

Gotcha 4) The python examples do not work. -vertexConnections flag doesn’t work, -attribute flag doesn’t work, so no saving skincluster metadata like ‘skinningMethod’, etc. Because of that, ‘barycentric’ and other methods that need vertex connection info do not work. The ‘deformer’ flag shows that it takes a list of deformers and writes them all to one file, but this is not true, it takes a single string name of a deformer.

I now know why the UI doesn’t have all these cool options! –they don’t work!

Gotcha 5) It doesn’t take a file path, to save a file to a path you need to specify the filename, and then the path separate.

cmds.deformerWeights ("testWeights.xml", path='d:\\myWeight\\export\\folder\\', ex=True, deformer="skinCluster1")

Perhaps someone fixed this stuff, documented it, and then reverted the fix, but this has been around since 2011.. I tried the above in maya 2016 latest service pack and all my links above are to that version of the documentation.

I wasn’t really intending to write this much, so now that we know this can import weights 125 times faster, we’ll make a tool to utilize it. Stay tuned!

The stream of our SIGGRAPH Realtime Live demo is up on teh internets. If you haven’t seen the actual live demo, check it out!

I feels amazing to win the award for best Realtime Graphics amongst such industry giants. There are so many companies from so many industries participating now, and the event has grown such much. Feels really humbling to be honored with this for a third year; no pressure!

Last month Dell had the Black Friday in July sale and this beauty was on sale for 500 dollars off, plus 10% if you ordered by phone. I decided it was time to replace my beloved Lenovo x220t.

The Alienware 13 might be ugly and lack a Wacom digitizer, but it does have an nVidia GTX 965M and an OLED screen with 211% sRGB coverage! As the Lenovo Yoga X1 only has integrated graphics, I think the Alienware is the machine for 3D artists.

If you’re a gamer who landed here because you wondered how to get the most out of your amazing display, or wondered why everything is super-red-pink, it’s time to put your big boy pants on! Calibrating the monitor wasn’t so straight forward, but let’s jump into it.

We are going to use an open source Color Management toolkit called ArgyllCMS [Download it here]. It can use many different hardware calibration devices, I have used it with the xRite Huey and the Spyder5.

One thing that’s important to know is that all the sensors are the same, you only pay for software features. If you don’t own a calibrator, you can buy the cheapest Spyder, because it’s the same sensor and you are using this software, not the OEM software.

Next we’re going to use a GUI front end built to make ArgyllCMS more user friendly. It’s called DisplayCAL, but it requires a lot of libs (numPy, wxWidgets, etc) so I recommend downloading this zero install that has everything.

Be sure to set the ‘White level drift compensation’ on. You will need to ignore the RGB adjustment it first asks you to fuss with because there is no RGB adjustment on your monitor.

When you are through, you will see the following (or something like it!):

Note: DisplayCAL can also output a 3d LUT for madVR, which works with any number of video playback apps. Check here to force your browser to use your color management profile. If it’s useful, I can make another post detailing all the different settings and color-managed apps to make use of your monitor.

I hope you found this useful, it should apply to the Lenovo Yoga X1 and any other OLED laptops in the coming months.

All ‘virtual’ joints that we place are based on virtual anatomical surface landmarks. By ‘virtual’ I mean the polygonal rendermeshes that our ‘puppet’ will drive. As a rigger, you have to be able to look at the surface anatomy (polygonal mesh) and determine where a joint should be placed, but the solution is not always obvious.

“The Dental Distress Syndrome” (Dr. A.C. Fonder). (1988)

The jaw is one of these tricky situations! Many people think the jaw rotates from the ‘socket’ or ‘fossa’ that the jaw (or mandible — or mandibular condyle) fits into, but this is not the case.

As riggers, we sometimes need to ignore internal anatomy and focus on surface anatomy, which is our final deliverable. This means think about the center of rotation for the entire mass of flesh (or vertices) that we’re moving. For the jaw of a human(oid) this pivot is under the earlobe when viewed from the side.

Rotating directly from the ‘socket’ would result in this incredible ‘derp‘ shown above, but the temporomandibular (TMJ) joint doesn’t work like this when we open our jaw/mouth.

Instead, the jaw/mandible slides forward as the mouth opens, like you see in this ‘live MRI’ slice above, resulting in this sexay jaw open below. You can see her mandible/jaw slide forward as she opens her mouth.

This post is of course ignoring the fact that you can rig the jaw in a way that, through a combination of rotation and translation uses the TMJ as a pivot and rotates around the true jaw mass center of rotation described above. You can do virtually anything, you can drive with your feet; that doesn’t mean it’s a good idea. This post is probably most important for riggers creating characters fast, using tools that generate skeletons from user-placed signposts or locators.

Speaking of signposts and locators, another tool to help you with placement is to constrain a toroid or circle to your joint, it can help you visualize the jaw swing quickly:

For those interest in further investigation, the paper ‘Rotation and Translation of the Jaw During Speech’ by Jan Edwards and Katherine Harris (1990) can be downloaded [here].

Jaw and Teeth Placement (Modeling)

Quite a few people have told me they found this helpful. I would like to add that teeth placement is very important and something you should check before rigging.  For modelers, here are some radiograms showing teeth placement in reference to the facial surface anatomy (click to enlarge):

  

As you see above, we’re primarily interested in 6, 7, 20, and 21. Rhine et al have created charts for caucasoid and negroid Americans of varying builds. (Click below to enlarge)

NOTE: Soft tissue thickness charts for the face are also a great place to gut-check your sub surface scattering maps and profiles!

Up next, placing the teeth so that they are large enough or wide enough is also important. Below, notice the item marked ‘J’, this is the average upper lip line in relation to the upper teeth.

Above is a forensic facial reconstruction proportion guideline. I like to augment that a bit to help with teeth placement. I find that it’s helpful to look at the incisors vs the nostrils, and the pupils vs the lip corners/molars. Here are some examples of that (click to enlarge):

In Maya 2016, by default, Node Editor cannot display more than 500 connections. Let’s put this in perspective. Let’s say you have two attributes and each drives 20 joints. BOOM, you’re over the limit bro.. what you doin? Why you breakin’ Maya?

The solution, the warning says is to set an option var. Oh yeah that’s a frickin’ charm, that’s like unlocking all streets on my GPS by loosening a nut on the underside of my car.

I count on this visual editor to display connections, that’s all it needs to do. I have had rigs that make this update once every 10 seconds, but at least I could find what I was looking for and troubleshoot it. I would much rather a drop in fidelity, like stop drawing nice curves or something –than just randomly not displaying data.

How is that even an option? Sometimes my mind wanders:

“Sir, we noticed that on some systems, the software can get slow if we display more than 500 connections.”
“How did you fix it?”
“Well we just decided not to show more than 500 connections.”
“Stellar work Raymond. Can this limit be adjusted in the Settings and Preferences? Maybe the first time Maya opens or the user opens Node Editor it can be set based on his hardware.”
“Well right now it’s an optionVar, I mean that works well enough for me.”

Here’s what you can enter into the mel evaluation line in the lower left to set your max display depth to ten million:

optionVar -iv "editorConnectionScanLimit" 10000000;

I think ADSK probably hates me by now. ¬.¬

This is something that I found when cleaning my old computer out at work this week after an upgrade. This is an example of using a look at controller and a blendspace to drive a fleshy eye deformation in UE4 live at runtime.

This is the technique Jeremy Ernst used on Smaug’s eye in the demo we did with Weta over a year ago. Create a 2d blendspace for the eye poses. Export out left, right, up, down poses and connect them:

Grab the local transform from the eye, and set two vars with it:

Create a Look At Controller and integrate the blendspace:

Oh the clavicle.. you’re like the bastard redheaded stepchild.. is this guy an animator or what?

Anyway, sixteen years after the publication of Pose Space Deformation: A Unified Approach to Shape Interpolation and Skeleton-Driven Deformation, and fourteen years after Michael Comet released the poseDeformer plugin for Maya, which became the industry standard: Autodesk has now implemented the feature into the software.

In what seems like my early 20’s being acted out before my eyes, ‘Combination Sculpting’, the way of associating a fixer shape when other shapes are in a certain configuration (pioneered by Bay Raitt on Gollum in 2002), is also not a feature that will ship in Maya.

Riggers rejoice! We have voted this up on all feedback forums for years, and ADSK created the tool with direct involvement from the rigging community, great that it can finally see daylight. Now, all you Extension2 users grab it and work out the remaining kinks before everyone gets it in Maya 2017!

And if you’re upgrading to EXT2 today, don’t forget, FBX anim export/bake is fixed!

Plutarch tells that Alexander the Great visited the Libyan Sibyl in search of the correct constellation of checkboxes and steps to get meshes from Max to Maya.

I have spent many years of my life in studios where characters are modeled in a package other than Maya (often Max) and imported into Maya via FBX. Having worked along side great character artists like Hanno Hagedorn, Abdenour Bachir, and most recently Kevin Lanning and his team here at Epic, I cannot tell you how many hours of our lives were devoted to trying to get mesh tangents into the final product that resembled what they were in the original sculpt/bake. Not to mention brilliant pipeline programmers like Bogdan Coroi, or James Goulding‘s team here at Epic, many hours have been spent trying to solve this issue.

Sometimes it seemed some mystical channeling whereby some constellation of export or import checkboxes along with maybe layering an edit mesh modifier on top of your character before export to Maya worked. Sometimes the solution seemed to have been exporting only triangulated meshes to Maya, whereby you needed a (fragile) pipeline to allow you to have quaded for skinning and triangulated from Max for export.

Well, as it turns out, Maya has always ignored all mesh tangent data on FBX import.

I hope this post saves you some headache. At Crytek we looked to change the pipeline to store all normal maps in world space, another, more pragmatic solution, proposed by Jeremy Ernst here at Epic is to give the engine the static mesh from Max and the skinned mesh from Maya and just transfer the original data. Scott Parrish told me that his team bakes against the skinned FBX as it comes out of UE4, this is another way of solving the issue.

I also understand this is not a simple issue, all DCC packages work differently. Max allows users to turn edges, etc.. But it’s good to know that we’re also not crazy.