Perforce is a wily beast. A lot of companies use it, but I feel few outside of the IT department really have to deal with it much. As I work myself deeper and deeper into the damp hole that is asset validation, I have really been writing a lot of python to deal with certain issues; but always scripts that work from the outside.
Perforce has a system that allows you to write scripts that are run, server side, when any number of events are triggered. You can use many scripting languages, but I will only touch on Python.
Test Environment
To follow along here, you should set up a test environment. Perforce is freely downloadable, and free to use with 2 users. Of course you are going to need python, and p4python. So get your server running and add two users, a user and an administrator.
Your First Trigger
Let’s create the simplest python script. It will be a submit trigger that says ‘Hello World’ then passes or fails. If it passes, the item will be checked in to perforce, if it fails, it will not. exiting while returning a ‘1′ is considered a fail, ‘0′ a pass.
print'Hello World!'print'No checkin for you!'sys.exit(1)
Ok, so save this file as hello_trigger.py. Now go to a command line and enter ‘p4 triggers’ this will open a text document, edit that document to point to your trigger, like so (but point to the location of your script on disk):
Close/save the trigger TMP file, you should see ‘Triggers saved.’ echo’d at the prompt. Now, when we try to submit a file to the depot, we will get this:
So: awesome, you just DENIED your first check-in!
Connecting to Perforce from Inside a Trigger
So we are now denying check-ins, but let’s try to do some other things, let’s connect to perforce from inside a trigger.
from P4 import P4, P4Exception
p4 = P4()try:
#use whatever your admin l/p was#this isn't the safest, but it works at this beginner level
p4.user = "admin"
p4.password = "admin"
p4.port = "1666"
p4.connect()
info = p4.run("info")print info
sys.exit(1)#this will return any errorsexcept P4Exception:
for e in p4.errors: print e
sys.exit(1)
So now when you try to submit a file to depot you will get this:
Passing Info to the Trigger
Now we are running triggers, accepting or denying checkins, but we really don’t know much about them. Let’s try to get enough info to where we could make a decision about whether or not we want the file to pass validation. Let’s make another python trigger, trigger_test.py, and let’s query something from the perforce server in the submit trigger. To do this we need to edit our trigger file like so:
Triggers:
test change-submit //depot/... "python X:/projects/2010/p4/test_trigger.py %user% %changelist%"
This will pass the user and changelist number into the python script as an arg, the same way dragging/dropping passed args to python in my previous example. So let’s set that up, save the script from before as ‘test_trigger.py’ as shown above, and add the following:
importsysfrom P4 import P4, P4Exception
p4 = P4()
describe = []try:
p4.user = "admin"
p4.password = "admin"
p4.port = "1666"
p4.connect()except P4Exception:
for e in p4.errors: print e
sys.exit(1)printstr(sys.argv)
describe = p4.run('describe',sys.argv[2])printstr(describe)
p4.disconnect()sys.exit(1)
So, as you can see, it has returned the user and changelist number:
However, for this changelist to be useful, we query p4, asking the server to describe the changelist. This returns a lot of information about the changelist.
Where to Go From here
The few simple things shown here really give you the tools to do many more things. Here are some examples of triggers that can be created with the know-how above:
Deny check-ins of a certain filetype (like deny compiled source files/assets)
Deny check-ins whose hash digest matches an existing file on the server
Deny/allow a certain type of file check-in from a user in a certain group
Email a lead any time a file in a certain folder is updated
Did you find this helpful? What creative triggers have you written?
In Python, a Decorator is a type of macro that allows you to inject or modify code in functions or classes. I was turned onto this by my friend Matt Chapman at ILM, but never fully grasped the importance.
class myDecorator(object):
def__init__(self, f):
self.f = f
def__call__(self):
print"Entering", self.f.__name__
self.f()print"Exited", self.f.__name__
@myDecorator
def aFunction():
print"aFunction running"
aFunction()
When you run the code above you will see the following:
So when we call a decorated function, we get a completely different behavior. You can wrap any existing functions, here is an example of wrapping functions for error reporting:
class catchAll:
def__init__(self, function):
self.function = function
def__call__(self, *args):
try:
returnself.function(*args)exceptException, e:
print"Error: %s"%(e)
@catchAll
def unsafe(x):
return 1 / x
print"unsafe(1): ", unsafe(1)print"unsafe(0): ", unsafe(0)
So when we run this and divide by zero we get:
unsafe(1): 1
unsafe(0): Error: integer division or modulo by zero
Using decorators you can make sweeping changes to existing code with minimal effort, like the error reporting function above, you could go back and just sprinkle these in older code.
I have really been trying to learn some Python fundamentals lately, reading some books and taking an online class. So: wow. I can’t believe that I have written so many tools, some used by really competent people at large companies, without really understanding polymorphism and other basic Python concepts.
Here’s an example of my sequence method from before, but making it a class using special class methods:
I have been parsing through the files of other people a lot lately, and finally took the time to make a little function to give me general information about a sequence of files. It uses regex to yank the numeric parts out of a filename, figure out the padding, and glob to tell you how many files in the sequence. Here’s the code and an example usage:
#returns [base name, padding, filetype, number of files, first file, last file]def getSeqInfo(file):
dir = os.path.dirname(file)file = os.path.basename(file)
segNum = re.findall(r'\d+', file)[-1]
numPad = len(segNum)
baseName = file.split(segNum)[0]
fileType = file.split('.')[-1]
globString = baseName
for i inrange(0,numPad): globString += '?'
theGlob = glob.glob(dir+'\\'+globString+file.split(segNum)[1])
numFrames = len(theGlob)
firstFrame = theGlob[0]
lastFrame = theGlob[-1]return[baseName, numPad, fileType, numFrames, firstFrame, lastFrame]
I know this is pretty simple, but I looked around a bit online and didn’t see anything readily available showing how to deal with different numbered file sets. I have needed something like this for a while that will work with anything from OBJs sent from external contractors, to images from After Effects…
So I have always been wondering how you can create almost like a ‘droplet’ to steal the photoshop lingo, from a python script. A while ago I came across some sites showing how to edit shellex in regedit to allow for files to be dropped on any python script and fed to it as args (Windows).
It’s really simple, you grab this reg file [py_drag_n_drop.reg] and install it.
Now when you drop files onto a python script, their filenames will be passed as args, here’s a simple script to test.
The script itself is the first arg, then all the files. This way you can easily create scripts that accept drops to do things like convert files, upload files, etc..
I got some good feedback from the last post and updated the script to export JPEG Stereo (JPS) and PNG Stereo (PNS, really.) This way you can convert your images into a single lossless image that you can pop into photoshop and adjust hsv/levels, etc.
This is a super simple python script, no error padding. Also, keep in mind that coming from most modern camera rigs, you are saving like a 20-40 megapixel PNG compressed file here, wait until it says it is done saving, it may take a few seconds.
Many stereo cameras are using the new MPO format to store multiple images in a file. Unfortunately, nothing really works with these files (Other than Stereo Photo Maker). Here is a simple python wrapper around ExifTool that will extract the Right and Left image, and return EXIF data as a dict. I think this is probably easier than explaining how to use ExifTool, but you can see from looking at the simple wrapper code.
import mpo
#Name of MPO file, name of output, whether or not you want all EXIF in a txt log
mpo.extractImagePair('DSCF9463.MPO', 'DSCF9463', True)#>>Created DSCF9463_R.jpg#>>Created DSCF9463_L.jpg#>>Writing EXIF data
The above leaves you with two images and a text file that has all the EXIF data, even attributes that xnView and other apps do not read:
exif = getExif('DSCF9463.MPO')print exif["Convergence Angle"]#>>0print exif["Field Of View"]#>>53.7 degprint exif["Focal Length"]#>>6.3 mm (35 mm equivalent: 35.6 mm)
I have been really amazing myself at how much knowledge I have forgotten in the past five or six months… Most of the work I did in the past year utilized the UIC module to load UI files directly, but I can find very little information about this online. I was surprised to see that even the trusty old Rapid GUI Programming with Python and Qt book doesn’t cover loading UI files with the UIC module.
So, here is a tiny script with UI file [download] that will generate a pyqt example window that does ’stuff’:
importsysfrom PyQt4 import QtGui, QtCore, uic
class TestApp(QtGui.QMainWindow):
def__init__(self):
QtGui.QMainWindow.__init__(self)self.ui = uic.loadUi('X:/projects/2010/python/pyqt_tutorial/pyqt_tutorial.ui')self.ui.show()self.connect(self.ui.doubleSpinBox, QtCore.SIGNAL("valueChanged(double)"), spinFn)self.connect(self.ui.comboBox, QtCore.SIGNAL("currentIndexChanged(QString)"), comboFn)self.connect(self.ui.pushButton, QtCore.SIGNAL("clicked()"), buttonFn)def spinFn(value):
win.ui.doubleSpinBoxLabel.setText('doubleSpinBox is set to ' + str(value))def buttonFn():
win.ui.setWindowTitle(win.ui.lineEdit.text())def comboFn(value):
win.ui.comboBoxLabel.setText(str(value) + ' is selected')if __name__ == "__main__":
app = QtGui.QApplication(sys.argv)
win = TestApp()sys.exit(app.exec_())
Change the path to reflect where you have saved the UI file, and when you run the script you should get this:
I have gotten back into some pyqt in my spare time, just because it’s what I used on a daily basis at the last place I worked at. However, I had trouble getting it to run in my text editor of choice. (SciTE)
I couldn’t find a solution with like 45 minutes of googling. When trying to import PyQt4 it would give me a dll error, but I could paste the code into IDLE and it would execute fine. I found a solution by editing the python preferences of SciTE. I noticed that it wasn’t running python scripts the way IDLE was, but compiling them (?). I edited the last line to just run the script, and viola! It worked.
The Vatican is not very open with it’s art, the reason they scream ‘NO PHOTO’ when you pull a camera out in the chapel is that they sold the ability to take photos of it to a Japanese TV Station (Nippon TV) for 4.2 million dollars. Because the ceiling has long been in the public domain, the only way they can sell ‘the right to photograph’ the ceiling is by screwing over us tourists who visit. If you take a photo, they have no control over that image –because they don’t own the copyright of the work.
Many of you who know me, know I am a huge fan of Michelangelo’s work, this data was just too awesomely tempting and when I saw it posted publicly online, I really wanted to get my hands on the original assets.
Here is a python script to grab all of the image tiles that the flash app reads, and then generate the 8k faces of the cubemap. In the end you will have a 32,000 pixel cubemap.
First we copy the swatches from the website:
def getSistineCubemap(saveLoc):
importurllib#define the faces of the cubemap, using their own lettering scheme
faces = ['f','b','u','d','l','r']#location of the images
url = 'http://www.vatican.va/various/cappelle/sistina_vr/Sistine-Chapel.tiles/l3_'#copy all the swatches to your local drivefor face in faces:
for x inrange(1,9):
for y inrange(1,9):
file = (face + '_' + str(y) + '_' + str(x) + '.jpg')urllib.urlretrieve((url + face + '_' + str(y) + '_' + str(x) + '.jpg'), (saveLoc + file))urllib.urlcleanup()print"saved " + file
So, before we looked at just outputting a list of the files that were on device1 and not device2, now I will copy the files to a folder on the main device.
The tricky thing about this is I want the directory structure intact. After looking as os.path, and pywin32, I didn’t see anything like ‘mkdir’ where it would make all the folders deep needed to recreate the branch that a file was in. I did however find a function online:
def mkdir(newdir):
ifos.path.isdir(newdir):
passelifos.path.isfile(newdir):
raiseOSError("a file with the same name as the desired " \
"dir, '%s', already exists."% newdir)else:
head, tail = os.path.split(newdir)if head andnotos.path.isdir(head):
mkdir(head)if tail:
os.mkdir(newdir)
To copy the files and create the directories, I altered the previous script a bit:
I am anal-retentive about data retention. There, I said it. There are many times when I find myself in the situation of having two storage devices, that may or may not have duplicate files. I then want to erase one, but do I have all those files backed up?
I use two existing programs to aid me in my anal-retentivity: TerraCopy and WinMerge. Terracopy replaces the windows default copy with something much better (can hash check files when they are copied, etc). With WinMerge, I can right click a folder and say ‘Compare To…’ then right click another and say ‘Compare’. This tells me any differences between the two file/folder trees.
However, here’s an example I have not yet found a good solution for:
I want to erase a camera card I have, I am pretty certain I copied the images off –but how can I be sure! I took those images and sorted them into folders by location or date taken.
So I wrote a small and I am sure inefficient python script to help:
This is a simple proof-of-concept showing how to implement a perforce animation browser via python for MotionBuilder. Clicking an FBX animation syncs it and loads it.
The script can be found here: [p4ui.py], it requires the [wx] and [p4] libraries.
Clicking directories goes down into them, clicking fbx files syncs them and loads them in MotionBuilder. This is just a test, the ‘[..]‘ doesn’t even go up directories. Opening an animation does not check it out, there is good documentation for the p4 python lib, you can start there; it’s pretty straight forward and easy: sure beats screen scraping p4 terminal stuff.
You will see the following, you should replace this with the p4 location of your animations, this will act as the starting directory.
path1 = 'PUT YOUR PERFORCE ANIMATION PATH HERE (EXAMPLE: //DEPOT/ANIMATION)'
info = p4i.run("info")print info[0]['clientRoot']
That should about do it, there are plenty of P4 tutorials out there, my code is pretty straight forward. The only problem was where I instanced it, be sure to instance it with something other than ‘p4′, I did this and it did not work, using ‘p4i’ it did without incident:
In 2007, my friend Jason gave an AutoDesk Masterclass entitled: Python Scripting for MotionBuilder Artists. It has been available online and I would like to mention it for anyone who is interested in Python and MotionBuilder.
Facial motion capture stabilization is basically where you isolate the movement of the face from the movement of the head. This sounds pretty simple, but it is actually a really difficult problem. In this post I will talk about the general process and give you an example facial stabilization python script.
Disclaimer: The script I have written here is loosely adapted from a MEL script in the book Mocap for Artists, and not something proprietary to Crytek. This is a great book for people of all experience levels, and has a chapter dedicated to facial mocap. Lastly, this script is not padded out or optimized.
To follow this you will need some facial mocap data, there is some freely downloadable here at www.mocap.lt. Grab the FBX file.
Stabilization markers
Get at least 3 markers on the actor that do not move when they move their face. These are called ’stabilization markers’ (STAB markers). You will use these markers to create a coordinate space for the head, so it is important that they not move. STAB markers are commonly found on the left and right temple, and nose bridge. Using a headband and creating virtual markers from multiple solid left/right markers works even better. Headbands move, it’s good to keep this in mind, above you see a special headrig used on Kong to create stable markers.
It is a good idea to write some tools to help you out here. At work I have written tools to parse a performance and tell me the most stable markers at any given time, if you have this data, you can also blend between them.
Load up the facial mocap file you have downloaded, it should look something like this:
In the data we have, you can delete the root, the headband markers, as well as 1-RTMPL, 1-LTMPL, and 1-MNOSE could all be considered STAB markers.
General Pipeline
As you can see, mocap data is just a bunch of translating points. So what we want to do is create a new coordinate system that has the motion of the head, and then use this to isolate the facial movement.
This will take some processing, and also an interactive user interface. You may have seen my tutorial on Creating Interactive MotionBuilder User Interface Tools. You should familiarize yourself with that because this will build on it. Below is the basic idea:
You create a library ‘myLib’ that you load into motionbuilder’s python environment. This is what does the heavy lifting, I say this because you don’t want to do things like send the position of every marker, every frame to your external app via telnet. I also load pyEuclid, a great vector library, because I didn’t feel like writing my own vector class. (MBuilder has no vector class)
Creating ‘myLib’
So we will now create our own library that sits inside MBuilder, this will essentially be a ‘toolkit’ that we communicate with from the outside. Your ‘myLib’ can be called anything, but this should be the place you store functions that do the real processing jobs, you will feed into to them from the outside UI later. The first thing you will need inside the MB python environment is something to cast FBVector3D types into pyEuclid. This is fairly simple:
#casts point3 strings to pyEuclid vectorsdef vec3(point3):
return Vector3(point3[0], point3[1], point3[2])#casts a pyEuclid vector to FBVector3ddef fbv(point3):
return FBVector3d(point3.x, point3.y, point3.z)
Next is something that will return an FBModelList of models from an array of names, this is important later when we want to feed in model lists from our external app:
#returns an array of models when given an array of model names#useful with external apps/telnetlib uidef modelsFromStrings(modelNames):
output = []for name in modelNames:
output.append(FBFindModelByName(name))return output
Now, if you were to take these snippets and save them as a file called myLib.py in your MBuilder directory tree (MotionBuilder75 Ext2\bin\x64\python\lib), you can load them into the MBuilder environment. (You should have also placed pyEuclid here)
It’s always good to mock-up code in telnet because, unlike the python console in MBuilder, it supports copy/paste etc..
In the image above, I get the position of a model in MBuilder, it returns as a FBVector3D, I then import myLib and pyEuclid and use our function above to ‘cast’ the FBVector3d to a pyEuclid vector. It can now be added, subtracted, multiplied, and more; all things that are not possible with the default MBuilder python tools. Our other function ‘fbv()‘ casts pyEuclid vectors back to FBVector3d, so that MBuilder can read them.
So we can now do vector math in motionbuilder! Next we will add some code to our ‘myLib’ that stabilizes the face.
Adding Stabilization-Specific Code to ‘myLib’
One thing we will need to do a lot is generate ‘virtual markers’ from the existing markers. To do this, we need a function that returns the average position of however many vectors (marker positions) it is fed.
#returns average position of an FBModelList as FBVector3ddef avgPos(models):
mLen = len(models)if mLen == 1:
return models[0].Translation
total = vec3(models[0].Translation)for i inrange(1, mLen):
total += vec3(models[i].Translation)
avgTranslation = total/mLen
return fbv(avgTranslation)
Here is an example of avgPos() in use:
Now onto the stabilization code:
#stabilizes face markers, input 4 FBModelList arrays, leaveOrig for leaving original markersdef stab(right,left,center,markers,leaveOrig):
pMatrix = FBMatrix()
lSystem=FBSystem()
lScene = lSystem.Scene
newMarkers = []def faceOrient():
lScene.Evaluate()
Rpos = vec3(avgPos(right))
Lpos = vec3(avgPos(left))
Cpos = vec3(avgPos(center))#build the coordinate system of the head
faceAttach.GetMatrix(pMatrix)
xVec = (Cpos - Rpos)
xVec = xVec.normalize()
zVec = ((Cpos - vec3(faceAttach.Translation)).normalize()).cross(xVec)
zVec = zVec.normalize()
yVec = xVec.cross(zVec)
yVec = yVec.normalize()
facePos = (Rpos + Lpos)/2
pMatrix[0] = xVec.x
pMatrix[1] = xVec.y
pMatrix[2] = xVec.z
pMatrix[4] = yVec.x
pMatrix[5] = yVec.y
pMatrix[6] = yVec.z
pMatrix[8] = zVec.x
pMatrix[9] = zVec.y
pMatrix[10] = zVec.z
pMatrix[12] = facePos.x
pMatrix[13] = facePos.y
pMatrix[14] = facePos.z
faceAttach.SetMatrix(pMatrix,FBModelTransformationMatrix.kModelTransformation,True)
lScene.Evaluate()#keys the translation and rotation of an animNodeListdef keyTransRot(animNodeList):
for lNode in animNodeList:
if(lNode.Name == 'Lcl Translation'):
lNode.KeyCandidate()if(lNode.Name == 'Lcl Rotation'):
lNode.KeyCandidate()
Rpos = vec3(avgPos(right))
Lpos = vec3(avgPos(left))
Cpos = vec3(avgPos(center))#create a null that will visualize the head coordsys, then position and orient it
faceAttach = FBModelNull("faceAttach")
faceAttach.Show = True
faceAttach.Translation = fbv((Rpos + Lpos)/2)
faceOrient()#create new set of stabilized nulls, non-destructive, this should be tied to 'leaveOrig' laterfor obj in markers:
new = FBModelNull(obj.Name + '_stab')
newTran = vec3(obj.Translation)new.Translation = fbv(newTran)new.Show = Truenew.Size = 20
new.Parent = faceAttach
newMarkers.append(new)
lPlayerControl = FBPlayerControl()
lPlayerControl.GotoStart()
FStart = int(lPlayerControl.ZoomWindowStart.GetFrame(True))
FStop = int(lPlayerControl.ZoomWindowStop.GetFrame(True))
animNodes = faceAttach.AnimationNode.Nodesfor frame inrange(FStart,FStop):
#build proper head coordsys
faceOrient()#update stabilized markers and key themfor m inrange(0,len(newMarkers)):
markerAnimNodes = newMarkers[m].AnimationNode.Nodes
newMarkers[m].SetVector(markers[m].Translation.Data)
lScene.Evaluate()
keyTransRot(markerAnimNodes)
keyTransRot(animNodes)
lPlayerControl.StepForward()
We feed our ‘stab‘function FBModelLists of right, left, and center stabilization markers, it creates virtual markers from these groups. Then ‘markers’ is all the markers to be stabilized. ‘leavrOrig’ is an option I usually add, this allows for non-destructive use, I have just made the fn leave original in this example, as I favor this, so this option does nothing, but you could add it. With the original markers left, you can immediately see if there was an error in your script. (new motion should match orig)
Creating an External UI that Uses ‘myLib’
Earlier I mentioned Creating Interactive MotionBuilder User Interface Tools, where I explain how to screenscrape/use the telnet Python Remote Server to create an interactive external UI that floats as a window in MotionBuilder itself. I also use the libraries mentioned in the above article.
The code for the facial stabilization UI I have created is here: [stab_ui.py]
I will now step through code snippets pertaining to our facial STAB tool:
This returns a list of strings that are the currently selected models in MBuilder. This is the main thing that our external UI does. The person needs to interactively choose the right, left, and center markers, then all the markers that will be stabilized.
At the left here you see what the UI looks like. To add some feedback to the buttons, you can make them change to reflect that the user has selected markers. We do so by changing the button text.
This also stores all the markers the user has chosen into the variable ‘rStabMarkers‘. Once we have all the markers the user has chosen, we need to send them to ‘myLib‘ in MBuilder so that it can run our ‘stab‘ function on them. This will happen when they click ‘Stabilize Markerset‘.
Above we now use ‘modelsFromStrings‘ to feed ‘myLib’ the names of selected models. When you run this on thousands of frames, it will actually hang for up to a minute or two while it does all the processing. I discuss optimizations below. Here is a video of what you should have when stabilization is complete:
Kill the keyframes on the root (faceAttach) to remove head motion
Conclusion: Debugging/Optimization
Remember: Your stabilization will only be as good as your STAB markers. It really pays off to create tools to check marker stability.
Sometimes the terminal/screen scraping runs into issues. The mbPipe function can be padded out a lot and made more robust, this here was just an example. If you look at the external python console, you can see exactly what mbPipe is sending to MBuilder, and what it is receiving back through the terminal:
Sending>>> selectedModels = FBModelList()
Sending>>> FBGetSelectedModels(selectedModels,None,True)
Sending>>> for item in selectedModels: print item.Name
['Subject 1-RH1', 'Subject 1-RTMPL']
All of the above can be padded out and optimized. For instance, you could try to do everything without a single lPlayerControl.StepForward() or lScene.Evaluate(), but this takes a lot of MotionBuilder/programming knowhow; it involves only using the keyframe data to generate your matrices, positions etc, and never querying a model.
I was talking to my friend Marco the other day. As he is a real programmer, he is somewhat equipped with the needed skills required to decode MotionBuilder’s procedurally-generated Python documentation. We were both frustrated, fighting with the ‘Python Console Tool’, when I showed him the telnet interface he was like “why don’t you just use that?”
And this is what I started doing. I now do much of my tests and work in the telnet console, because, unlike the built in console that Motion Builder offers, the telnet window at least offers copy/paste, and you can press the up arrow to cycle through previous arguments that you have entered. I would suggest using this until Autodesk adds usable features to their ‘Python Console Tool’.
I have been researching quick ways to output MotionBuilder data visually, which I might post about later (doing some matplotlib tests here at home). The following is probably a ‘no-brainer’ to people with a programming background, but I found it interesting. Below I am using simple hashes to graph values visually in the console.
data = [20, 15, 10, 7, 5, 4, 3, 2, 1, 1, 0]for i in data: print'#'* i
This is another post in my short quest to profile per-frame functions in Motion Builder. This script below uses hotshot, the built in python profiler, to profile not only a function I write, but built in MBuilder c++ modules (FBPlayerControl().StepForward()).
8
9
10
#wrap motionbuilder c module functions you want to profile in python functionsdef stepForward():
lPlayerControl.StepForward()
Above is a snippet of the script below. We are wrapping the StepForward() MBuilder call in a function called stepForward(). Hotshot can profile any python function, so we trick it by wrapping a function around the c module.
from pyfbsdk import*importhotshotimporthotshot.statsimportsys
lPlayerControl = FBPlayerControl()#wrap motionbuilder c module functions you want to profile in python functionsdef stepForward():
lPlayerControl.StepForward()#this is our function that we want to profiledef myFn():
FStart = int(lPlayerControl.ZoomWindowStart.GetFrame(True))
FStop = int(lPlayerControl.ZoomWindowStop.GetFrame(True))
FBPlayerControl().GotoStart()for frame inrange(FStart,FStop):
stepForward()#here we set up the hotshot profiler to create a profile file (binary)#by running our function (myFn)
prof = hotshot.Profile("c:\\myFn.prof")
prof.runcall(myFn)
prof.close()#now we load the profile stats
stats = hotshot.stats.load("c:\\myFn.prof")
stats.strip_dirs()
stats.sort_stats('time', 'calls')#and finally, we print the profile stats to the disk in a file 'myFn.log'
saveout = sys.stdout
fsock = open('c:\\myFn.log', 'w')sys.stdout = fsock
stats.print_stats(20)sys.stdout = saveout
fsock.close()
This is what the file we generated (myFn.log) looks like:
1305 function calls in 15411.020 CPU seconds
Ordered by: internal time, call count
ncalls tottime percall cumtime percall filename:lineno(function)
1304 15398.358 11.809 15398.358 11.809 speedtest.py:9(stepForward)
1 12.662 12.662 15411.020 15411.020 speedtest.py:13(myFn)
0 0.000 0.000 profile:0(profiler)
As you can see, FBPlayerControl().StepForward() is very resource intensive, and should rarely be used when crunching a lot of keyframe data. I hope this helps anyone in finding resource bottlenecks in large scripts, here’s an example of output from a larger script where I am trying to see how long scene.evaluate() and stepforward() are taking, and get rid of them by querying the keyframe data the way KxL showed before:
KxL on Autodesk’s ‘The Area‘ forum has really shed some light on to why MotionBuilder was so slow when iterating through frames. It all boils down to the usage of FBSystem().Scene.Evaluate(). This evaluates the entire scene and is very, very slow. Motionbuilder does not allow for getting a parameter at time i, or do partial scene evaluation.
So this basically means that any time you need to get the position, rotation, or matrix of a model, you should go to the keyfame data directly. (unless it’s a constraint!) Here is an example function that prints position every frame. It does so by getting the information from the fcurves themselves, and *not* querying the nodes.
#prints position given a model and a framedef PrintPosition(pModel,pFrame):
lFrameTime = FBTime(0,0,0,pFrame)
lAnimationNode = pModel.Translation.GetAnimationNode()
lPositionV = FBVector3d(
lAnimationNode.Nodes[0].FCurve.Evaluate(lFrameTime),
lAnimationNode.Nodes[1].FCurve.Evaluate(lFrameTime),
lAnimationNode.Nodes[2].FCurve.Evaluate(lFrameTime))print lPositionV
del(lFrameTime, lAnimationNode, lPositionV)
Here is another example, showing how to build a transformation matrix from animation data (nodes). This is required because the normal way of requesting the matrix of a model in MBuilder (model.GetMatrix(pMatrix)), requires an Evaluate(). The following requires importing the following libraries: [math].
Soon, I hope to build an example tool showing how to iterate through frames and do facial motioncapture stabilization. That was sort of how all this came up, I wrote the tool, but it was ~20 times slower than it should have been.
This is a copy of a tutorial stored in the Tutorial section of my site.
————————————————————————————
One of the biggest complaints about MotionBuilder, from a Tech Artist/Animator standpoint is it’s Python implementation. Everyone seems to agree: It is very hard to make user-friendly tools in MotionBuilder. Which is why, as you will see, this tutorial is about creating user-friendly tools outside of MotionBuilder.
Why Is Impossible to Create an Interactive UI in MotionBuilder? Is There Another Way?
Because of threading issues, any time you create a window or tool UI in MB, the whole app freezes until you close your window. This makes it very hard to create tools that people interact with and use. I heard from someone at Sony Pictures Imageworks that it is possible to create user-friendly UI, they did it there as a seperate Python app that ‘talks’ to MB through a telnet terminal.
So this is what I did. I first scoured the ‘internets’ and found little or no information on using the telnet ‘Remote Server’ Python implementation in MB, so I decided to create this tutorial.
Python/MotionBuilder Resources
You don’t need to be experienced with Python to follow this tutorial, this is actually the first Python script I have written. You do need two things to follow along: [Python] [wxPython]
Plug: Also, if you are serious about Python Scripting in MotionBuilder, You should grab Jason Parks’ Autodesk Master Class: Python Scripting for MotionBuilder Artists, not only because he’s a friend of mine, but also because it’s an invaluable rescource (the only resource) when just starting out.
Getting Started: Remote Server and Telnetlib
Open up MB and select Window>Python Console Tool, under the ‘Configuration’ tab, you should see the following:
The checkboxes ‘Enable‘ and ‘Allow Network Connections‘ are usually off by default. Turn them both on. Now there is a telnet server running on port 4242 of your PC that can accept Python commands and execute them in MB. If you press ‘Telnet Console‘ a terminal will pop open and connect. Entering commands into this terminal is like entering them into the MB Python Console.
Now open IDLE, the Python GUI/Script editor. Create a new script as follows, my comments in green:
#Import the Python telnet libraryimporttelnetlib#Create a telnet console to your PC (127.0.0.1) and MotionBuilder (4242)
host = telnetlib.Telnet('127.0.0.1', 4242)#Find the prompt, so you can enter a command (seriously)
host.read_until('>>>', 5)#Create a null called 'locator'
host.write('myNull = FBModelNull("locator")\n')#Close the terminal
host.close()
If you alt+tab back to MB, you should see a null named ‘locator’ in the Schematic View. What we are basically doing is creating an invisible terminal window, like the one we saw when we clicked ‘Telnet Console‘. We then want to send the command myNull = FBModelNull(“locator”) over, but in order to do so, we first need to find the prompt. ‘\n‘ means ‘new line’, so we are essentially entering a command at the prompt and pressing [ENTER].
Communication Issues: Writing a Function to Bridge the Gap
First, you do not want to have to do this stuff every time you need to enter a Python command. Second, you need a way of getting data that is returned as a result of your command/query. This is what a simple function to do that looks like in Python:
def mbPipe(command):
host.read_until('>>>', .01)#write the command
host.write(command + '\n')#read all data returned
raw = str(host.read_until('>>>', .1))#removing garbage i don't want
raw = raw.replace('\n\r>>>','')
raw = raw.replace('\r','')#make an array, 1 element per line returned
rawArr = raw.split('\n')return rawArr
So now we have a function that makes it easier for us to communicate with MB through telnetlib. Here is an example of it’s use:
mbPipe('selectedModels = FBModelList()')
mbPipe('FBGetSelectedModels(selectedModels,None,True)')
mbPipe('for item in selectedModels: print item.Name')
The code above returned the names of the currently selected items in MB as an array. (['Null', 'Null 1', 'Null 2', 'locator', 'Test']) Because this returns an array, it is easy for us to write something a little more complicated like this:
for item in(mbPipe("for item in selectedModels: print item.Name")):
self.selListBox.Append(item)
This adds all items selected in MB to a listbox. Now that you have an
understanding of how to communicate with MotionBuilder through the
Remote Server and an external Python process via telnetlib, let’s now
build a tool to take advantage of this.
Practical Example of Interactive, External, MotionBuilder UI
I’m not really going to walk you through this one. It’s basically some modified wxPython example code from the internets bent to my bidding. You will see our mbPipe function and some things from above.
I decided to use wxPython because it’s a very simple and efficient UI library for Python that allows you to quickly create decent custom interfaces. You can of course use anything other tools to build a UI.
To the left here is a little tool. When you press ‘Get Selected Items‘, the listbox will fill with the ‘models’ currently selected in MB. As you click on items in the listbox, the position/rotation text at the bottom will change reflecting the position and rotation of the ‘models’ in MB. Pressing “Delete This Model‘ will delete the object in MB.
import wx
importtelnetlib
host = telnetlib.Telnet("127.0.0.1", 4242)def mbPipe(command):
host.read_until('>>>', .01)#write the command
host.write(command + '\n')#read all data returned
raw = str(host.read_until('>>>', .1))#removing garbage i don't want
raw = raw.replace('\n\r>>>','')
raw = raw.replace('\r','')
rawArr = raw.split('\n')#cleanArr = [item.replace('\r', '') for item in rawArr]return rawArr
class MyFrame(wx.Frame):
def__init__(self):
# create a frame, no parent, default to wxID_ANY
wx.Frame.__init__(self, None, wx.ID_ANY, 'mbUI test',pos=(200, 150),size=(175,280))#create listboxself.selListBox = wx.ListBox(self, -1, choices=[],pos=(8,38),size=(150, 145))self.selListBox.Bind(wx.EVT_LISTBOX, self.selListBoxChange)#create model metadataself.translationLabel = wx.StaticText(self,-1,'Pos:',pos=(8,185))self.rotationLabel = wx.StaticText(self,-1,'Rot:',pos=(8,200))#make buttonsself.selItems = wx.Button(self, id=-1,label='Get Selected Items',pos=(8, 8),size=(150, 28))self.selItems.Bind(wx.EVT_BUTTON, self.selItemsClick)self.selItems.SetToolTip(wx.ToolTip("Gets all 'models' selected in Motion Builder"))self.delItems = wx.Button(self, id=-1,label='Delete This Model',pos=(8, 217),size=(150, 28))self.delItems.Bind(wx.EVT_BUTTON, self.delItemsClick)self.delItems.SetToolTip(wx.ToolTip("Will delete selected models"))# show the frameself.Show(True)def selItemsClick(self,event):
mbPipe("selectedModels = FBModelList()")
mbPipe("FBGetSelectedModels(selectedModels,None,True)")self.selListBox.Set([])for item in(mbPipe("for item in selectedModels: print item.Name")):
self.selListBox.Append(item)def delItemsClick(self,event):
mbPipe('FBFindModelByName("' + str(self.selListBox.GetStringSelection()) + '").FBDelete()')
mbPipe("selectedModels = FBModelList()")
mbPipe("FBGetSelectedModels(selectedModels,None,True)")self.selListBox.Set([])for item in(mbPipe("for item in selectedModels: print item.Name")):
self.selListBox.Append(item)def selListBoxChange(self,event):
trans = mbPipe('FBFindModelByName("' + str(self.selListBox.GetStringSelection()) + '").Translation')
rot = mbPipe('FBFindModelByName("' + str(self.selListBox.GetStringSelection()) + '").Rotation')self.translationLabel.SetLabel(('Pos: ' + trans[0]))self.rotationLabel.SetLabel(('Rot: ' + rot[0]))
application = wx.PySimpleApp()
window = MyFrame()
application.MainLoop()
Conclusion: This Sucks, but it Makes You A Better Person
Sure, this is a pretty unconventional way to have to create tools, but suffering this will make you a better Technical Artist/Animator. Not to mention you can now create great, interactive tools, without resorting to C++ and the Open Reality SDK.
Sync Issues?: It seems that sometimes MB just closes the connection, or perhaps telnetlib does. Either way, if your tool becomes unresponsive, the trick I found is to close the tool, then uncheck/recheck ‘Allow Network Connections‘ in the ‘Python Console Tool‘ Configuration tab shown above. Then re-run your script.
PyWin32: There is another Python library called Python Win32 Extentions, that allows you to get your hands dirty with COM and things. You can use some of these to make it so that your tool shows up as a child window of the MB app itself, this way users don’t even have to press alt+tab to use your app and it looks like it belongs in MBuilder. Of course there is no mention of this on teh internets, but I found an example for 3DSMax/Python in Adam Pletcher’s GDC talk: Python for Technical Artists, and a Maya/Python example on cgTalk. This is somewhat more difficult; using the above sources I have written a MotionBuilder example of this [here].
This is a follow-up to my post about per-frame operations in MotionBuilder. For some reason they take forever. I will show you an example, and compare the MB Python implementation to other scripting implementations by executing the same code in another Autodesk product (3dsMax).
Time Stamping
First, lets find a way to see how long it takes to execute a block of code. In the example below, the script will
from pyfbsdk import*importtime#get the time
t1 = time.time()#get selected items
selectedModels = FBModelList()
FBGetSelectedModels(selectedModels,None,True)#print each items positionfor item in selectedModels: print item.Translation#tell us how long that took (time start - time finish)print('Process took ' + str((time.time() - t1)) + ' seconds')
I can run this on some selected models and it reports: Process took 0.0 seconds. Now if you run this on, say, 60 markers, the output window has no scrollbar, and unless you have a large screen, you will not see the last output telling you how long the process took. So we can replace the print with a messagebox:
Let’s take our example from before, where we go through every frame and print the position of an object in MB. Alter that code by inserting the code from above. Add the ‘import time‘ at the top, and then set t1 before you run posEveryFrame(selectedModels[0]). Now after that function, add your line that generates a messagebox.
When I run this on a single object across 1369 frames I get the messagebox above: it takes almost 10 seconds! Remember, we are just printing a position of an object here. I wrote the same code in Maxscript and it took 0.703 seconds. This is 13 times faster than MotionBuilder, a program that prides itself on dealing with keyframe data!
I think this is because MotionBuilder is actually moving the timeslider itself, even though the UI is not updating! This must be the case, because it is just unbelievably slow. So next we will research whether or not we can grab scene info at a certain time.
In any animation package, there are many times technical artists/animators will need to iterate through all the frames of animation and do something. This is a given.
Accessing The Timeline
To access timeline functionality in MBuilder you can either get the current take and then request it’s start/stop or use the following, which grabs the start and stop in the current take/timeline:
However, this only returns a frame number in the range of your FPS, for instance, if your end frame is 1369, it will return the value 19L. To better understand this, enter 1369%30 into your Python Console, it will return 19. More info on modulo operations [here]
So how do we get time from MotionBuilder in a usable format? Here are two examples:
#this retuns the real frame number
FBPlayerControl().ZoomWindowStop.GetFrame(True)#this returns in seconds</span></code>
FBPlayerControl().ZoomWindowStop.GetSecondDouble()
Iterating Through the Timeline
Now that we can get the range of the frames in the timeline, let’s do stuff! Here is an example you can execute in MB showing how to iterate through time and do something for every frame (print frame number):
from pyfbsdk import*
lSystem=FBSystem()
lScene = lSystem.Scene
playaCtrl = FBPlayerControl()#this returns the start/stop as integers<code>
fStart = int(playaCtrl.ZoomWindowStart.GetFrame(True))
fStop = int(playaCtrl.ZoomWindowStop.GetFrame(True))for frame inrange(fStart,fStop): print frame
Now, this is not to be confused with doing something on every frame. Above, we are just iterating through the frame numbers themselves. If there are 1000 frames, you are just going from number 1 to 1000 printing the current number.
So now let’s iterate through frames and do something every frame.
from pyfbsdk import*def posEveryFrame(model):
lSystem=FBSystem()
lScene = lSystem.Scene#we instance the player control to read from it faster
playaCtrl = FBPlayerControl()#set our start and stop (as integers)
fStart = int(playaCtrl.ZoomWindowStart.GetFrame(True))
fStop = int(playaCtrl.ZoomWindowStop.GetFrame(True))#tell player control to go to the start of the scene
playaCtrl.GotoStart()#now iterate through the framesfor frame inrange(fStart,fStop):
#evaluate the scene to make sure the data is updated
lScene.Evaluate()#output the models positionprint model.Translation#tell the player to go to the next frame
playaCtrl.StepForward()
selectedModels = FBModelList()
FBGetSelectedModels(selectedModels,None,True)#run our function on the first model in the selection
posEveryFrame(selectedModels[0])
So, executing this with a model selected should print the models translation (position) every frame for all frames in your timeslider (transport control).
Now you can start messing around with iterating through frames, and you will immediately notice two things:
1) Iterating through frames in MotionBuilder is slow. Almost slower than my grandmother entering all the values into a calculator (not even one with large numbered buttons)
2) The Python Console that you print to does not have a scroll bar, and does not have selectable text. (complete and utter fail)