I can finally talk about something I have been working on in the past two years.  One of the reasons I returned to Crytek was to push the use of game engines in linear content creation like film and television. On Avatar I saw how much time and effort went into layout, blocking, virtual sets, etc. The tools were archaic, the feedback loop was abysmal at times. In games we have to layout massive levels that people can roam through for 8-15 hours or more and CryEngine’s tools are some of the best for that.

I have been working as Product Manager with a small team of great guys, where I basically define the goals and backlog. It’s thrilling to finally get to see things like Catmull-Clark subd in runtime, or multi-channel EXR output, or Alembic support. It’s been really fun to define what the product is and prioritize features largely without external dependencies or politics, I thank Crytek for trusting me to helm such a project.

We had a live demo kiosk at GDC; check out the Cinema Sandbox Website for more info.

I will be speaking at FMX about CineBox and the whole idea of using game engines for previs and virtual production: The Long Road to Film / Game Convergence

I am still finding my feet in Maya, on my project, some files have grown to 800mb in size. Things get corrupt, hand editing MAs is common; I am really learning some of the internals.

In the past week I have had to do a lot of timeline walking to switch coord spaces and get baked animations into and out of hierarchies. In 3dsMax you can do a loop and evaluate a node ‘at time i’, and there is no redraw or anything. I didn’t know how to do this in Maya.

I previously did this with looping cmds.currentTime(i) and ‘walking the timeline’, however, you can set the time node directly like so: cmds.setAttr(“time1.outTime”, int(i))

Unparenting a child with keyed compensation (1200 frames)
10.0299999714 sec – currentTime
2.02 sec – setAttr

There are some caveats, whereas in a currentTime loop you can just cmds.setKeyframe(node), I now have to cmds.setKeyframe(node, time=i). But when grabbing a matrix, I don’t need to pass time and it works, I don’t think you can anyway.. I guess it gets time from the time node.

Here’s a sample loop that makes a locator and copies a nodes animation to world space:

#function feeds in start, end, node
	if not start: start = cmds.playbackOptions(minTime=1, q=1)
	if not end: end = cmds.playbackOptions(maxTime=1, q=1)
	loc = cmds.spaceLocator(name='parentAlignHelper')
	for i in range(start, (end+1)):
		cmds.setAttr("time1.outTime", int(i))
		matrix = cmds.xform(node, q=1, ws=1, m=1)
		cmds.xform(loc, ws=1, m=matrix)
		cmds.setKeyframe(loc, time=i)

Maybe it’s me, but I often find myself parsing weird ascii text files from others. Sometimes the authors knew what the data was and there’s no real markup. Take this joint list for example:

143 # bones
root ground
-1
0 0 0
root hips
0
0 0.9512207 6E-08
spine 1
1
4E-08 0.9522207 1.4E-07
spine 2
2
3E-07 1.0324 8.3E-07
spine 3
3
5.6E-07 1.11357 1.53E-06
spine 4
4
8.2E-07 1.194749 2.22E-06
head neck lower

So the first line is the number of joints then it begins in three line intervals stating from the root outwards: joint name, parent integer, position. I used to make a pretty obtuse loop using a modulus operator. Basically, modulus is the remainder left over after division. So X%Y gives you the remainder of X divided by Y; here’s an example:

for i in range(0,20+1):
	if i%2 == 0: print i
#>> 0
#>> 2
#>> 4
#>> 6
#>> 8
#>> 10

The smart guys out there see where this is goin.. so I never knew range had a ’step’ argument. (Or I believe I did, I think I actually had this epiphany maybe two years ago, but my memory is that bad.) So parsing the above is as simple as this:

jnts = []
for i in range(1,numJnts*3+1,3):
	jnt = lines[i].strip()
	parent = int(lines[i+1].strip())
	posSplit = lines[i+2].strip().split(' ')
	pos = (float(posSplit[0])*jointScale, \
	float(posSplit[1])*jointScale, float(posSplit[2])*jointScale)
	jnts.append([jnt, parent, pos])

Thanks to phuuchai on #python (efnet) for nudging me to RTFM!

I gave a workshop/talk at SIGGRAPH geared toward introducing people to Python. There were ~25 people on PCs following along, and awkwardly enough, many more than that standing and watching. I prefaced my talk with the fact that I am self-taught and by no means an expert. That said, I have created many python tools people use every day at industry-leading companies.

Starting from zero, in the next hour I aimed to not only introduce them to Python, but get them doing cool, usable things like:

• Iterating through batches/lists
• Reading / writing data to excel files
• Wrangling data from one format to another in order to create a ‘tag cloud’

Many people have asked for the notes, and I only had rough notes. I love Python, and I work with this stuff every day, so I have had to really go back and flesh out some of what I talked about. This tutorial has a lot less of the general chit-chat and information. I apologize for that.

Installation / Environment Check

So Python is correctly installed, for the following you can either follow along in the cmd window (more difficult) or in IDLE, the IDE that python ships with (easier). This can be found by typing IDLE into the start menu:

Variables

Strings

Strings are pieces of text. I assume you know that, so let’s just go over some quick things:

string = 'this is a string'
print string
#>>this is a string
num = '3.1415'
print num
#>>3.1415

One thing to keep in mind, the above is a string, not a number. You can see this by:

print num + 2
#>>Traceback (most recent call last):
#>>  File "basics_variables.py", line 5, in
#>>    print num + 2
#>>TypeError: cannot concatenate 'str' and 'int' objects

Python is telling you that you cannot add a number to a string of text. It does not know that ‘3.1415′ is a number. So let’s convert it to a number, this is called ‘casting’, we will ‘cast’ the string into a float and back:

print float(num) + 2
#>>5.1415
print str(float(num) + 2) + ' addme'
#>>5.1415 addme

Lists

Lists are the simplest ways to store pieces of data. Let’s make one by breaking up a string:

txt = 'jan tony senta michael brendon phillip jonathon mark'
names = txt.split(' ')
print names
#>>['jan', 'tony', 'senta', 'michael', 'brendon', 'phillip', 'jonathon', 'mark']
for item in names: print item
#>>jan
#>>tony
#>>senta
#>>michael
...

Split breaks up a string into pieces. You tell it what to break on, above, I told it to break on spaces txt.split(‘ ‘). So all the people are stored in a List, which is like an Array or Collection in some other languages.
You can call up the item by it’s number starting with zero:

print names[0], names[5]
#>>jan phillip

TIP: [-1] index will return the last item in an array, here’s a quick way to get a file from a path:

path = 'D:\\data\\dx11_PC_(110)_05_09\\Tools\\CryMaxInstaller.exe'
print path.split('\\')[-1]
#>>CryMaxInstaller.exe

Dictionaries

These store keys, and the keys reference different values. Let’s make one:

dict = {'sascha':'tech artist', 'harry': 142.1, 'sean':False}
print dict['sean']
#>>False

So this is good, but these are just the keys, we need to know the values. Here’s another way to do this, using .keys()

dict = {'sascha':'tech artist', 'harry': 142.1, 'sean':False}
for key in dict.keys(): print key, 'is', dict[key]
#>>sean is False
#>>sascha is tech artist
#>>harry is 142.1

So, dictionaries are a good way to store simple relationships of key and value pairs. In case you hadn’t notices, I used some ‘floats’ and ‘ints’ above. A float is a number with a decimal, like 3.1415, and an ‘int’ is a whole number like 10.

Creating Methods (Functions)

Let’s say that you have to do something many times, you want to re-use this code and not copy/paste it all over. Let’s use the example above of names, let’s make a function that takes a big string of names and returns an ordered list:

def myFunc(input):
	people = input.split(' ')
	people = sorted(people)
	return people
txt = 'jan tony senta michael brendon phillip jonathon mark'
orderedList = myFunc(txt)
print orderedList
#>>['brendon', 'jan', 'jonathon', 'mark', 'michael', 'phillip', 'senta', 'tony']

Basic Example: Create A Tag Cloud From an Excel Document

First, download the excel sheet from me here: [info.csv] The CSV filetype is a great way to read/write docs easily that you can give to others, they load in excel easily.

file = 'C:\\Users\\chris\\Desktop\\intro_to_python\\info.csv'
f = open(file, 'r')
lines = f.readlines()
f.close()
print lines
#>> ['always late to work,13\n', 'does not respect others,1\n', 'does not check work properly,5\n', 'does not plan properly,4\n', 'ignores standards/conventions,3\n']

‘\n’ is a line break character, it means ‘new line’, we want to get rid of that, we also want to just store the items, and how many times they were listed.

file = 'C:\\Users\\chris\\Desktop\\intro_to_python\\info.csv'
f = open(file, 'r')
lines = f.readlines()
f.close()
dict = {}
for line in lines:
	split = line.strip().replace(' ','~').split(',')
	dict[split[0]] = int(split[1])
print dict
#>>{'ignores~standards/conventions': 3, 'does~not~respect~others': 1, 'does~not~plan~properly': 4, 'does~not~check~work~properly': 5, 'always~late~to~work': 13}

Now we have the data in memory in an easily readable way, let’s write it out to disk.

output = ''
for key in dict.keys():
	for i in range(0,dict[key]): output += (key + '\n')
f = open('C:\\Users\\chris\\Desktop\\intro_to_python\\test.txt', 'w')
f.write(output)
f.close()

There we go. In one hour you have learned to:

• Read and write excel files
• Iterate over data
• Convert data sets into new formats
• Write, read and alter ascii files

If you have any questions, or I left out any parts of the presentation you liked, reply here and I will get back to you.

As many of you know, I feel the whole ‘autorigging’ schtick is a bit overrated. Though Bungie gave a great talk at GDC09 (Modular Procedural Rigging), Dice was to give one this year at SIGGRAPH (Modular Rigging in Battlefield 3), but never showed up for the talk.

At Crytek we are switching our animation dept from 3dsMax to Maya. This forces us to build a pipeline there from scratch; in 3dsMax we had 7 years of script development focused on animation and rigging tools. So I am looking at quite a bit o Maya work. The past two weeks focusing on a ‘rigging system’ that I guess could be thought of as ‘procedural’ but is not really an ‘autorigger’. My past experience was always regenerating rigs with mel cmds.

Things I would like to solve:

• Use one set of animator tools for many rigs – common interfaces, rig block encapsulation (oh god i said ‘block’)
• Abstract things away, thinking of rigging ‘units’ and character ‘parts’ instead of individual rig elements, break reliance on naming, version out different parts
• Be fluid enough to regenerate the ‘rigging’ at any time

First Weekend: Skeleton ‘Tagging’

I created a wrapper around the common rigging tools that I used, this way, when I rigged, it would automagically markup the skeleton/elements as I went. This looked like so:

The foundation of this was marking up the skeleton or cons with message nodes that pointed to things or held metadata.  This was cool, and I still like how simple it was, however, it didn’t really create the layer of abstraction I was after. There wasn’t the idea of a limb that I could tell to switch from FK to IK.

Second Weekend: Custom Nodes

That Bungie talk got a lot of us all excited, Roman went and created a really cool custom node plugin that does way more than we spec’d it out to do. I rewrote the rigging tools to create ‘rigPart’ nodes, which could be like an IK chain, set of twist joints, expression, or constraint. These together could form a ‘charPart’ like an arm or leg. All these nodes sat under a main ‘character’ node. I realize that many companies abstract their characters into ‘blocks’ or ‘parts’, but I had never seen a system that had another layer underneath that. Roman also whipped up a way that when an attr on a customNode changes, you could evaluate a script. So whether it’s a human arm or alien tentacle arm, the ‘charPart’ node can have one FK/IK enum. I am still not sure if this is a better idea, because of the sheer legwork involved..

Third Weekend: A Mix of Both?

So a class like ‘charParts.gruntLeg()’ not only knew how to build the leg rigParts, but also only the leg ‘rigging’ if needed. This works pretty well, but the above was pretty hard to read. I took some of my favorite things about the tree-view-based system and I created a ‘character’ outliner of sorts. This made it much easier to visualize the rigParts that made up individual ’systems’ of the character, like leg, spine, arm, etc. I did it as a test, but in a way that I easily swap it out with the treeWidget in the rigging tools dialog.

So how do you guys solve some of these issues?

Pythons have probably my favorite skull of any animal. The reticulated python has a jaw that is in four movable parts, and the lower two can swing open over 120 degrees. Here are the main parts of a reticulated python skull:

Via SkullsUnlimited.com

Four, independently moving jaw bones, in case you thought Predator or the Covenant were original: Mother Nature has had them beat for a while!  –And look at the angle of those teeth: nothing that goes into that mouth is ever coming out! Here’s a video of a guy getting bitten and his friends have to push the skull forward very hard, then open the mouth, and then pull it away. With these four independent jaw parts, they have the ability to really get their mouths around prey that is much larger than their bodies, like this African antelope:

I marked the mouth in red above

As if four independently moving jaw parts wasn’t cool enough: they have a second row of teeth! These are situated on the roof of their mouth, yes, you read that right, reticulated pythons have palatine teeth, circled below:

This is a central row of teeth behind the maxillary teeth on the upper jaw! Here are some better pics to make it harder for you to get to sleep at night:

Here’s a link to a 3d Burmese Python skull 360 render (roll) from the DigiMorph website.

The Nikon 400mm 2.8 has a lens hood that costs $400. For this price, you would think they use pretty solid parts, but there is a block that a thumbscrew goes into that’s actually hard plastic. If your lens is in a backpack with the hood attached, this retaining screw will strip.

I have read many forum posts where people begrudgingly REPLACED THE ENTIRE HOOD because of this. People said they contacted Nikon directly and were told there are no replacement parts.

After contacting Nikon, I would like anyone googling for a solution to know that you can order the parts from Nikon, the parts and numbers are pictured above. Contact the Nikon Parts Service at: 310-414-5121

I am volunteering again in the Studio; giving three small talks at SIGGRAPH, drop me a line if you will be in Vancouver.

Rigging Characters for CryENGINE

How to rig, skin, and export a character for CryENGINE 3. Topics include physics setup, building characters from many skinned meshes, and creating Character Definitions and Character Parameter files. These rigging basics are applicable to most run-time game engines.

Introduction to Python Scripting

In this introduction to Python, a powerful scripting language used by many 3D applications, attendees learn the basics and explore small example scenarios gleaned from actual game and film productions. The sessions are taught in a way that should empower attendees to immediately begin creating time-saving python scripts and applications.

World Creation in CryENGINE

Have you ever wanted to make a videogame? This session shows how to build a small level in the freely available CryENGINE 3 SDK. Topics include: world building and tools (FlowGraph, CryENGINE’s visual scripting language, and Trackview, the camera sequencing and directing tools). In less than an hour, attendees create their own playable video games.

GoPro Cameras

GoPro makes small $250 no-frills video cameras that record 1080p and come in waterproof polycarbonate housings rated to 60m depth. They have a 170º angle of view, glass lens, fixed focus (2.5ft – ∞), f/2.8 aperture, and 2.5 hour battery life. These cameras are ‘bare bones’; there is no way to even know if it’s recording but to look directly into the lens, no backfacing LCD or even blinking LED!

Not Usable Under Water! – I did some test dives as soon as I received the housing and was in for a rude awakening. The glass domed ports blur the image underwater. This is because domed ports create a secondary focal point or ‘virtual image’ underwater that must be focused on. It seems that GoPro did not take this into account; after contacting them directly I was told: “It is not possible at this time for the GoPro Hero to focus in an underwater environment.” One funny thing to point out, the cool underwater videos on GoPro’s own site are not shot with their own lens/housing!

SOLUTION: 3rd Party Lenses or Housing – That’s right, to use your GoPro Hero underwater you have to buy a replacement housing from a 3rd party with flat ports (crisp images underwater). At the time of my writing this, there were none available for the 3d Hero System, so I purchased replacement lenses from Pursuit Diving. These lenses are very soft polycarbonate, and you might want to carry some Novus 2 polish with you as they scratch easily [image]. Mine also had some small areas of blurriness: this is not an ideal solution. Eye of Mine has a complete 3d housing replacement in the works, and GoPro themselves say they are ‘working’ on a solution. Either way, be warned: These cameras are unable to produce decent images underwater!

Poor Dynamic Range / Image Quality - As you see below, bright highlights easily get blown out. They claim the 1/2.5″ CMOS sensor is great for low light (>1.4 V/lux-sec), this may be, but it is woefully bad at images that vary in bright and dark.

Highlights are easily blown out, and create bad image artifacts

The H264 (12mbit) really butchers the image at times (PNG)

Rolling Shutter Artifacts (Wobble or Jelly) – Like most CMOS video cameras, the GoPro has some rolling shutter issues; I would say more than other CMOS cameras I have used. Unfortunately, for a camera that is meant to be strapped to moving objects –this is pretty bad! You have no control over the shutter speed, so unfortunately the less light, the more rolling shutter artifacts. Here’s an example looking out my window, but you can also see this in the Thistlegorm wreck footage below.

There is a great free solution for VirtualDub called DeShaker. For the HD Hero you should enter a rolling shutter amount of 82%.

Poor Battery Retention – On more than one occasion I left full batteries in the camera and did not turn it on for one to two days. I was often surprised to find the batteries low or half-drained. I have many other smaller canon, fuji, etc cameras and they have much better retention.

3D Hero System

Before, if you wanted to make a GoPro s3d rig, you had to put both cameras on a plate, then clank it to later sync the videos by audio waveform. Not only that, but the cameras dropped frames, so you had to time warp the footage to take into account drift: It was less than ideal.

In March (2011) GoPro released the ‘3d Hero System’ which is a new housing and a sync cable for two existing cameras. They also purchased CineForm Studio and skinned the software to make for a slightly less painful s3d workflow; unless you know what you’re doing, then their CineForm app can be pretty obnoxious/unintuitive.

Somewhat Buggy / Unreliable
I was surprised by the bugs I encountered. It’s very frustrating to think you are recording something and later realize that the camera rig left you with unusable data. For instance, I thought the cameras were recording for an entire dive, but it turned out that they somehow entered a state where they made 400+ individual one second MP4 files. Other times one camera would turn off, or unsync and begin recording in a separate format (like one eye 1080p, the other 5mp stills). Many times one battery would run out well before the other, in which case you at least have 2d video, but still annoying.

Sync Cable does not ‘Sync’ Cameras
The ‘Sync Cable’ does not really ’sync’ the cameras; treating it as a sync cable will only lead to complete frustration. This can really cause some issues, you need to think of the cable as a ‘controller cable’, where one camera is the master and the other a slave, or you will end up with only one camera recording. Again, the functions of the cameras are not sync’d! The Camera with the sync cable marked ‘R’ must start/stop the recording for both cameras to record. It is easy to place the cameras in the housing so that the ’slave’ camera shutter button is pressed, this does not work, so be careful!

Here is a schematic of the ’sync cable’ for DIY people.

Sync’d Recording Not Perfect (Don’t think ‘Genlock’)
While better than clanging a metal bar and timewarping to the audio, the sync cable doesn’t really sync the sensors. The videos seem a full frame off, so maybe the CineForm software compensates for this.

As you can see, the right (master) camera is a frame or so ahead of the left

Camera Settings Like White Balance and Exposure NOT Sync’d
Many times I find myself with stereo video where each eye is widely different. Whether it’s exposure, white balance, etc.. it’s frustrating, and the included CineForm software doesn’t offer much of a solution.

CineForm Software is Slow, Can be Frustrating
An example of some frustration advanced users may have is: “EXPORT TO MP4″: just a big button, nothing about datarate or other export options.. just a button. Unfortunately the UI has been dumbed down to the point of ambiguity and frustration. They should have continued and just made a “UPLOAD TO YOUTUBE 3D” because the software is dumbed down to the point of not being useful to advanced users, but not being easy enough for novices.

Fixed Interocular
The interocular of the housing is ~3.5cm which is a bit too close for my liking. Reducing the interocular to something smaller than the distance between your eyes causes the 3d effect to be weakened and things to appear larger than they really are. The interocular was decent for underwater, and I guess if you are filming yourself on a surfboard, but not great for driving through the Serengeti. The sync cable is of fixed length, so you cannot use it with other GoPro housings.

Unable to Use Other Attachments
Because the sync cable uses the expansion port, and the housing doesn’t accomodate, you cannot use the LCD backpac or the larger battery with the 3D Hero System.

Final Thoughts and Some Footage!

Sure I pointed out a lot of issues that I had, but for the price, the GoPro system is pretty great. The housing, though cheap, never flooded (many 30m dives). This is the first footage I have posted, and I have not post-processed it much. I will maybe make another post once I figure out the best ways to automatically post-process the footage to remove artifacts and distortion.

I recently wrote a custom tool to diff CryEngine layer files in P4, and was surprised how simple it was. What follows is a quick tutorial on adding custom python tools to Perforce.

Start by heading over to Tools>Manage Custom Tools… Then click ‘New’:

You can pass a lot of information to an external tool, here is a detailed rundown. As you see above, we pass the client spec (local) file name (%f) to a python script, let’s create a new script called ‘custom_tool.py’:

import sys
from PyQt4 import QtGui    
 
class custom_tool(QtGui.QMessageBox):
	def __init__(self, parent=None):
		QtGui.QMessageBox.__init__(self)
		self.setDetailedText(str(sys.argv))
		self.show()
 
if __name__ == "__main__":
	app = QtGui.QApplication(sys.argv)
	theTool = custom_tool()
	theTool.show()
	sys.exit(app.exec_())

What this does is simply spits out the sys.argv in a way you can see it. So now you can feed any file you right click in Perforce into a python script:

If you would like to actually do something with a file or revision on the server and are passing the %F flag to get the depot file path, you then need to use p4 print to redirect the file contents (non-binary) to a local file:

p4.run_print('-q', '-o', depotFile, localFile)

So last time I more introduced you to the idea of triggers, here’s a more complex example. This worked on my db, but if you have branching you would need to check each returned file against the branch you are submitting to.

Check if The File is Already in the Depot

This is a trigger that checks the hash digest of the incoming file against that of the server. This way you can see if the user is checking in a file that already exists.

import sys
from P4 import P4, P4Exception
 
p4 = P4()
describe = []
try:
	p4.user = "admin"
	p4.password = "admin"
	p4.port = "1666"
	p4.connect()
	lst = sys.argv[2]
	stat =  p4.run('fstat', ('-Ol','//depot/...@'+ str(lst)))
	hash = stat[0]['digest']
	fname = stat[0]['depotFile']
	m =  p4.run('fstat', ('-Ol','-F', ('digest = ' + str(hash)),'//depot/...'))
	existing = []
	for file in m:
		if file['depotFile'] != fname: existing.append(file)
 
	if existing != []:
		print '\n\nFILE EXISTS IN DEPOT!!'
		print  'YOUR FILE:  ' + (fname.split('/')[-1])
		for  file in existing: print 'EXACTLY MATCHES:  ' + file['depotFile']
		print 'P4 DIGEST:  ' + hash
		print 'SOLUTION: Contact your lead if you believe this message was generated in error.'
		sys.exit(1)
 
except Exception, e:
	print "Error: %s" % (e)
	sys.exit(1)
 
p4.disconnect()
sys.exit(0)

Then your trigger line looks like this:

Triggers:
	dupeCheck change-submit //depot/... "python X:/projects/2010/p4/dupe_trigger.py %user% %changelist%"

This is what the user will see when they try to check in:

Perforce is a wily beast. A lot of companies use it, but I feel few outside of the IT department really have to deal with it much. As I work myself deeper and deeper into the damp hole that is asset validation, I have really been writing a lot of python to deal with certain issues; but always scripts that work from the outside.

Perforce has a system that allows you to write scripts that are run, server side, when any number of events are triggered. You can use many scripting languages, but I will only touch on Python.

Test Environment

To follow along here, you should set up a test environment. Perforce is freely downloadable, and free to use with 2 users. Of course you are going to need python, and p4python. So get your server running and add two users, a user and an administrator.

Your First Trigger

Let’s create the simplest python script. It will be a submit trigger that says ‘Hello World’ then passes or fails. If it passes, the item will be checked in to perforce, if it fails, it will not. exiting while returning a ‘1′ is considered a fail, ‘0′ a pass.

print 'Hello World!'
print 'No checkin for you!'
sys.exit(1)

Ok, so save this file as hello_trigger.py. Now go to a command line and enter ‘p4 triggers’ this will open a text document, edit that document to point to your trigger, like so (but point to the location of your script on disk):

Triggers:
	hello_trigger change-submit //depot/... "python X:/projects/2010/p4/hello_trigger.py"

Close/save the trigger TMP file, you should see ‘Triggers saved.’ echo’d at the prompt. Now, when we try to submit a file to the depot, we will get this:

So: awesome, you just DENIED your first check-in!

Connecting to Perforce from Inside a Trigger

So we are now denying check-ins, but let’s try to do some other things, let’s connect to perforce from inside a trigger.

from P4 import P4, P4Exception
 
p4 = P4()
 
try:
	#use whatever your admin l/p was
	#this isn't the safest, but it works at this beginner level
	p4.user = "admin"
	p4.password = "admin"
	p4.port = "1666"
	p4.connect()
	info = p4.run("info")
	print info
	sys.exit(1)
 
#this will return any errors
except P4Exception:
	for e in p4.errors: print e
	sys.exit(1)

So now when you try to submit a file to depot you will get this:

Passing Info to the Trigger

Now we are running triggers, accepting or denying checkins, but we really don’t know much about them. Let’s try to get enough info to where we could make a decision about whether or not we want the file to pass validation. Let’s make another python trigger, trigger_test.py, and let’s query something from the perforce server in the submit trigger. To do this we need to edit our trigger file like so:

Triggers:
	test change-submit //depot/... "python X:/projects/2010/p4/test_trigger.py %user% %changelist%"

This will pass the user and changelist number into the python script as an arg, the same way dragging/dropping passed args to python in my previous example. So let’s set that up, save the script from before as ‘test_trigger.py’ as shown above, and add the following:

import sys
from P4 import P4, P4Exception
 
p4 = P4()
describe = []
 
try:
	p4.user = "admin"
	p4.password = "admin"
	p4.port = "1666"
	p4.connect()
 
except P4Exception:
	for e in p4.errors: print e
	sys.exit(1)
 
print str(sys.argv)
describe = p4.run('describe',sys.argv[2])
print str(describe)
 
p4.disconnect()
sys.exit(1)

So, as you can see, it has returned the user and changelist number:

However, for this changelist to be useful, we query p4, asking the server to describe the changelist. This returns a lot of information about the changelist.

Where to Go From here

The few simple things shown here really give you the tools to do many more things. Here are some examples of triggers that can be  created with the know-how above:

• Deny check-ins of a certain filetype (like deny compiled source files/assets)
• Deny check-ins whose hash digest matches an existing file on the server
• Deny/allow a certain type of file check-in from a user in a certain group
• Email a lead any time a file in a certain folder is updated

Did you find this helpful? What creative triggers have you written?

click to enlarge

I have been researching the best options available for the D300 when it comes to quickly generating some lightprobes/panoramas. This of course means fisheye lenses. Currently, Sigma is the only company that makes a 180 degree circular fisheye. They come in two flavors, 8mm, and 4.5mm. The 8mm projects a full circle onto a full 35mm sensor (full frame), but on an APS-C sensor it is cropped. The 4.5mm however, throws a perfect circular image onto an APS-C sized sensor; I believe it is the only lens that does this.

The Pixels

You would think that the 4.5mm would be the way to go, I did until I took a look at both. It really comes down to the pixels. The width in pixels of the image thrown by the 4.5mm lens is roughly 2285px in diameter. So while you can shoot less, an entire panorama taking about 3 shots, it will come out as a <4k equirectangular. However, using the 8mm, you need 4 shots, plus one zenith (5 shots total) and it generates an 8k image.  While the 4.5mm does generate a 180 degree image across, as you can see it is very wasteful.

So why doesn’t the lens have full coverage in at least the short dimension? I think it’s because it’s a lens designed to fit Canon and Sigma cameras, not just Nikon. Canon sensors have a 1.6 crop factor and Sigma’s Foveon X3 has a 1.7 crop factor (13.8mm)! The coverage is so small because Nikon DX format has a 1.5 crop factor, the APS-C sensor is much larger than Canon or Sigma. The actual circle measures 12.3mm, even small for the Sigma, which makes me believe they future-proofed it for Four Thirds.

For an APS-C sensor like the D300, I would recommend the 8mm, unless you really need a full uncropped image. The 4.5mm, while being more expensive, also has an aperture of 2.8, compared to the 8mm (f/3.5)

I am not super constrained on time, if you are on set and shooting bracketed probes between takes or something, the 4.5mm will save you two shots (18 pictures) and this might be preferable. That said, it will only generate a 4k image in the end (which might be enough)

In Python, a Decorator is a type of macro that allows you to inject or modify code in functions or classes. I was turned onto this by my friend Matt Chapman at ILM, but never fully grasped the importance.

class myDecorator(object):
	def __init__(self, f):
		self.f = f
	def __call__(self):
		print "Entering", self.f.__name__
		self.f()
		print "Exited", self.f.__name__
 
@myDecorator
def aFunction():
	print "aFunction running"
 
aFunction()

When you run the code above you will see the following:

>>Entering aFunction
>>aFunction running
>>Exited aFunction

So when we call a decorated function, we get a completely different behavior. You can wrap any existing functions, here is an example of wrapping functions for error reporting:

class catchAll:
	def __init__(self, function):
		self.function = function
 
	def __call__(self, *args):
		try:
			return self.function(*args)
		except Exception, e:
			print "Error: %s" % (e)
 
@catchAll
def unsafe(x):
  return 1 / x
 
print "unsafe(1): ", unsafe(1)
print "unsafe(0): ", unsafe(0)

So when we run this and divide by zero we get:

unsafe(1):  1
unsafe(0):  Error: integer division or modulo by zero

Using decorators you can make sweeping changes to existing code with minimal effort, like the error reporting function above, you could go back and just sprinkle these in older code.

I have really been trying to learn some Python fundamentals lately, reading some books and taking an online class. So: wow. I can’t believe that I have written so many tools, some used by really competent people at large companies, without really understanding polymorphism and other basic Python concepts.

Here’s an example of my sequence method from before, but making it a class using special class methods:

class imSequence:
	def __init__(self, file):
		dir = os.path.dirname(file)
		file = os.path.basename(file)
		segNum = re.findall(r'\d+', file)[-1]
		self.numPad = len(segNum)
		self.baseName = file.split(segNum)[0]
		self.fileType = file.split('.')[-1]
		globString = self.baseName
		for i in range(0,self.numPad): globString += '?'
		self.images = glob.glob(dir+'\\'+globString+file.split(segNum)[1])
 
	def __len__(self):
		return len(self.images)
 
	def __iter__(self):
		return iter(self.images)

Here’s an example of use:

seq = imSequence('seq\\test_00087.tga')
print len(seq)
>>94
print 'BaseName: %s  FileType: %s  Padding: %s' % (seq.baseName, seq.fileType, seq.numPad)
>>BaseName: test_  FileType: tga  Padding: 5
for image in seq: print image
>>seq\test_00000.tga
>>seq\test_00001.tga
>>seq\test_000002.tga
...

[More info and examples: Dive Into Python: Special Class Methods]

This is what the final product will look like. Two D300s, mounted as close as possible, sync’d metering, focus, flash, and shutter. Rig cost: Less than 30 dollars! Of course you are going to need two d300s and paired lenses, primes or zooms with a wide rubberband spanning them if you are really hardcore. Keep in mind, the intraoccular is 13.5cm, this is a tad more than double the normal human width, but it’s the best we can do with the d300 [horizontal].

Creating the Camera Bar

This mainly involves you going to the local hardware store with your D300 and looking at the different L-brackets available. It’s really critical that you get the cameras as close as possible, so mounting one upside down is preferable. It may look weird, but heck, it already looks weird; might as well go full retard.

I usually get an extra part for the buttons, because they will need to be somewhere that you can easily reach

Creating the Cabling

Nikon cables with Nikon 10-pin connectors aren’t cheap! The MC-22, MC-23, MC-25, or MC-30 are all over 60 dollars! I bought remote shutter cables at DealExtreme.com. I wanted to make my own switch, and also be able to use my GPS, and change the intraoccular, so the below describes that setup. If you just want to sync two identical cams, the fastest way is to buy a knock-off MC-23, which is the JJ MA-23 or JJ MR-23. I bought two JueYing RS-N1 remote shutters and cut them up. [$6 each]

I only labeled the pins most people would be interested in, for a more in depth pin-out that covers more than AE/AF and shutter (GPS, etc), have a look here. I decided to use molex connectors from RC cars, they make some good ones that are sealed/water-resistant and not too expensive.

So the cables have a pretty short lead. This so that I can connect them as single, double, have an intraoccular as wide or as short as any cable I make.. The next thing is to wire these to a set of AF/AE and shutter buttons.

Black focuses/meters and red is the shutter release. It’s not easy to find buttons that have two press states: half press and full press. If you see above, shutter is the combination of AF/AE, ground, and shutter. This is before the heat shrink is set in place.

Altogether

So that should be it. Here’s my first photo with sync’d metering, focus, flash, and shutter. They can even do bursts at high speed. Next post I will try to look into the software side, and take a look at lens distortion, vignetting, and other issues.

I have been parsing through the files of other people a lot lately, and finally took the time to make a little function to give me general information about a sequence of files. It uses regex to yank the numeric parts out of a filename, figure out the padding, and glob to tell you how many files in the sequence. Here’s the code and an example usage:

#returns [base name, padding, filetype, number of files, first file, last file]
def getSeqInfo(file):
	dir = os.path.dirname(file)
	file = os.path.basename(file)
	segNum = re.findall(r'\d+', file)[-1]
	numPad = len(segNum)
	baseName = file.split(segNum)[0]
	fileType = file.split('.')[-1]
	globString = baseName
	for i in range(0,numPad): globString += '?'
	theGlob = glob.glob(dir+'\\'+globString+file.split(segNum)[1])
	numFrames = len(theGlob)
	firstFrame = theGlob[0]
	lastFrame = theGlob[-1]
	return [baseName, numPad, fileType, numFrames, firstFrame, lastFrame]

Here is an example of usage:

print getSeqInfo('E:\\data\\data\\Games\\Project\\CaptureOutput\\Frame000547.jpg')
>>['Frame', 6, 'jpg', 994, 'E:\\data\\data\\Games\\Project\\CaptureOutput\\Frame000000.jpg', 'E:\\data\\data\\Games\\Project\\CaptureOutput\\Frame000993.jpg']

I know this is pretty simple, but I looked around a bit online and didn’t see anything readily available showing how to deal with different numbered file sets. I have needed something like this for a while that will work with anything from OBJs sent from external contractors, to images from After Effects…

So I have always been wondering how you can create almost like a ‘droplet’ to steal the photoshop lingo, from a python script. A while ago I came across some sites showing how to edit shellex in regedit to allow for files to be dropped on any python script and fed to it as args (Windows).

It’s really simple, you grab this reg file [py_drag_n_drop.reg] and install it.

Now when you drop files onto a python script, their filenames will be passed as args, here’s a simple script to test.

import sys
 
f = open('c:\\tmp.txt', 'w')
for arg in sys.argv:
    f.write(arg + '\n')
f.close()

When you save this, and drop files onto its icon, it will create tmp.txt, which will look like this:

X:\projects\2010\python\drag_and_drop\drag_n_drop.py
X:\photos\2010.04 - easter weekend\fuji\DSCF9048.MPO
X:\photos\2010.04 - easter weekend\fuji\DSCF9049.MPO
X:\photos\2010.04 - easter weekend\fuji\DSCF9050.MPO
X:\photos\2010.04 - easter weekend\fuji\DSCF9051.MPO
X:\photos\2010.04 - easter weekend\fuji\DSCF9052.MPO

The script itself is the first arg, then all the files. This way you can easily create scripts that accept drops to do things like convert files, upload files, etc..

I got some good feedback from the last post and updated the script to export JPEG Stereo (JPS) and PNG Stereo (PNS, really.) This way you can convert your images into a single lossless image that you can pop into photoshop and adjust hsv/levels, etc.

import mpo
mpo.makePNS('DSCF9463.MPO')
#>>Saving image: DSCF9463.PNS
#>>Save complete.

This is a super simple python script, no error padding. Also, keep in mind that coming from most modern camera rigs, you are saving like a 20-40 megapixel PNG compressed file here, wait until it says it is done saving, it may take a few seconds.

Many stereo cameras are using the new MPO format to store multiple images in a file. Unfortunately, nothing really works with these files (Other than Stereo Photo Maker). Here is a simple python wrapper around ExifTool that will extract the Right and Left image, and return EXIF data as a dict. I think this is probably easier than explaining how to use ExifTool, but you can see from looking at the simple wrapper code.

import mpo
 
#Name of MPO file, name of output, whether or not you want all EXIF in a txt log
mpo.extractImagePair('DSCF9463.MPO', 'DSCF9463', True)
#>>Created DSCF9463_R.jpg
#>>Created DSCF9463_L.jpg
#>>Writing EXIF data

The above leaves you with two images and a text file that has all the EXIF data, even attributes that xnView and other apps do not read:

exif =  getExif('DSCF9463.MPO')
print exif["Convergence Angle"]
#>>0
print exif["Field Of View"]
#>>53.7 deg
print exif["Focal Length"]
#>>6.3 mm (35 mm equivalent: 35.6 mm)

I have been really amazing myself at how much knowledge I have forgotten in the past five or six months… Most of the work I did in the past year utilized the UIC module to load UI files directly, but I can find very little information about this online. I was surprised to see that even the trusty old Rapid GUI Programming with Python and Qt book doesn’t cover loading UI files with the UIC module.

So, here is a tiny script with UI file [download] that will generate a pyqt example window that does ’stuff’:

import sys
from PyQt4 import QtGui, QtCore, uic
 
class TestApp(QtGui.QMainWindow):
	def __init__(self):
		QtGui.QMainWindow.__init__(self)
 
		self.ui = uic.loadUi('X:/projects/2010/python/pyqt_tutorial/pyqt_tutorial.ui')
		self.ui.show()
 
		self.connect(self.ui.doubleSpinBox, QtCore.SIGNAL("valueChanged(double)"), spinFn)
		self.connect(self.ui.comboBox, QtCore.SIGNAL("currentIndexChanged(QString)"), comboFn)
		self.connect(self.ui.pushButton, QtCore.SIGNAL("clicked()"), buttonFn)
 
def spinFn(value):
	win.ui.doubleSpinBoxLabel.setText('doubleSpinBox is set to ' + str(value))
def buttonFn():
	win.ui.setWindowTitle(win.ui.lineEdit.text())
def comboFn(value):
	win.ui.comboBoxLabel.setText(str(value) + ' is selected')
 
if __name__ == "__main__":
	app = QtGui.QApplication(sys.argv)
	win = TestApp()
	sys.exit(app.exec_())

Change the path to reflect where you have saved the UI file, and when you run the script you should get this:

EDIT: A few people have asked me to update this for other situations

PySide Inside Maya:

import sys
from PySide.QtUiTools import *
from PySide.QtCore import *
from PySide.QtGui import *
 
class TestApp(QMainWindow):
	def __init__(self):
		QMainWindow.__init__(self)
 
		loader = QUiLoader()
		self.ui = loader.load('c:/pyqt_tutorial.ui')
		self.ui.show()
 
		self.connect(self.ui.doubleSpinBox, SIGNAL("valueChanged(double)"), spinFn)
		self.connect(self.ui.comboBox, SIGNAL("currentIndexChanged(QString)"), comboFn)
		self.connect(self.ui.pushButton, SIGNAL("clicked()"), buttonFn)
 
def spinFn(value):
	win.ui.doubleSpinBoxLabel.setText('doubleSpinBox is set to ' + str(value))
def buttonFn():
	win.ui.setWindowTitle(win.ui.lineEdit.text())
def comboFn(value):
	win.ui.comboBoxLabel.setText(str(value) + ' is selected')
 
win = TestApp()

PyQT Inside Maya:

import sys
from PyQt4 import QtGui, QtCore, uic
 
class TestApp(QtGui.QMainWindow):
	def __init__(self):
		QtGui.QMainWindow.__init__(self)
 
		self.ui = uic.loadUi('c:/pyqt_tutorial.ui')
		self.ui.show()
 
		self.connect(self.ui.doubleSpinBox, QtCore.SIGNAL("valueChanged(double)"), spinFn)
		self.connect(self.ui.comboBox, QtCore.SIGNAL("currentIndexChanged(QString)"), comboFn)
		self.connect(self.ui.pushButton, QtCore.SIGNAL("clicked()"), buttonFn)
 
def spinFn(value):
	win.ui.doubleSpinBoxLabel.setText('doubleSpinBox is set to ' + str(value))
def buttonFn():
	win.ui.setWindowTitle(win.ui.lineEdit.text())
def comboFn(value):
	win.ui.comboBoxLabel.setText(str(value) + ' is selected')
 
win = TestApp()

My friends Judd and Rich gave a talk on some of the Character Tech behind Uncharted 2. Here are the slides.

I have gotten back into some pyqt in my spare time, just because it’s what I used on a daily basis at the last place I worked at. However, I had trouble getting it to run in my text editor of choice. (SciTE)

I couldn’t find a solution with like 45 minutes of googling. When trying to import PyQt4 it would give me a dll error, but I could paste the code into IDLE and it would execute fine. I found a solution by editing the python preferences of SciTE. I noticed that it wasn’t running python scripts the way IDLE was, but compiling them (?). I edited the last line to just run the script, and viola! It worked.

Find this line (usually the last):

command.1.*.py=python -c "import py_compile; py_compile.compile(r'$(FilePath)')"

And change it to:

command.1.*.py=python "(r'$(FilePath)')"

I don’t really know if this messes anything else up, but it does allow the PyQt4 libs to load and do their thing.

The Vatican recently put up an interactive Sistine Chapel flash application. You can pan around the entire room and zoom in and out in great detail.

The Vatican is not very open with it’s art, the reason they scream ‘NO PHOTO’ when you pull a camera out in the chapel is that they sold the ability to take photos of it to a Japanese TV Station (Nippon TV) for 4.2 million dollars. Because the ceiling has long been in the public domain, the only way they can sell ‘the right to photograph’ the ceiling is by screwing over us tourists who visit. If you take a photo, they have no control over that image –because they don’t own the copyright of the work.

Many of you who know me, know I am a huge fan of Michelangelo’s work, this data was just too awesomely tempting and when I saw it posted publicly online, I really wanted to get my hands on the original assets.

Here is a python script to grab all of the image tiles that the flash app reads, and then generate the 8k faces of the cubemap. In the end you will have a 32,000 pixel cubemap.

First we copy the swatches from the website:

def getSistineCubemap(saveLoc):
	import urllib
	#define the faces of the cubemap, using their own lettering scheme
	faces = ['f','b','u','d','l','r']
	#location of the images
	url = 'http://www.vatican.va/various/cappelle/sistina_vr/Sistine-Chapel.tiles/l3_'
	#copy all the swatches to your local drive
	for face in faces:
		for x in range(1,9):
			for y in range(1,9):
				file = (face + '_' + str(y) + '_' + str(x) + '.jpg')
				urllib.urlretrieve((url + face + '_' + str(y) + '_' + str(x) + '.jpg'), (saveLoc + file))
				urllib.urlcleanup()
				print "saved " + file

Next we use PIL to stitch them together:

def stitchCubeMapFace(theImage, x, y, show):
	from PIL import Image, ImageDraw
	from os import path
 
	file = theImage.split('/')[-1]
	fileSplit = file.split('_')
	im = Image.open(theImage)
	#create an 8k face from the first swatch
	im = im.resize((8000, 8000), Image.NEAREST)
	thePath = path.split(theImage)[0]
 
	xPixel = 0
	yPixel = 0
	#loop through the swatches, stitching them together
	for y_ in range(1, x+1):
		for x_ in range(1,y+1):
			if yPixel == 8000:
				yPixel = 0
			nextImage = (thePath + '/' + fileSplit[0] + '_' + str(x_) + '_' + str(y_) + '.jpg')
			print ('Merging ' + nextImage + ' @' + str(xPixel) + ',' + str(yPixel))
			loadImage = Image.open(nextImage)
			im.paste(loadImage, (xPixel, yPixel))
			yPixel += 1000
		xPixel += 1000
	saveImageFile = (thePath + '/' + fileSplit[0] + '_face.jpg')
	print ('Saving face: ' + saveImageFile)
	#save the image
	im.save(saveImageFile, 'JPEG')
	#load the image in default image viewer for checking
	if show == True:
		import webbrowser
		webbrowser.open(saveImageFile)

Here is an example of the input params:

getSistineCubemap('D:/sistineCubeMap/')
stitchCubeMapFace('D:/sistineCubeMap/r_1_1.jpg', 8, 8, True)