josh | Tube – Epic Production Notes

Crowd Simulation and Auto-walking

23-01-2011

Hey everyone, this is my first post since I’ve been back in England! In these dark depths of winter I really miss the summer I spent over in the US as a Bit Films intern; nice weather, good food, great crew. Since leaving the states I’ve been hard at work at college, and on various other projects, but now and again I get enough free time to work on Tube. Something which has been on my to-do list since B-Conf is feasibility tests for crowd simulation and auto-walking, and combinations of the two.

I’ve always struggled to set up Blender’s boids sim exactly how I needed it for ground based creatures (trying hard not to give anything away about the film here!), especially in the case of collision critical simulations when its important that two bodies don’t get tangled up together. Coupled with this I know that for Tube we will need more direct control over the ‘boids’ and being able to explicitly direct them and override the ‘boid brain’ is a must. On top of that we need more than just fly, walk and sleep actions. It seemed that rather than trying to strap something on the back of Blender’s built in boids, it might be easier to code something more specific (and more limited in scope) for a few scenes we have to animate, so I started out on a now heavily WIP crowd sim.

There used to be an auto-walker script for blender 2.4x. I came up with my own algorithm before finding it, but did end up mimicking a few of the features (leg sequencing for one) though my implementation had to be completely different because of the way the rest of my code works. It only walks, as such, not runs, yet…

A couple of weeks ago I tested out the code and rendered a couple of demo crowd shots. Then I got completely carried away and implemented a few special effects using PIL and the ‘scribbler’ algorithm, and generated sound from the fcurves of hundreds of marching spiders. The project uncovered a few pitfalls and some more work still to do before the crowd sim is production ready, but enough of the worrying, here’s a demo video!

“Spiders” auto-walking and flock simulation for blender from Josh Wedlake on Vimeo.

Here’s the video on youtube where the compression is slightly nicer at the larger sizes.

Modeling the Train, with Automatic Baking

By: josh

13-09-2010

12 comments

blender, pipeline, production, python

In between bouts of python coding I’ve been working on the model for the train and carriages. The model is asymmetric and modular so in theory loads of variations could be created by combining bits from the two models shown below. The two ends of each train are different, as are the pantographs and all of the side panels. The design is based on a mix of old NYC subway cars and Soviet-era Eastern European engines. I built the model on top of an early design for the undercarriage which had been modeled by Jean-Sébastien Guillemette and Jarred de Beer.

The model for the train is currently very high poly as it will probably be used in a few close-up shots (these renders are all geometry – no texture normal/bump maps!). There’s a bit of work still left to do (naming 3000 objects and parenting them in a nice ordered heirarchy for one!) and making a low poly version, not to mention further tweaking of the design and rigging the moving parts! To make the low poly version lots of the objects can simply be moved to a hidden layer (or excluded from the ‘HI’ group – we use group instances to bring models into scene files), but many of the meshes need to be split up into smaller parts so that the smaller parts can be excluded from the low poly renders – for example we need to move the rivets out of the body panel meshes as they will be so small in many of the long shots they won’t be noticeable – even at 2K!

For continuity’s sake we need to make sure the rivets are in the same place in the high and low poly versions. To save the texture painters some time later down the pipeline I’ve written a script which goes through every object in the mesh and separates the details from the main lower poly mesh component (I can define these using vertex groups etc) and then bakes the details’ AO ‘shadow’ onto the lower poly part of the mesh and saves the texture in a maps file. The script also intelligently names all of these textures in case links get broken up as the SVN gets reorganized over time. For example if the script finds an object called ‘side_door’ it will split the object into ‘LOW0001side_door’ (for low poly) and ‘DET0001side_door’ (for details) and then bake out the AO to an image called ‘IMG0001side_door’ while making sure that all the meshes stay linked to save memory (rendering without any textures is already taking up over 2GB of memory).

Unlike the normal P-key behaviour, the script makes sure that separating meshes affects all the linked duplicates, not just one of them. The numerical prefix helps identify one specific detail mesh with its corresponding specific lower poly mesh. For example if there are 5 duplicates of ‘side_door’ (‘side_door.001′,’side_door.002′…), the first will be renamed ‘LOW0001side_door’ with its details saved in ‘DET0001side_door’, the second will be renamed ‘LOW0002side_door’ and its details saved in ‘DET0002side_door’ and so on, so future ‘tubers’ to work on the model won’t have to spend hours searching the outliner list to find the right object! The reason the number is at the start and not the end of the name is to stop blender messing with the numbers, and to help sorting in an alphabetized list!

Fingers crossed there won’t be any surprising bugs in the script, as baking out 1000 unique meshes is likely to take some time and we haven’t written a resume function for the script yet!

Mushroomer Documentation Published

By: josh

02-09-2010

0 comments

production

Thanks for all the positive feedback we’re had on the demo video! Due to the interest shown I’ve started a documentation wiki and written a substantial amount of info about the less-than-obvious features that are included. Its more of a technical reference so everyone working on our project knows how to use the generator than a straightforward tutorial. If anyone is interested in writing/screencasting a tutorial I’d be happy to offer help when I can! We’ve also released an example blend file with a simple setup.

UPDATE: Documentation Wiki and latest available script version for r31856 [Fri Sep 10 16:54:53 CEST 2010]

[UPDATE] While the blender python API is constantly changing I’ll try to keep track of which releases of the Mushroomer work with which releases of blender on the wiki. If you have trouble getting the script to work first check your console for any kind of python errors. If there are any post them here. Check the documentation on the wiki and see if you’re doing anything wrong, and we’ll have a look and see if the script’s broken again!

while the cat is away…

By: josh

01-09-2010

14 comments

announcement, pipeline, production

So Bassam’s been away in Bulgaria for a couple of weeks and Henri and I mostly had the studio to ourselves. I’ve been working on a mushroom generator for a couple of timelapses – not that tube will be filled with glowing mushrooms, more that we wanted something half way between the ivy generator and a fully fledged particle system. Henri modeled some funky mushrooms and together we spun together a quick demo video just to show off some of the effects that are possible, and to exercise our fetish for indirect lighting and luminous pink.

Mushroom Generator Blender 2.5 from Henri Hebeisen and Josh Wedlake on Vimeo.

The script is still heavily in development but if you like alpha stuff and you’re happy to do your own debugging, then feel free to download and run.

UPDATE: Find the latest build here.
UPDATE: release 4 is fixed for r31856 [Fri Sep 10 16:54:53 CEST 2010]… don’t expect it to last long though!

Essentially you need to model a couple of mushrooms (just a generic term – you can model flowers or trees or robotic arms) with some shapes for their animation which will be blended sequentially, some shapes for random variation, some shapes in which they bend up the y axis, a painted vertex group for shrinkwrapping the base of the mushroom, all scaling and rotation applied and the origin at the base of the mesh.

above: creating the shapes for auto animation

above: adding manual animation to a mushroom

You also need a target mesh which has nice topo (ideally no elongated tris or nasty convex quads), optionally painted vertex groups named OBmymushroomname… and MAmymushroommaterial… to control the distribution of your various materials and different object types, optionally a limit vertex group (ie only faces within this will receive mushrooms), and a lot of patience. Select the mushrooms then make the target active and tab into edit mode, select the start face(s) then hit spacebar>Mushroomer (remembered to run the script first).

above: mission control, godspeed

Adjust the settings and hit go. I suggest you run blender from the terminal so you can watch for progress and any hangups. You might well want to abort if it starts slowing right down from too many mushrooms.

above: when its done

Development on this script has been a bit of a nightmare. Currently it is not possible to create multiple linked objects and have different shape key and material animation blocks on them without having to duplicate the mesh and/or material blocks (thus unnecessarily eating up huge amount of memory and removing the possibility to edit the mesh for one mushroom and have all of the update). This is in the blender bug tracker as #23546 and #23547. If they get fixed you can uncomment the deep data path keyframe adding and enable all the code for migrating actions to object level. Another limitation which slowed development is that it is currently not possible to merge two actions into one. This is necessary if you are moving mesh level and material level animation data back to object level. Essentially you need to combine two actions into one. Not only is this not possible, but its also not possible to copy an fcurve in its entirity, but rather python has to iterate through very slowly copying every handle one by one, and even this is susceptible to some bugs (not yet reported). This is because the collection of fcurves is read only even though each fcurve itself if read/write. I was also held up by bug #23532 which prevented me from doing the action block combining in the NLA editor, and for a short while by bug #23548 which caused blender to crash when creating new fcurves. Finally the script can’t currently support animation data on the mushrooms’ materials due to bugs #23593 and #23594 [UPDATE Campbell fixed these, great news! Material & texture animations are now supported!] – I’ve chosen to block this feature rather than risk an inescapable hang and data loss – uncomment those lines at your own risk! Fingers crossed the devs will iron these out sooner or later and the mushroom generator will be running faster, with lower memory requirements, smaller file sizes and more stability!

Its also been a sad day today as we waved goodbye to Henri who is on his way back to France as I write. Its been great fun working together for the last 2 months. Luckily we’ll only be a stones throw apart when we’re back at our separate homes in Toulouse and London. If all goes well we’ll be at the Blender conference this October as well. Rest assured, the world hasn’t seen the end of the glowing mushrooms saga – I’m hope we’ll have another chance to work together soon!

some quick concepts…

By: josh

24-08-2010

1 comment

production

Been meaning to post these for ages… from 4 or 5 weeks ago…

I’ve been modeling the train on and off when I’ve not been pythoning – don’t want to give too much away now, just to say that the model is going to be very high poly. Most of the details are now complete but I still have all the big panels to go. I’ve built a low(ish) poly base mesh which all of the details will be deformed to (using shrinkwrapped 2D lattices). All of the side panels are modular so we can swap around the order for variations etc. Thoughts gladly received.

Introducing Smooth Tools – better control for smooth operators

By: josh

10-08-2010

16 comments

production

Blender only has one choice of smooth algorithm which can be a bit frustrating at times. Blender’s smooth can limited by the world X,Y and Z axes which is great, but it would be better if you could also constrain smoothing by object local axes or transform orientations. What about if you could also mask smoothing by vertex groups, or stop the shrinkage effect by using algorithms which don’t shrink objects or automatically shrink/flatten each vertex to retain the object’s volume, or even if you could change the way smoothing handles non-manifold edges and get smooth to leave them be or ignore them completely? How about the edges of the selection – blender currently pretends that anything which isn’t selected doesn’t exist, but what if you wanted to take account for the unselected vertices but not actually change their position? All of these were things which were bugging me last week when I was modeling the smooth curved body panels of the train in ‘tube’. I put my mind to it and coded a new add-on for blender which provides much more advanced options for all you smooth operators out there. Introducing the “Full Fat Smoothie Maker”!

DOWNLOAD v0.1

The Full Fat Smoothie Maker

All these options might be a bit confusing to the first time user, and a lot of getting to grips with the tool is just experimenting. There are well over 1000 ways to combine the basic choices! I produced a quick reference chart which shows how the different algorithms handle some difficult test objects which I created. A high-res version of this is in the zip download.

As always let me know if you find any bugs in the comments and happy blending!

More Snapping and Double Removal Options with Python

By: josh

03-08-2010

1 comment

blender, film, production

Yet another python add-on for today! I hope this is one that will be useful to a great many of you, even though its pretty hard to explain how it works… bear with me as I expect it will speed up your modelling work flow.

I’ve long been frustrated by the limitations of ‘Remove Doubles’ in blender. Remove doubles searches for verts which are close together (in all axes) and merges them, which is great. However sometimes its nice to be able to constrain this action to a single (or pair of) axes. For instance if I have a row of vertices running along the X axis and I want all of them to have the same Y and Z coordinates I could select the entire row then press “s” (to scale), “shift-x” (to constrain to Y and Z), then “0″ (to snap all of them together) . This is fine if you only have one row of vertices. Say now that you have 100 such rows of vertices, all running roughly in the X axis direction, and you want each row to run exactly in the x-axis, so for every single one of those 100 rows you have to select each row, and press “s, shift-x, 0, enter”. Does it get a bit boring? Would you prefer if it was automated?

Have a look at the quick tutorial image below to see what I mean, and then if you want, download the add-on and install it. As always let me know if you find any bugs in the comments below and I’ll try to fix them ASAP. Happy (faster) blending!

DOWNLOAD: http://urchn.org/wp/wp-content/uploads/2010/08/snap_with_axis_constraints.zip

Tutorial:

On another note we’ve been out to the pictures a few times recently. We saw a double feature of “I am Love” and “Winter’s Bone” on Sunday. Personally I thought that the cinematography in Winter’s Bone was a refreshing break from the all-to-common extreme-DOF-fatigue that we got to see in the europudding (see Truffaut) of a movie that was “I am Love”, a film laced with religious salvation metaphor and comic (decency induced) visual-analogy-as-substitute-for-the-explicit. Conversely the portrayal of value-less material items as symbolic of ‘our’ personal hollow ambitions for wealth (in the form of tacky debris spread around the dwelling; toy horses, kitsch garden ornaments displaying cherubic qualities), set against the wholesome immaterial but valuable actions of the film’s main character provided an interesting subtext for what would otherwise have been a straightforward plot. Sadly some of our friends didn’t see it the same way and criticized the ‘off’ direction and cinematography. One way to look at it is that they were only judging according to their own personal objective perceptions of how films ‘should’ be directed and shot, rather than to how an audience subjectively perceives a film. Fire off in the comments; lets hear your own views!

Edge Tools Expanded

By: josh

02-08-2010

30 comments

production

Just a quick one to say that I’ve expanded the functionality of the bevel_weights add-on to now include the following functions:

Ctrl+E Set Edge Bevel Weights (for use with bevel weight modifier)
Ctrl+E Average Selected Bevel Weights
Select Menu > Select Same Bevel Weights (adds to the selection any edges which have an equal bevel weight to the active edge)
Select Menu > Select Same Creases (ditto but for edge crease values for use with subsurfs)
Select Menu > Select All Creased Edges
Select Menu > Select All Beveled Edges
Select Menu > Select All Marked-Sharp Edges (as opposed to the built in select sharp which works on edge angles)
W-key > Bevel – bringing the functionality of the old 2.49 style ‘destructive’ beveling is back.
Ctrl+V Smooth Vertex Group Weights (instead of the blur brush use to create nice gradations in VG weighting)

The add-on is now called edge_tools and is available here:

http://www.pasteall.org/19308/python

UPDATE: should work with r34878 onwards

I had a terrible time trying to make sure that the ‘destructive’ style beveling didn’t affect any other edges’ bevel weights. Basically it works by setting bevel weights for any selected edges then it adds a bevel modifier and applies it. Of course the user only wants the selected edges to be ‘destructively’ beveled, and not any edges which already have bevel weights so the script needs to preserve the pre-existing bevel weights. I tried various methods: first I created a custom “FloatProperty” for each edge to store the existing bevel weight in (these seemed to survive the modifier but got their values deleted). Then I tried creating edge custom properties whose names included the bevel weight’s numerical value which I could get as a string by parsing the properties with dir() (I then discovered that properties from one edge got propagated to all edges by the modifier stack). Then I noticed that edge crease values seemed to be stored to floating point accuracy. I thought that by shuffling bits around I would be able to store two numeric values (the edge crease and the bevel weight) in the edge crease’s value temporarily, which would have worked fine, except for the fact that by a quirk of the python API the edge creases get rounded up to 2 decimal places when setting them. Finally with all other plans foiled I wrote a function to generate identifier strings based on edges’ vertex locations which I could then store as dictionary keys with the old bevel weights as the values! So you can see how several hours of experimentation and head-scratching later I finally managed to implement a very simple feature… hope you find it useful! Josh

A Quick Python Time-saver

By: josh

22-07-2010

22 comments

production

Just a very quick post to share some speed-ups we’re using in the studio. Some of you may have noticed that blender 2.5′s UI is missing its bevel weight options from 2.49. Luckily all the bevel weights are editable in python. We’ve coded this quick add-on to bring back the missing functionality (which you can install from the user preferences add-on tab [remember to enable it] or run from a blender text window).

EDIT: Updated yet again for 2.55 (thanks for the comments below – sorry for sluggishness on my part!)

http://www.pasteall.org/17726/python

After you’ve installed the add-on tab into edit mode and hit spacebar and type ‘Bevel’ and you’ll see the three functions. Remember to add a bevel modifier and set it to ‘weights’ mode.

We’ve also been using Richard Olsson’s Batch Renamer to help keep scenes organised. If you add a custom keybinding to it then there’s no more copying and pasting into the buttons window.

That’s all for now, I’ll blog again soon with some of the models I’ve been working on!

Josh

Edit: Nathan Letwory (AKA Jesterking) made a nice tutorial video about this on youtube:

Across the Pond

By: josh

12-07-2010

3 comments

announcement, blender, production, team

Hello Everyone,

I’m Josh, one of new crew members for tube. I travelled over to the US from England last week, and I’m here until mid September undertaking cultural studies, and working on the models for the film. While I attempt to mediate transatlantic differences (such as introducing the rest of the team to Branston Pickle and Marmite, explaining the etymology of ‘bangers and mash’, and justifying why I need a knitted cosy for my teapot), my hosts are offering an American exchange programme complete with drive-in movies, fried dough, Independence Day Celebrations and Root Beer. In my time away from the screen I’ve been out enjoying the fresh air, beautiful countryside and very un-British weather. I’ve been running to and from work each day (8 ½ miles each way), and when I had a few hours to spare last weekend I biked up to the Sugarloaf Mountain.

Fresh out of finishing a long and traumatic Architecture degree at Cambridge University, I vowed never to work in the industry again. My first task in the studio, however, was to design the station roof and columns, and to provide general advice to the rest of the team on all things architectural! Being British, and naturally strongly resistant to change, I was slightly thrown when I realised the team was working with up to the minute svn builds of Blender. Back home in my own work I’d been hanging on to the 2.49 vintage with its historical interface not unlike the quirky 400 year old tumbledown cottage I lived in at uni. 2.5 comes with its own breed of glossy newness, an impersonal homogeneity with other 3D apps akin to the monotony of the skyscrapers in downtown LA and a feature set which sprawls on and on like the city-edge of Phoenix, Arizona.

Bewildered at first, I was tempted away from the path of the righteous by the glowing red devil’s tail of Maya on one side and the swirling captivating vortex of 3DS on the other, but eventually I found my way through the valley of darkness. I still miss many of the 2.49 features which haven’t yet been ported – skinning loops and multi-knife-cuts to name a few, and in my first few days I’ve spent a considerable amount of time filing bug reports, hopefully for the greater good.

There are still some very simple features I wish had been integrated into the new release. As what Pirsig might call a ‘mechanic of the photographic mind school’, all of my previous organisation and labelling systems have been tainted with a certain amount of… dyslexic logic. To make life easier for everyone else on the project I have to name every object, bone, group and file according to a strictly prescribed style, not least so our python automation knows what’s going on! While I don’t mind accumulating road miles on my way to and from Amherst every day, I hate the unnecessary mouse miles blender’s UI demands. I’ve illustrated one of the key issues (which could be solved by a simple hotkey and under-mouse-dialog) using an analogy that will be familiar even to non-blender users.

After breaking free from summers spent as a CAD-monkey in local architecture firms, I now find myself pining for the logical and consistent snapping and tracking systems I was so familiar with from hours spent in front of Rhino, Vectorworks, AutoCAD and the like. At least the resulting ‘errors’ present in my incapable use of Blender’s snaps often results in a more derelict and aged look! Here’s a work-in-progress snapshot of the interior of the train which I’ve been working on today, by virtually bashing it up:

That’s all for now,

Josh

THE ‘TUBE’ OPEN MOVIE PROJECT