Rigging Tips: Methods for Extra Character Deformation (in Game Dev)

13 min readNov 13, 2020

A question I’m often asked as a rigger in game dev is ‘when should I use blendshapes?’ or ‘when should I use helper joints?’ when creating more advanced character definition. Because of the limited nature of real-time deformers —skinned joints and blendshapes are often the only two real-time deformers rigging artists have at their disposal that can run in a real-time environment.

This doesn’t mean what is described below isn’t applicable outside of a real-time environment — just that the limitations for pre-rendered aren’t as strict and the tools and techniques at hand are a lot more complex and powerful. This information will be useful in your arsenal — but consider the restrictions on them a little looser.

(Note: I’ve done this job for a decent while now — but I’m not the arbitrator of all knowledge. What I say might not be true for your pipeline or production — but I operate on general guidelines here that are hopefully useful for everyone. If you have any questions, comments or feedback feel free to ping me.)

Firstly, let’s define what some of these terms so we are all sharing the same vocabulary.

Helper Joints:

Helper Joints are additional skeletal joints in a character that are purely added to improve deformation. These might go by other names in your pipeline, but for the sake of this article I’m calling them ‘helper joints’. Example of these can be something as commonly needed twist joints to support deformation in the shoulder, forearm and legs. The goal here is that there is the core skeleton that drives all the major limbs and appendages, but helper joints are critical if you want to drive extra motion to simulate the contraction or twisting of muscle, as well as mass preservation. In essence, a series of joints can be used to stabilise deformation and counteract a lot of the common pitfalls of classic linear skinning.

A wonderful example demonstrating these joints in action can be seen here on Ellie’s wrist rig from Naughty Dog’s The Last of Us.

Helper joints on Ellie’s wrist rig from The Last of Us.

You can see as the wrist bends forward and backwards — 3 joints will move to fill in the volume in areas where it is needed. You can view this in Judd Simantov’s great overview of the rigs from The Last of Us here.

However — there are some real limitations to this. Firstly, game engines often have hard limits on how many joints can have influence on a single vertex. Modern AAA games often have 4 joints per vertex as a max, but this can be pushed to 8 (and even 16 in some of the most extreme cases in modern games) for cinematics and high quality characters. Mobile games, lower quality assets and skinned objects drawn in the distance (as LODs — LOD meaning ‘level of detail’) will have even less. If you are skinning your pelvis, you may be sharing influence between your left and right hip joint — as well as your pelvis and lower spine. You can see how in these complex areas you will reach your max very quickly.

In Maya, you can see the weights on an individual vertex in the Component Editor under Smooth Skins. Here, you can see the selected vertex is part of a skin cluster where the max influence per vert is set to 4.

Secondly, whenever you add a joint — you are removing influence from another joint. All joints must add to a total of 1. This means there is often a delicate balance when skinning — and when you move a joint it will be competing with all the surrounding skinned joints. The further away you move the joint, especially when rotation is taken into consideration, the more mass your skinning will lose. An example of this can be seen below, as I rotate the hip around 55 degrees — the mass on the buttocks seems to disappear. When I flip the skinning evaluation from linear to dual quaternion (which is another form of skinning evaluation that is built to combat this mass loss) you can see how much mass is lost from a single joint rotation. (Note: Dual Quaternion is possible in some engines, but it is more expensive and is an either/or situation that means linear skin clusters come out as the winner in general).

You can see the mass loss as the joint rotates — as I switch between the skinning evaluation modes you can see the mass change.

Because of this, achieving the exact shapes you need can be a very delicate and time-consuming balance. There are ways to counter this, like scripts that transfer weights from blendshape to joints (an open source version of this can be found here. Also check out Hans Godard’s skinning converter here if you want to see how this looks) or using a skinning tool like ngSkinTools (that operates on layers). However, this can get messy and overtly complex very quickly if you are running up against engine limitations.

So with that said, let’s move on to other other solution — blendshapes.

Blendshapes:

Blendshapes are target meshes that your mesh will deform to — normally within a 0–1 range. To achieve this — the original object and it’s target objects must have identical vert order and topology. If these do not match — the results will be wildly unpredictable or just plain broken. When you create a blendshape and add a ‘target shape’ — Maya will examine the difference between them (the delta) and create the ‘target blendshape’ from that.

Below I’ve detailed a hypothetical blendshape example. As you can see, there are two targets. When I animate the value of the blendshapes ‘target_1’ — you will see the original mesh deform to match the target in a linear fashion.

The same can be seen for target 2:

These shapes are linear and additive. Because of this, when I animate both targets at the same time, it results in the below:

Animating both target_1 and target_2 on the blendshape

You can see this creates a new composite shape — something that didn’t exist before. This is the very basics of how blendshapes work.

The cool thing about blendshapes is if you have active targets you can edit them in real-time. Maya has pretty robust shape editing tools in the Shape Editor these days — so it’s worthwhile taking a look at that.

Editing a blendshape target while it is active.

Now, when we factor this into a skinned model — the skin cluster has to be taken into the evaluation (and must be performed last — always check the order of evaluation when you create the blendshape node as it doesn’t always default to this behaviour in Maya). When your character is posed, you will want to make a blendshape from that pose and modify it. Because the skinning and movement of joints is taken into consideration from this — your delta (the shape that is calculated and added to the blendshape whenever you add a target) may look bizarre. I may cover this later in another post — but for now just know that adding skinning complicates the process of generating the delta shapes.

So let’s make an example from this: that of an arm bending. When this happens, the bone in your lower arm will rotate and will cause your elbow to become more pointed.

Below is an X-ray of the bend in motion:

X-ray image of an elbow bending and the movement of the bone.

In this case — because of the simplicity of the movement (it’s a hinged movement in a single axis) you could craft a blendshape to capture this. In classic linear skinning — no matter how you skin a simple 3 joint bone chain — this bend will look rounded (unless you use some sneaky modelling techniques detailed in this great example video! However in this case, I’m using this generic example to show the root problem with linear skinning).

Default classic linear skinning behaviour on an elbow bend.

To combat this rounded elbow — you might craft a elbow bend blendshape. Below is an example of one:

On the left — the original skinned mesh with the elbow shape. On the right, the correct shape (static mesh — no skinning)

When we create a blendshape deformer with this corrective shape and hook it up so that it activates when the arm begins to bend, we get the following:

The arm bending with an active blendshape. You can see the elbow sharpens here.

Here’s a side by side comparison of with and without:

You can see the blendshape activating on the bend on the right, but not on the left.

However! With blendshapes you are able to add resolution and finesse that might be harder with helper joints. Let’s take a look at the comparison from another angle!

Look at the volume where the arrows point! Can you see the difference?

In this example above, you can see in the arm with the blendshape active that the skin around the bend of the elbow gets pushed out and up — much like the mass of a real arm. This kind of resolution is easily sculpted with a blendshape — but is much harder to achieve with joints only.

Now that we know a little about both of these methods — let’s review the pros and cons so you might know which system might best fit your problem and pipeline.

Pros and Cons: Helper Joints:

Pros

Joints setups can be scripted and much more reusable across characters. For example if you have a knee bending joint solution — it’s likely that that can be used across all bipeds with minimal upkeep
Excel in areas where blendshapes might not — for example twist joints! Replicating a twist joint setup (the joints that distribute twist between two joints) with blendshapes would be very cumbersome and difficult because of the variable distribution and also the linear movement of blendshapes. Helper joints handle arcs a lot better than blendshapes do. On top of that — you can get a lot of movement for free depending on how it’s setup — games can use these twist joints to create smears frames like as seen in Overwatch:

Look at the arc on that arm! Achieved by pulling twist joints off the rigid IK arm. The smear on the leg is achieved by using non-linear scale (often supported in engine on leaf nodes — the last node in a joint chain)

Artists can use helper joints to craft/tweak the appeal of frames if they are animatable and accessible in the DCC app (the program where you create the content — like Maya). If riggers expose these to animators they can really craft shapes down to the detail in those special cases (like high-quality cinematics)!
Easy to script and write into rigs as a default! Since blendshapes have to be hand-crafted — joints can do a lot of the guess work and be automated by a script.
Can take memory burden off character with lots of vertices. Blendshapes aren’t free in a game engine. They cost memory to store! Joints instead of blendshapes can cheapen characters that are vertex heavy.
When people talk about driving helper joints — set driven keys (SDKs) are often one of the more commonplace techniques. But this is only one way — it can be driven by a wide array of drivers such as RBF nodes, cones, maths, curves, etc… Helper joints can have very elegant and versatile solutions — as creative as the rigger making them can get!
If you can drive joints real-time in engine (instead of having to drive them in DCC) you can get truly real-time reactive joints! Great for when your character enters a ragdoll like state or between animation blends.

Cons

If it’s for a game it’s very easy to hit max vertex limit (mentioned above) in complex areas like shoulder/armpit or hips/thigh/pubic area. Balancing these areas can be extremely time consuming and frustrating.
Mentioned above — it’s a delicate balance of skinning between the helpers and movement. Improving one area can potentially reduce the effectiveness of another shape. You have to be very careful when adjust skinning on the helper joints. The more weight they hold — the more they have to be doing the right thing and doing the heavy lifting — but if the weight on them is too soft it can be very difficult to get them achieving the shapes you need — or filling in the volume in areas where it gets lost.
Speaking of — volume will be lost as you do bigger movements — especially with rotations (a problem that’s linked to classic linear skinning)
If your joints are baked and exported from your DDC program — then there will be a memory cost to the curves in engine. If you have large animation sets (say an enemy type that is used everywhere) then this can seriously bloat the data.
Say goodbye to nice interpolation when your character is blending between animations or enters a ragdoll state. This can something cause pops or jittering artifacts. This is also true to blendshapes — the only way to resolve this is to have the helper joints solving at runtime.
If you DO have real-time helper joints running real-time in engine — without a solution on the DCC side for animators to see that — they could be working blind and will find that massively frustrating that they can’t see their character as close to 1:1 with game as possible.
If the joints aren’t solved real-time — every time you adjust how they are driven you need to re-export the animation on all the affected animations that use that skeleton.
If you are adding helper joints on to characters but they are running animation from another character that doesn’t include them — what happens to these joints? Your engine might not have an elegant solution for this!
If your helper joints are not perfectly mirrored you can lose accurate and good-looking engine-side animation mirroring which can cut the memory of animation sets on characters considerably!

Pros and Cons: Blendshapes

Pros:

You can craft really unique or desirable shapes — no messing with multiple joints and weight limits! Some deformations are literally not achievable in games without blendshapes! Specifically high-frequency detail shapes or shapes in areas where there isn’t enough joint resolution to fix it adequately.
Depending on your pipeline and tools — blendshapes can be easy to fix and update. Regenerating and integrating can be as simple as a button click and re-export.
Updating the shapes means you only need to re-export the mesh they are associated with (unless how they are triggered is modified). Much easier on a pipeline for iteration.
Can be parameterised for reuse across characters (for example, L_leg_out can be reused between characters if naming is generic enough) — allowing you to have unique shapes per character but have them automatically driven via channels on the rig.
Can be potentially outsourced (depending on the expertise you have). Whether this is inhouse or externally — this takes the burden of making quality deformation off your valuable riggers and frees them up to build more rigs or tools!
Can be easily batched transferred between characters! Your results may vary, but there are lots of ways to transfer blendshapes that affect volume between similar characters.

Cons:

Memory — game engines might be limited on how many blendshapes can be active on a character or everything in the game at once. Your engine might not support blendshapes at all! Often there is a cap at which lesser shapes (blendshapes that affect less verts) will be pruned. This can a cause pops or artifacting if this isn’t carefully managed as blendshapes fight against each other to be active.
Time intensive — often useful to have a dedicated sculpting artist. Sculpting blendshapes is not an easy process and dedicated blendshape artists are few and far between — especially within games development. Many a good character artist has quit their role after being relegated to ‘the blendshape artist’. Make sure if you have someone who is dedicated to this task — that they enjoy this role! Otherwise you will have to spread the love across the team to make sure none of your character artists quit from sheer frustration.
Not easily reusable in DCC — can be transferred to other meshes as mentioned above but results can be really off depending on the mesh and shapes being transferred. Often, the transferring process is very slow — especially if you need to regenerate for LODs too. Often they are also not re-usable between characters at all at runtime too — unless you are very restrictive and clever about how you set them up.
Need to be regenerated every time a mesh is updated or skinning is adjusted. Changing the skinning changes the corrective delta — thus regenerating is a must. If you have tools for this it is potentially not as big of a deal.
Export time! Meshes with high numbers of blendshapes that need to be processed at export time can have massive export times/load times into engine making iteration slow and time consuming.
Blending between shapes can cause ‘swimming’. If you have in-betweens or breakdown shapes (pretty much necessary with large rotation/arcs) then you have to be very careful about how/what vertices you move. If the movement and degree of which they move is not consistent between shapes then you will likely get ‘swimming’ — where a vertex will move back and forth as it blends between the shapes. (I might cover this in a later post if there is interest!)
Because of the above, you have to be very technical and careful about what vertices you move if you want to minimise these issues — which can be incredibly easy to spot in meshes with high/fine texture detail.
Any movement with arc is going to suffer because of the linear nature of blendshapes (eyelids, for example) — either meaning you must create more blendshapes for inbetweens or be very selective of what blendshapes you choose to craft.
Crafting shapes takes a lot of technical knowledge of how the maths is moving vertices underneath (combined with the skinning) that isn’t often natural to artists. Even using tools that help artists visualise this can be cumbersome or expensive!
Some engines don’t update normals on blendshapes at runtime! This can give strange or undesirable shading results if the movement on the blendshape is considerable and the surface of the affected faces deform.

In an ideal world — the best rigs have blendshapes and helper joints working together to achieve the best deformation. Which parts of a rig you use what techniques on is up to you, your characters, your team and your pipeline.

This isn’t an exhaustive list by any means — and as our tools rapidly improve (especially with the new generation of games now reaching our shelves) some of these pros/cons may become non-issues. However, I’d love to hear feedback! Please leave a comment below if you have anything to add (and I can add it to the list) or feel free to email me at sol.g.brennan@gmail.com.