One of the things we do consistently when animating is to keep track of a large amount of spatial and temporal information related to a specific set of performance moments. Analysis, deconstruction and reconstruction of spatial information is our bread and butter.
Any workflow element that consistently requires the use of the spatial processing part of our brain for other tasks is likely to impact deep focus. There is a distinct advantage to keeping that specific cognitive processing part of the animator's brain as clear from extraneous work as possible.
It took some time to get used to working with real-time capture, but once I gained facility and confidence I found that it was an effective way to produce organic motion and saved a huge amount of time over keyframing when laying in certain classes of movement.
In this same period I did some work with Henson's team and gained an appreciation for the serendipitious strengths of live performance. I felt that the puppeteers lacked a certain amount of discipline in working through fine detail, but their loose performance style brought performances to life in ways that sometimes eludes character animators.
I guess I got greedy. I wanted the best of both worlds, and set out to develop systems that would allow animators to maintain the detail control that we crave, and still have access to live performance advantages, without losing the ability to deliver on a production schedule.
Of course, every major application that has been designed in the last 30 years is extremely mouse-centric, and Maya is no different. The one saving grace is that an animator can still achieve a level of deep focus when using curves for control selection directly in the viewport, making the impact of mouse operation less egregious than with most other operations, but the process is still not optimal for a few reasons.
I would submit that deep focus can be further improved with UI modifications, particularly when additional viewer UI is combined with tactile tools and real-time methods.
Implementing a system was problematic though. Maya wasn't yet really ready to support real-time editing. I'd been using MotionBuilder since early FilmBox versions and focused on taking advantage of its real-time recording and playback core. What I found was that the way MB exposed those features - designed to support mocap - imposed constraints that made it impossible to achieve an efficient workflow. Doing charcter animation with this type of system requires having solid control over control selection and an efficient way to apply a sampled recording to the primary animation layer. After a number of plug-in development rounds, I simply couldn't streamline the workflow enough to make it worth chosing over hand-keying. To make tangible progress, I needed a solid content-creation platform to work with. That took a decade longer than seemed necessary, but CG CC software development is naturally driven by the large-market pipelines, which continue to be narrowly focused on the hand-key process. Only recently has the primary content-creation tool reached a point that it can really support directly capturing and manipulating real-time input. Autodesk's strategy to retire MB is, IMO, an indication that they now view Maya as a viable real-time platform - finally!
There are some really useful tricks to reduce text processing, particularly with regard to elements that are used repeatedly. Numeric memory association for often used lists is a great one. Anything groups of elements that are used often enough to commit associated numbers to memory can be recalled quickly without a big hit to deep focus. Add this to a 10-key and an operation command keypad - again to reduce mouse use - and the combination of numeric indexing and tactile muscle-memory can make a really big difference.
I do this for as many list-like structures as possible. When I sit down to animate, I print out pertinent numbered lists and tape them around my monitor. After a few days of using the referene sheets, I've generally memorized the numeric associations well enough to recall them without looking at the lists. Even when I have to use the reference sheet, I still use the command keypad and 10-key to reduce the impact on deep focus.
I tend to think in terms of four main workflows:
[1] Pose-Testing - blocking in high-level performance with holds between poses to make it easy to quickly adjust broad timing and layout decisions.
[2] Long-form Real-time Capture - Laying in recorded movement either in multi-axis blocks with a Waldo, or one or two at a time with individual knobs or sliders. It takes some training to do this well, but for some types of work it can be extremely valuable. The challenge is determining if the performance freshness and acquisition speed of the live capture exceeds the amount of detail cleanup required.
[3] Short-form Real-time Capture - This is mainly to 'punch-in' little moments that have very specific organic timing requirements that would take a while to get right through keyframing. Grabbind knob and doing a three second patch a few times (to experiment with different timing details) and then quickly cleaning up the ends can often save hours of noodling.
[4] Curve Editing - Once the performance is close, the fine precision of curve editing is hard to beat.
A key feature in making this all work is having a way of managing the different keyframe environments of sampled and sparse-key hand animation. There are some different strategies for handling this - no room to get into the details here, but it's a critical part of the mix.
The best first step I know of to address this is to make create a smooth on-ramp to the new highway, both for the artist and the production pipeline. We wouldn't be taking anything away from the existing toolkit, just adding some new tools to it. But there is still risk in how a system is implemented. Once fully adopted, at least some artists are likely to feel most comfortable with the biggest, most versatile tactile 'busy-box' they can get. But at first, and for some artists permanently, a version of the system that doesn't suck up a lot of desktop space is likely to be better.
The design challenge that creates is that this system depends on custom hardware and hardware fabrication follows the laws of physics and doesn't have quite the flexibility of code. There are lots of ways to manage this, but it is definitely a consideration in the design process.
This class of tools can help by bridging some of the bigger gaps in the process, allowing the animator to step over the line with a bit more confidence, directly editing data in the same sampled format it comes in with a robust toolset. When translation into sparse-key format for hand-key editing is still necessary, the process should be more graceful and creatively manageable.
A game production environment will have a different set of design parameters and so, naturally, an effective solution will be specialized as well. So, while I've included some images here to demonstrate the evolution of this class of tools, the actual tools that are most applicable to a specific game pipeline might be dramatically different.
I'm looking forward to figuring out what form that would take and how I can help your animation team reach a bit higher.
But I don't believe it really has to be that way.
(note: this is not as true of some limited-scope visually-focused indy games - which I've found to be mesmorizing)
Anticipation and overlap are concepts to focus on. Follow-through may be a bridge too far in this case, so let's stick a pin in that, but anticipation and overlap can generally be broken down into a fairly small number of objective rules and applied with existing parameters without muddying the waters of timing and complexity so much we lose responsiveness. When applied with care, those rules may not produce perfect animation always, but should produce better animation all or most of the time. They can fool most of the people most of the time.
In this case, we're mainly targeting autonomic issues - muscle physiology and low level nervous system control. It takes longer to get big muscles up to speed than small ones. When doing big things with an appendage, our low-level brain tends to lead big muscles first to get them ramping up and then bring in smaller muscle groups in series to make a specific move. Overlapping motion has other practical purposes, such as managing force on both ends of an action so we don't tear our ligaments apart any faster than is necessary, which is also a driver for anticipation. Having some parts lag behind helps distribute force at the start of a big motion.
For this type of effort, that correlation becomes particularly valuable. If we can just thread the needle in how we design an animation system so those relationships between art and engineer are not only recognized but embedded in the DNA of the system, the long-term value is likely to be significantly greater as we learn how to better apply and manipulate automation without sacrificing expression.
Thanks Pierre.
Each mode of movement has specific physiologic and/or behavioral rules. For example, a character considering switching direction (while standing still) doesn't need to move their feet if the adjustment is +/- 60 degrees or less, they can just turn their head and maybe make a small torso adjustment. Starting a walk from that position may require a different intro segment to get the legs and weight shift correct, but the complexity isn't crazy. Beyond 60 degrees things can get more complicated as emotional state/alertness starts to impact behavior significantly enough to matter even if the POV is a God-level 3rd person, but is still thoroughly objectively definable.
Getting those bits and pieces right is important. Players may not immediately notice the difference, but the character will feel more alive and when combined with other performance details, the whole will add up to a stronger performance.
When changing direction, the head almost always leads the torso, with the torso leading the body. Whether pupils lead or follow depends the priority of focus, but leads generally when someone is changing direction. This could pretty effectively be made procedural, following rules based on what the player is likely trying to look at. If the legs follow a different set of transition rules, but still consistent with the amount of orientation change, the overall impression is likely to look pretty natural.
There are secondary advantages to this approach. For example, procedural head moves could be triggered when there is activity within peripheral percepltion - sights or sounds - adding to positive performance cues without interfering with overall player control.
I believe that there is a natural limit to how far we can push game development and stick with a canned data model. But as long as we stay within that limit, we can still bring a lot of life to game characters, even in the pressure-cooker of in-game action.
Character animation is still problematic for film though. It's incredibly laborious, which has had a natural impact on its evolution, largely constrained to a niche market. At least until the advent of CG - now animoators get to branch out into mocap editing too.
Game development requires levels of dynamics that go way beyond film. There's a lot going on in a game enviroment and the characters need to live in that chaotic space and still be believable. Film development economics could really use a system that captures meaning in the motion data and streamlines production, but game development truly needs it.
After high school, before I realized that giving up art was a terrible idea for me, my major in college was computer science for a couple years. This was back when most computer work involved punch-cards. I did quite a bit of assembly language coding, along with early high-level languages. Then I changed paths, became an animator and watched the OOP revolution unfold from the 'outside'. Software engineers could suddenly develop ideas with a much stronger set of complexity-management tools. And it just kept evolving. It was an amazing thing to witness.
But I and my animator friends were all still doing animation with the same basic toolset, even after we made the jump to CG animation. Game projects pretty quickly became focused on mocap to address content productivity needs and performance requirements as engines and graphics systems improved. Now, a couple decades later, animators are still working with the equivalent of assembly and machine code. We accept it because it's all we have and we've spent so very much time learning to use it well.
I consider that pretty tragic. And unecessary. The artists and the industry deserve better. Yes, the artist still needs to be able be in full control, making sure everything moves just right, but we can still be control freaks with a more robust toolset. I doubt that many software engineers would choose to go back to coding search algorithms from scratch each and every time they need one. The concepts of encapsulation, abstraction, inheritance and polymorphism apply just as well to movement as they do to computational logic.
In my mind, the baseline character definition in a behavioral animation system should be a working in-engine equivalent of a character model sheet document, showing any animator who needs to work with a character how it should move. That baseline may be created from scratch or by applying deviations and details to an existing, more generic baseline character definition. Either way, the first step is to establish a solid artist-driven example.
In addition to other advantages, this can be compared to the difference between doing a quick one-off cast for a sculpture or doing investment casting when you know you will need to make many, many copies of a sculpt. In the case of a behavioral animation profile 'making copies' may mean handing the animation to other animators on the team or in other arenas (e.g. outside studios or for marketing efforts). The IP of the movement is baked right in, as much as the model and texture. A significant amount of IP control over character motion goes with the character.
For production, in this should significantly lower the amount of low-level animation decision-making that has to be done as the content scales up, and for the equivalent of re-targeting to similar-but-different creature versions. It should also add layers to the advantages of retargeting and helps keep the animation team focused more on high-level acting than low-level keyframe matching.
On some past projects I haven't been able to corral off-the-shelf engines to support this effectively and permit non-linear triggering and selective overriding and transition-blending of groups with overlapping joints, so I had to have a custom playback engine developed. That level of control becomes pretty critical when you start managing an entire performance with procedural animation. I don't believe either Unreal or Unity support that feature set natively, but I believe it will be needed.
Most of the time, the animator would be focused mainly on the higher two levels, with indications of the impacts of influence from the lower two levels and a means of overriding or allowing their direct influence.
But to be honest, I'm just throwing up words - experimentation and more than a couple brains will need to mix together to settle on a definition that actual makes sense. I just feel that the system will probably have to work for a pretty wide variety of representational models.
Abberant behavior also exists of course, and some people and animals exhibit behavior that isn't strictly 'normal', which can be seen as wasting energy, but they don't really do that by choice, so I tend to put it back in the autonomic category.
The down side is that when a character is active - like during most gameplay, there can be a dozens to hundreds, maybe thousands, of independent behaviors strung together and overlapping each other before anything like a real rest occurs. So we still need a robust way to link everything. Still, the idea of keeping track of appendage action without rest may facilitate some interesting ways to handle exhaustion, strength, accuracy, etcetra during play.
For all the animators that might use this type of system in the future, I - we - get to be the baby.
My best guess is that there just weren't many character animators in the room when the software engineers built early keyframing editing tools and appropriated the term. But that implementation doesn't really represent what keyframes really were historically.
Cel animation isn't exactly like CG animation, and much of that notation was done in a timing chart was done to simulate things very much like a well-placed B-spline which practically happens automatically with CG. But identifying meaningful moments that don't necessarily correlate to specific data points is likely to really important to this effort.
Then each of those timing reference classes can be defined relative to any and all applicable anatomical groups and in terms of motion durations before and after for more robust transition processing. It's the beginning of a motion markup language that can be integrated into a high-level procedural scripting system, allowing us to strat to blur the line between procedural and static data types.
In that situation, humans were communicating to other humans, so quite a lot could be done with shorthand notes. Here we are trying to communicate performance-critical information to a machine, so we need to be pretty explicit.