Semantic Deformation

Semantic Deformation

Ilya Baran, Daniel Vlasic, Eitan Grinspun, Jovan Popović SIGGRAPH 2009 Reading group presentation (and nothing at all to do with the real guys who did this at MIT) – twak

Animation is Expensive ●

Artist time is expensive

●

Job is thankless, unrewarding etc...

●

Common solution is outsourcing

●

●

Large mocap libraries exist with lots of movement data that we could reuse But how to apply that mocap data to different models?

Skeletal Retargeting Vertices are mapped to a weighed set of frames ● Animation only moves frames ●Splice many small animation clips

●

“a skeletal model may not be able to capture the full subtlety of the poses”

Example-based control of human motion, Hsu et al., 2004

Deformation Transfer

●

Literal mapping from target to source instead of “semantic correspondence”

●

Arbitrary vertices in input and output model

●

User specified correspondence map

Mesh Modification Using Deformation Gradients, Robert W. Sumner 2005

Semantic Deformation Transfer

Given a set of pose correspondences between two meshes...

...and an arbitrary pose as input...

...what's the output?

Method

Create a space that can be manipulated via standard linear algebra techniques, so we can apply these techniques to targeting meshes.

Projection - just find the weights!

[

x 1 , x 2 ... x n

][ ] [ ] w1 w2 = p ... wn

x, p are given and projected poses w are weights

The End

Shape Space

"For semantic deformation transfer to work, the shape space must satisfy two requirements: linear interpolation between points in the shape space must produce blended poses without artifacts, and projection of a pose onto a subspace of the space must produce the most similar pose in the subspace to the original."

Why not an existing space?

All likely points in space should correspond to desirable deformations of the model Matrix animation and polar decomposition, Ken Shoemake & Tom Duff, 1992

Three techniques

●

Matrix logarithms

●

Deformation Gradients

●

Linear Rotation-invariant Coordinates

Interpolating Rotations We can decompose an arbitrary transform into rotation and skew

M =QS

Matrix animation and polar decomposition, Ken Shoemake & Tom Duff, 1992

Neat trick to find rotation matrix:

Q 0= M −T Q i Q i Q i1= 2 Can parameterize to interpolate Q between two rotations. But still doesn't help – we need one matrix we can linearly interpolate

Matrix Logarithms ●

●

“Provide a mapping between the group of 3D rotation SO(3) and the Lie algebra so(3) of skewsymmetric 3x3 matricies” so(3) allows linear interpolation

exp  ∑ w i log Q i

Mesh-Based Inverse Kinematics, Sumner et al., 2005 A Mathematical Introduction to Robotic Manipulation, Murray, 2002

Deformation Gradients ●

Translation insensitive

●

Stores face rotation relative to original model

●

Linear system must be solved for reconstruction

v2

v1

v1

v4 v2

v4

KV = V v3

v3

V =[v 2−v 1 , v 3−v 1 , v 4−v 1 ]

V =[ v2 − v1 , v3− v1 , v4 − v1 ]

K =V V

−1

Deformation transfer for triangle meshes, Sumner et al. 2004

Problems with Deformation Gradients

When interpolating each face may take a different rotation.

Linear Rotation Invariant Coordinates

Linear rotation-invariant coordinates for meshes, Lipman et al., 2005

●

Rotation invariant mesh representation

●

Each vertex encodes the location of it's neighbors

●

Cylindrical coordinates

●

Solve a pair of linear systems to reconstruct ●

Absolute Frame Orientations

●

Vertex Positions

Here's a video from a similar technique Free-Form Motion processing, Kircher & Garland. , 2008

Properties of LRI ●

LRI accumulates noise over the mesh

●

Much faster than other non-linear techniques

●

Linear interpolation fails if polar coordinates stored naively

Putting it all together ●

●

●

● ●

Now we want to combine these techniques to create a linear space with the desired properties. It's got to be rotation invariant (for arbitrary input meshes)... ...so we store the differences between a set of different coordinate systems. And we want “semantic correspondence”... ...so we'll use deformation coordinates relative to patches defined on the input and target models.

Input mesh

Feature Vector

Output Mesh

Reconstruction ●

Recover patch frames from adjacencies

∑ ∥G i −G j G i , j∥ ●

Solve for G's

Recovering Verts ● ●

Deformation Gradients from G's and S's Not vertex positions, so solve minimizing distance from base pose

∑ ∑ v f

●

j

' i j1

' ij

2

−v −Q f S f  v i − v i  j1

(f's are faces, j's are vertex indexes, v' are reconstructed and tildes are rest poses)

j

Patches!

Criticism ...we expect to end up with...a crocodile swing dancing as if it were an adult human. Admittedly, this faithfulness to the original motion is not always artistically desirable. However, we prefer to relegate the difficult creative decisions (How do crocodiles dance?) to the user's selection of an initial motion. Retargetting motion to new characters, Gleicher, 1998

●

Engineered accumulation of other people's work

●

Feet don't stick to floor

●

●

●

●

●

Skeletons have hardware support, deformation gradients don't Needs to have a good set of base poses and their correlating transform Doesn't show re-encoding the base pose? Assumes topology invariant (people don't grow another arm) No exploitation of temporal similarities