Visual Effect Tutorial

SM4123 Procedural Animation

Workshop 03: Video- and Audio-driven Animation Video-driven animation

We use image data to control 3D elements

In this exercise we will use the pixel color of a video to control the 3D elements. First of all, you have to understand the basic workflow: Step 1) Under /img catagory, we construct an Image Network (called a COP2 network). In Houdini, an Image Network is used to manipulate 2D images. We will see the details soon.

Step 2) Whenever you want to grab the pixel data to control a parameter (say, the xcenter of a SPHERE OP), you can give this expression to the parameter: pic( a, x, y, d ) where a is the name of OP in the Image Network; x and y specify the location of the pixel (in the range [0-1]); d is one of the following: D_CR, D_CG, D_CB, D_CA, D_CHUE, D_CSAT, D_CVAL, D_CLUM for the red, green, blue, alpha, hue, saturation, value, or luminance of the pixel.

1

SM4123 Procedural Animation

A step-by-step example Step 1) First you need a video-clip. Note also that Houdini does not read video file. You have to convert it into an image sequence using other software, such as Adobe Premiere. In this exercise, you are given a sequence of BMP files under the W:\SM4123\week03\image_sequence. Note also the naming of those files. Next, you can go to the /img category, and choose a Composite View to view the 2D image processing result (instead of using a Scene View).

By default, Houdini creates an empty Image Network called “comp1” for you. You can use it, or you can create your own one. Go into the Image Network by double-clicking it. We are going to construct an Image Network (a COP2 network) to manipulate the image sequences.

Step 2) Press TAB and create a FILE OP. Give your image sequence file names (and don’t forget turning on the blue-flag of the OP):

Note that this FILE OP is called /img/comp1/dancerF3_bmp if you haven’t renamed it. You will need this name in the later steps.

Step 3) Now, go back to /obj. Create a GEOMETRY and go into it. Construct some shape for this geometry. In this simple exercise we will create a grid (with 30x30 points), and copy spheres onto the grid.

A SPHERE with radius 0.02.

A GRID with: Orientation: XY plane Size: 1 and 1 Rows:30 Columns:30 Center: 0.5, 0.5, 0

A POINT OP for modifying the points of the grid.

Recall that whenever you want to grab a pixel’s color, you can use the expression: pic( COP2 name, pixel x-location, pixel y-location, pixel color ) Say, for example you want to use the luminance of a pixel to modify the grid point’s position. To modify a point’s attribute, we can use a POINT OP. In Houdini, POINT is an useful OP: it can be used to modify points’ position, color, alpha, normal, etc. Let’s try this: In the z-position of the POINT OP, type the following: 02 * pic("/img/comp1/dancerF3_bmp", $TX, $TY, D_CLUM)

2

SM4123 Procedural Animation

Note that $TX and $TY are two internal (given) variables: $TX is the x-position of the point, and $TY is the y-position of the point, both of them are within [0,1] because of our GRID’s setting. Therefore, they can be used for the pic() function. The luminance of the “corresponding” pixel will then be used to control the z-position of the grid points. You can play the animation to see the effect.

Exercise 1 Continue working on the previous example, try to set the color of the spheres based on the “corresponding” pixel’s color.

How to make it better? You may want to delete some points that are not interesting enough, say, the points with luminance equals to black color. You can use the DELETE OP. To use the DELETE OP, you have to specify which points you want to delete. Say, we can use an expression to specify that “delete all points with $TZ equals to zero”:

Note that in this example we assume the $TZ will be linked to the luminance. If not, you may need to use, say, “($CR + $CG + $CB)/3 < 0.01”, where $CR, $CG and $CB are the red, green and blue color of the point.

We set the size of GRID to be 1x1, with center at (0.5, 0.5), so that the variable $TX and $TY are in the range [0-1], exactly match what we want to give to the expression pic(). However, if we scale up the GRID size (or move the grid’s center), the $TX and $TY are no longer in the [0-1] range. In this case, a better solution is to use the variable $BBX and $BBY (the location of the point with respect to the bounding box of the object). No matter how large or small your GRID is, the $BBX and $BBY will always be in the range [01]. So, in the POINT OP, you can replace: 02*pic("/img/comp1/dancerF3_bmp", $TX, $TY, D_CLUM) by 02*pic("/img/comp1/dancerF3_bmp", $BBX, $BBY, D_CLUM) Now you can scale (or even change the center of the GRID) to whatever value, without affecting the expression formula.

3

SM4123 Procedural Animation

Moreover, I will suggest in the Image Network, add a NULL OP after your FILE OP, and then modify all the expression: pic( “/img/comp1/dancerF3_bmp”,……) to pic( “/img/comp1/null1”, ……) In doing so, you can insert any number of OP between the FILE and NULL, for doing some 2D image processing, without need to modify your expressions on the SOP side. For example, you can insert an INVERT to invert the color of the image sequence:

4

SM4123 Procedural Animation

Audio-driven animation

We use audio data to control 3D elements

In this exercise we will use the volume of an audio clip to control the 3D elements. It contains the following workflow: Step 1) Under /ch, we construct a CHOP Network. A CHOP network is used to manipulate channel. The “channel” in Houdini means “value changes over time”. Audio data is also treated as channel in Houdini.

Step 2) Whenever you want to grab the audio data to a parameter, you can use the expression chopcf( a, b, c ) where a is the name of OP in CHOP; b is the channel number; c is the frame number.

5

SM4123 Procedural Animation

A step-by-step example Step 1) First, you need an audio-clip. Houdini supports AIFF, WAV and MP3 format. I have put a WAV file under W:\SM4123\week03\sound1.wav. You should go to the /ch category, and create a CHOP Network. By default, the newly created CHOP network is called /ch/ch1.

You can also create a Motion View tab for viewing the CHOP data:

Go into the /ch/ch1, press TAB and create a FILE OP. Locate your audio file, and turn on the blue-flag:

Note that this FILE OP is called /ch/ch1/file1 if you haven’t renamed it. You will need this name in the later steps. Moreover, middle-click the FILE OP, you may find that there are two channels inside this OP (it is because my audio file is stereo). They are called “chan0” and “chan1”. The “chan0” is the channel #0, and “chan1” is the channel #1. You also need the channel number later.

6

SM4123 Procedural Animation

Step 2) Now, go back to /obj. Create a GEOMETRY and go into it. In this example we use a simple SPHERE. Under whatever parameter you like (say, the y-center of the sphere), type the following expression: chopcf( “/ch/ch1/file1”, 0, $F )

Now, the volume of the audio data is connected to the y-center of the sphere. If you want to hear the audio at the same time, you should do the following: Turn on the real-time playback option:

Then, open the Audio Panel Window:

Turn on the “Scrub” option, and choose your CHOP OP:

7

SM4123 Procedural Animation

Let’s try a more complicate example: a row of boxes, each access the audio data at a different frame.

A BOX with center y = 0.5

A GRID with rows=1, columns=10.

Use 5 * chopcf(“…”, 0, $F-$PT) to make different copies of the boxes grabs different frame of the audio clip.

How to make it better? Similarly, I will suggest adding a NULL OP under your FILE OP, and modify all the expression: chopcf( “/ch/ch1/file1”, ……) to chopcf( “/ch/ch1/null1”, ……)

In doing so, you can insert any number of CHOP OP between the FILE and NULL, without need to modify your expressions on the SOP side. For example, you may want to insert a DELETE OP to delete the un-used channel:

Then, you want to insert a LAG OP, which can be used to “smooth” out the audio data:

8

SM4123 Procedural Animation

Lastly, you may also want to insert a RESAMPLE OP, which re-sample the audio data, i.e. to reduce the number of data without affecting the result on the SOP side:

Final version: this is an example of “reduce and smooth” audio data:

After DELETE

After LAG

After RESAMPLE

CHOP provides some other OPs for processing audio data. If you are familiar with audio and sound, you can try to explore more possibilities. Exercise 2 Try to apply the above techniques, and also using PARTICLE, to create the following audiodriven effect (the animation will be shown during the class).

Hints: I used chopcf() to control the y-scale of the TUBE OP, and “Birth”, “Force (y-axis)” and “Turbulence” of the PARTICLE OP.

** Week 03 END **

9