Animating Lip-Sync
Adding a lip-sync to a project can really enhance its quality and storytelling. However, it can be difficult to shape a character's mouth to match the sound at a precise frame.
To solve this problem, Harmony provides a lip-sync feature which analyzes the contents of a sound element and generates a mouth chart (see below) based on the eight animation phonemes (A, B, C, D, E, F, G, and X for silence).
The mouth shapes used by Harmony are based on the conventional mouth chart used in the animation industry.
Here is an approximation of the sound each mouth shape can produce:
• | A: m, b, p, h |
• | B: s, d, j, i, k, t |
• | C: e, a |
• | D: A, E |
• | E: o |
• | F: u, oo |
• | G: f, ph |
• | X: Silence, undetermined sound |
You can lip-sync the traditional way or let the system automatically create the basic detection.
You can refer to the mouth chart positions as you draw the shape of the character's mouth.
- In the Timeline or Xsheet view, select the sound layer.
The options for that layer will appear in the Layer Properties view.
- In the Layer Properties, click Detect.
A progress bar appears while Harmony analyzes the selected sound clips and assigns a lip-sync letter to each sound cell.
- Click the Map button to open the Lip-Sync Mapping dialog box.
- From the Destination Layer menu, select the layer that contains the mouth positions for the character's voice track.
- If the selected layer contains symbols, you can map the lip-sync using drawings located directly on the layer or use the symbol's frames. In the Symbol Layer field select Don't Use Any Symbol if you want to use the drawings or select the desired symbol from the drop-down menu.
- In the Mapping section, type the drawing name or Symbol frames in the field to the right of the phoneme it represents. If your drawings are already named with the phoneme letters, you do not have to do anything.
- Click OK.
- Press the Play button in the Playback toolbar to see and hear the results in the Camera view. To play back your scene with sound, enable the Sound button in the Playback toolbar.
Harmony allows for the manual swapping of mouth position drawings to match a voice track. For this process, both sound scrubbing (listening to a sound wave broken up frame-by-frame) and drawing substitutions are used.
- In the Playback toolbar, enable the Sound Scrubbing button.
- In the Timeline view, drag the red playhead along the sound layer waveform.
- When you reach a frame where a mouth position should go, for example, an open mouth with rounded lips for an "oh" sound, click on that frame on your mouth shapes layer.
- In the Parameters section, staying on your mouth shapes layer, place your cursor on top of the drawing name (often a letter) until it changes to the swapping pointer.
- Pull the cursor to see the list of mouth shape names and choose the one you want. The current drawing automatically changes to the new selection.
- In the Playback toolbar, click the Sound Scrubbing button.
- In the Timeline view, drag the red playhead along the waveform of your sound layer.
- When you reach a frame where a mouth position should go, for example, an open mouth with rounded lips for an "oh" sound, click on that frame on your mouth shapes layer.
- In the Drawing Substitution window of the Library view, drag the slider to choose a mouth shape. The current drawing is swapped for the one in the preview window.