Lip-sync

Adding a lip-sync to a project can really enhance its quality and storytelling. However, it can be difficult to shape a character's mouth so it matches the sound at the precise frame. To solve this problem, Studio provides a lip-sync feature which analyzes the content of a sound element and generates a mouth chart based (see below) on the eight animation phonemes (A, B, C, D, E, F, G, and X, which is used to represent silence). You can lip-sync the traditional way or let the Studio automatically create the basic detection.

The mouth shapes used by Studio are based on the conventional mouth chart used in the animation industry. The letters used to represent the shapes do NOT correspond to an actual sound.

Here is an approximation of which sound each mouth shape can produce:

Mouth Shape

Approximate sound the mouth shape matches to

A

b, m, p

B

d, h, i, j, k, s

C

a, e

D

A, E

E

o

F

oo, u

G

f, ph

X

Silence, undetermined sound

You can refer to the mouth chart positions as you draw the shape of your character's mouth.

This section explains:

Modifying the Lip-sync Detection
Creating a Lip-sync Using a Mouth Template
Creating a Lip-Sync Using Your Own Drawings
Lip-sync
Automatic Lip-sync Detection
Modifying the Lip-sync Detection

Lip-sync View

The Lip-sync view is where you can create and map mouth charts, as well as import mouth templates for your characters.

To open the Lip-sync view:

1. Select a sound layer from the Timeline view or a cell in the Exposure Sheet view.
2. Do one of the following:
From the Sound toolbar (Top toolbar on Mac OS X), click the Lip-sync button.
In the Properties panel, click the Lip-sync button.

Creating a Lip-sync Using a Mouth Template

Studio comes with a variety of mouth templates that you can play and experiment with on your characters. You can create a lip-sync and use an existing sound layer or create the lip-sync and then import the sound.

To create a lip-sync using a mouth template:

1. From the Sound toolbar, click the Lip-sync button (Top toolbar on Mac OS X).

2. Do one of the following to add sound:
Click the Import Sound button. From the Open dialog box, select a sound file and click Open.
From the Sound list, select a sound file you have already imported.
3. If needed, edit the sound by clicking the Edit Sound button and making any adjustments in the Sound Element Editor.
4. Click the Import Template button.

The template dialog box opens to display the mouth templates.

5. Select a mouth template and click OK.

In the Timeline view, a Drawing layer is created containing eight mouth drawings.

6. In the Lip-Sync view, click Apply to generate the lip-sync with the sound file.

The mouth shapes are synced with the sound file.

7. In the Drawing view, use the Select tool in the Animation Tools toolbar to scale the mouth drawings on the layer.
8. In the Camera view, place the mouth at the correct location on your character.

Creating a Lip-Sync Using Your Own Drawings

You can use your own drawings to generate a lip-sync.

To create a lip-sync using your own drawings:

1. In the Timeline view, select the drawing layer that contains your eight mouth drawings.
2. From the Sound toolbar (Top toolbar on Mac OS X), click the Lip-sync button.

The Lip-sync view displays. Your eight mouth drawings are automatically mapped with the animation phonemes.

3. Do one of the following to add sound:
Click the Import Sound button. From the Open dialog box, select a sound file and click Open.
From the Sound list, select a sound file you have previously imported.
4. If needed, edit the sound by clicking the Edit Sound button and making any adjustments in the Sound Element Editor.
5. If needed, you can adjust the mouth mapping by clicking the Previous Drawing and Next Drawing arrows beside any of the mouths.

6. Click Apply to generate the lip-sync with the sound file.

The mouth shapes are synced with the sound file.

Lip-sync Detection

You can detect the lip-sync for a sound in the Exposure Sheet view or Sound Element Editor.

To detect lip-sync for a sound in the Exposure Sheet view:

1. In the Exposure Sheet view, right-click on a cell of the sound column and select Show Lip-sync.

Studio analyzes the selected sound clips and assigns a lip-sync letter to each sound cell. The number in each frame is replaced by a letter that represents the mouth shape.

You can enable the Thumbnails display option to show a preview image of the mouth shape. Once you enable the Show Lip-sync option, click the Thumbnail button in the Exposure Sheet View toolbar.

To detect lip-sync for a sound using the Sound Element Editor:

1. In the Timeline or Exposure Sheet view, select the Sound element from which you want to generate a chart of mouth shapes.
2. In the Sound toolbar or Properties panel, click the Edit Sound button.

The Sound Element Editor opens.

3. In the Sound Element panel, select the waveform from which you want to generate the mouth chart and click Lip-sync.

A progress bar appears as Studio analyzes the selected sound clips and assigns a lip-sync letter to each sound cell.

4. Click the Play button to hear the sound clip.

The Lip-sync Preview area shows a preview of the result.

Automatic Lip-sync Detection

Studio can automatically map drawings in an element to the mouth chart you have generated for a sound using the Lip-sync button in the Sound Element Editor dialog box. This can save time when lip-syncing a voice track.

This means that in the Lip-Sync Mapping dialog box, you can identify each lip drawing of a character mouth element. Studio then automatically labels all of the cells in the character's element with the appropriate name.

To automatically map lip-sync drawings to a mouth chart:

1. In the Exposure Sheet view, right-clickon the header of the sound element was already lip-synced and select Modify Lip-sync Mapping.

The Lip-Sync Mapping dialog box opens.

Source element: The Sound element from which the lip-sync mapping will be based.
Destination element: Lets you select the element that contains the mouth shapes of your character.
Mapping: In this section, type the destination drawing name in each field corresponding to the mouth phoneme that the thumbnail on the left represents. If your drawings are already named using the standard phoneme letters, you do not need to change anything.
2. Click OK.
3. Click the Play button in the Playback toolbar to see and hear the results in the Camera view.
4. To play back your scene with sound, enable the sound if you not already done so. From the top menu,select Play > Turn On Sound Playback.

Modifying the Lip-sync Detection

If you think that another position would be more appropriate, you can change the mouth position that has been assigned to a frame.

For example, you may have a character who remains silent for ten frames between two speeches. Studio would normally assign an X image to the silence. However, if you want your character's mouth to hang open in astonishment for these ten frames, you can change the lip assignment for these frames from an X to an E.

You can modify the lip-sync from the Exposure Sheet view or Sound Element Editor.

Modifying the Lip-sync in the Exposure Sheet View

To change the lip assignment of a sound in the Exposure Sheet:

1. In the Exposure Sheet view, right-click in the cell that contains the mouth position you want to change.
2. Select Lip-sync > A, B, C, D, E, F, G or X. Choose the letter that reflects the mouth position you want to use for that sound.

The drawing automatically updates in the Exposure Sheet and Timeline view.

Modifying the Lip-sync in the Sound Element Editor

To change the mouth position assignment of a sound in the Sound Element Editor:

1. In the Timeline or Exposure Sheet view, select the Sound element with the lip-sync you want to modify.
2. In the Sound toolbar or Properties panel, click the Edit Sound button.
3. In the Sound Element panel, drag the red frame slider to the frame with the mouth position you want to change.
4.

In the Lip-sync image area, the image on top represents the lip position currently assigned to the frame.

5. To change the image assigned to the frame, select a mouth position below the preview image.

The preview image changes to the mouth position image you selected. The position is also updated in the Exposure Sheet and Timeline views.

Recomputing the Mouth Chart

Recomputing the mouth chart erases any manual modifications you made to the mouth assignments in the mouth chart.

When you generate the mouth chart for a sound in a sound element, you can either accept the mouth positions assigned by Studio or assign your own lip-sync images.

However, if you change the sound's start frame, edit it or reassign its mouth position afterwards, you must reanalyze the sound, and regenerate the mouth chart for it.

To recompute the mouth chart for a selected sound in the Exposure Sheet:

Remember that recomputing erases modifications (see warning above).

1. In a Sound column in the Exposure Sheet, right-click on a cell that contains the sound file you want to recompute and select Recompute Lip-sync.

The mouth position assignment for each frame is recomputed, erasing any modifications you made.

To recompute the mouth chart for a selected sound in the Sound Element Editor:

Remember that recomputing erases modifications (see warning above).

1. In the Timeline or Exposure Sheet view, select the Sound element.
2. In the Sound toolbar or Properties panel, click the Edit Sound button.

The Sound Element Editor opens.

3. In the Sound Element panel, select a waveform to recompute and click Lip-sync.