Generate Skeleton

Requirements

Input Streams: Mesh stream and texture stream (REQUIRED)

Output Streams: Animation stream

Important

This stage uses images to identify human poses. Be sure your texture stream contains accurate textures before running Generate Skeleton.

Overview

The Generate Skeleton stage adds rigging data, commonly called a “Skeleton”, to your volumetric clip. The stage attempts to match this skeleton as closely as possible to the provided mesh stream on each frame. The resulting animation stream can be reviewed in HoloEdit and exported for editing in external software using the “Maya” or “Animation” export types.

This stage works by capturing multiple images, called “views”, from different perspectives surrounding the clip each frame. Using those images, it produces a 3D skeleton based on the 3D figure recognzied in those images. In complex poses, where one or more parts of the actor might be obscured from most possible camera angles, you can increase the “Number of Views” setting to generate more unique views automatically spaced around the clip.

Note

After this stage has finished processing, it’s normal to see slight frame to frame jitter in your final skeleton. You can use the Stabilize Skeleton stage to smooth out jitter.

Warning

This stage will not function correctly following a Compress Mesh Segments stage.

Parameters

  • Autocorrect Left Right Inversions: Toggle to enable Left Right Inversion. This setting will automatically detect possible cases where the left and right limbs of the skeleton have been flipped (resulting in the leg or arm bones being stretched to the wrong side) and correct them. This may produce false positives when used on areas that were correctly identified

  • View Background Color: Set a background color for the images used by the Skeleton Generation process. If the default color, “magenta”, or a color very close to it, is present on your actor, change this to a unique color to improve skeleton accuracy. Accepts hex codes that begin with “#”, or color names from the following list: red, cyan, blue, darkblue, lightblue, purple, yellow, lime, fuchsia, white, silver, grey, black, orange, brown, maroon, green, olive, navy, teal, aqua, magenta

  • Number of Views: Sets the number of 2D images used to calculate the skeleton pose. Higher numbers might improve generation in some cases, but will significantly increase export and processing time

  • Autoframe view: When enabled, the views used for skeleton generation will be framed to ensure the mesh is visible

  • Autoframe Height Bias: Factor to scale the mesh bounds used for the autoframe view option. Larger numbers will result in images further away from the capture, which can correct common skeleton generation issues such as the head or feet bones generating significantly out of place.

  • Pose Model: Determines the Pose Estimation Model used in skeleton generation. The default setting “BODY_25” is recommended in most cases. The older model “COCO_18” is available if necessary, but typically results in lower quality and accuracy

Working with Generate Skeleton

The “Autoframe view” and “Autoframe Height Bias” settings determine how those views are positioned.

With “Autoframe view” enabled, HoloEdit will automatically center the clip in the images for best results. Sometimes, this may result in the captured views being too close to the model, and extremities such as feet, hands, or often the head will be out of frame in some or all views. This can result in significantly misplaced joints for extremities.

To resolve this issue, try raising the “autoframe height bias” setting gradually from the default setting to 1.5 or even 2.0. Note that higher autoframe height bias settings will produce images further away from the capture, which can sometimes produce less precise skeleton positions.