Blend Mesh (Beta)¶
The Blend Mesh stage is currently in Beta. Please be aware that not all sources will blend smoothly. For best results, clips you hope to blend should be somewhat similar in form and volume.
Requirements¶
Input Streams: Two mesh streams, two texture streams
Output Streams: Modified mesh stream, original or modified texture stream
Overview¶
Note
Blend Mesh uses the stabilization algorithm. For in-depth information on deformation, see the Deformation algorithm page.
The Blend Mesh stage takes the mesh from two source stages and creates a blended result. Textures can either be transferred from the original source stages, or also blended between the two sources.
Two inputs are required for the Blend Mesh stage to process, Source Stage A and Source Stage B. Source Stage A should be higher in the track hierarchy than the stage being used for Source Stage B.
When processing, the Blend Mesh interval will be divided into two sections: The first half of the interval uses the topology of Source Stage A for its deformation, while the second half of the interval uses the topology of Source Stage B.
How much each frame should be blended between the surface pose of Source Stage A and Source Stage B is determined by the Blend Curve Shape, Blend Curve Start and End Values. At default settings the first frame of the blend interval will prioritize the surface shape of Source Stage A. Over the course of the Blend Curve it will change from matching the surface of A to a blend of the two surface shapes, finally ending by prioritizing the surface shape of Source Stage B.
When executed Blend Mesh will deform each Source Stage frame, applying the level of interpolation required based off of it’s position along the Blend Curve.
Note
This stage should be run on a cleaned mesh stream, with degenerate triangles fixed, and a polygon count under 100k triangles. Higher triangle counts are supported, but compute hour costs may increase dramatically. For higher triangle count meshes, start with a higher ‘Sample Radius Factor’, or lower ‘Hierarchy Node Sampler Max Nodes’.
Parameters¶
Source Stage A: Click and drag a stage from the current track into this field to use as the first source stage used for the blend.
Source Stage B: Click and drag a stage from the current track into this field to use as the second source stage used for the blend.
Blend Curve Shape: Determines the curve of the pose influence between the blended stages. Where 0 is fully prioritizing the pose of Source Stage A and 1 is fully prioritizing the pose of Source Stage B. Use the dropdown menu to select different curve shapes.
Blend Curve Start: A value between 0 and 1 that determines the weight of importance of the blend source stage pose, where 0 is fully prioritizing Source Stage A’s pose, and 1 is fully prioritizing Source Stage B’s pose. Changing this value will affect the left edge of the blend curve. Default starting value is 0. When an interval containing a blend curve is cut using the Razor tool the values within the new intervals will automatically update to preserve the original blend curve.
Blend Curve End: A value between 0 and 1 that determines the weight of importance of the blend source stage pose, where 0 is fully prioritizing Source Stage A’s pose, and 1 is fully prioritizing Source Stage B’s pose. Changing this value will affect the right edge of the blend curve. Default starting value is 1. When an interval containing a blend curve is cut using the Razor tool the values within the new intervals will automatically update to preserve the original blend curve.
Texture Blend Mode: Select texture processes for blend from None (default) and Cross-fade
None directly passes the original textures from the source stage whose topology is being used as the base mesh for the blend’s deformation. This transfer method should be used in most cases.
Cross-fade performs a closest point texture transfer for each of the two surfaces. The two textures are then overlaid and blended based on the frame’s position in the blend curve.
Progressive Vertex Pairs: This setting provides faster stabilization by performing initial deformations with fewer potential Correspondence Pairs. Recommended for all clips.
Match Cosine Limit: Directional limit for pairs. The ‘Match Cosine’ is calculated by taking the Dot Product of the Normal Vector of each vertex in the potential pair. Potential pairs whose dot product is below this value will be ignored. Decrease this value for blends with more variance in the source poses being blended.
Match Distance Limit: Distance limit in meters for pairs. Potential pairs whose distance exceeds this value will be ignored. Increase this value for blends with more variance in the source poses being blended.
Use Hierarchy Node Sampler: Enables/disables Hierarchy Node Sampler. More information on the Hierarchy Node Sampler can be found in the Deformation Algorithm Document
Hierarchy Node Sampler Max Nodes: Determines the maximum number of starting nodes used for deformation when the Hierarchy Node Sampler is enabled. This value will start node creation as a percentage of the number of vertices in the mesh. This parameter only affects deformation if Use Hierarchy Node Sampler is enabled
Sample Radius Factor: This setting influences how many Deformation Nodes will be created for the Deformation process. This setting is multiplied by the average edge length of the mesh to determine the “Sample Radius”. Nodes are evenly distributed based on Sample Radius, so higher Sample Radius values create fewer nodes, and lower Sample Radius values create more nodes. Higher values will typically result in faster, less accurate stabilization. This will only be used if the Hierarchy Node Sampler is disabled.
Working with Blend Mesh¶
The Blend Mesh stage requires two input stages to blend between. These inputs can be from the same clip imported into the composition twice or two different clips, each imported into the same composition.
The Blend Mesh stage combines the two source stages using the same deformation algorithm as the Stabilize Mesh and Match Template Mesh stages. For More information, please review the Deformation Algorithm documentation.
Mesh samples being blended should be processed first using the Clean Mesh stage to avoid risk of deformation failures while blending.
It’s important to consider the clips being blended and what the most natural place to blend them is. Algorithmic mesh blending is not performed the same way an actor would naturally change poses; finding key limb positions, body position, posing and timing are all important considerations that should be taken into account when blending or looping clips.
Aligning foot positions in a separate track before processing can help prevent your capture from feeling ungrounded or floaty during a blend. Planning so that the general movement between the source stages’ is going in parallel directions will help achieve a more natural flow. Ensuing there is no crossing or overlapping movement in the limbs will help prevent confusing results or loss of details in important areas.
Once you’ve found the portions of frames that will be blended create the Blend Mesh stage, then create an interval containing the frame range you will be blending.
With the Blend Mesh stage selected and the Inspector panel open, click and drag the stage containing the mesh stream (that will lead into your blend) into the Source Stage A field, next click and drag the stage containing the mesh stream (that will lead out of your blend) into the Source Stage B field.
Make adjustments to the stabilization parameters as needed per your specific data.
Note
Keep in mind that with many blends and loops it’s possible that there may be more variance between the mesh being deformed to the target than standard Stabilize Mesh jobs. Please increase the Match Cosine and Match Distance Limit for blends that have higher levels of surface positions between the source stages and selected blend frames.
Textures with the Blend Mesh stage have two options: None and Cross-fade. In most cases we suggest using ‘None’ as there is significant risk of ghosting overlap of the blended textures with the cross-fade method. None will apply the existing textures from the source stages. Cross-fade texture blend mode works best in the case of a transition between two widely different clips where a single frame transition of textures would be more obvious.