Stabilize Mesh¶
Overview¶
Note
Stabilize Mesh uses a complex ‘Deformation’ algorithm to produce new poses. For in depth information on deformation, see the Deformation algorithm page.
The Stabilize Mesh stage automatically converts your clip into one or more “Stabilized Segments”, a kind of geometry that empowers fast editing and powerful Compress Mesh Segments compression.
When you execute a Stabilize interval, HoloEdit chooses one or more keyframes and begins deforming them to produce Segments, with the goal of all frames in the interval being captured in the fewest number of segments as possible.
A Stabilized Segment is a series of frames with a single consistent topology for every individual frame, or pose.
Most clips in HoloEdit should be stabilized. Without stabilization, advanced compression and editing features inside and outside of HoloEdit are unavailable.
This stage should be run on a cleaned mesh stream, with degenerate triangles fixed, and a vertex count under 100k, for faster processing. Higher vertex counts are supported, but compute hour costs may increase dramatically. For higher vertex count meshes, start with a higher Sample Radius Factor.
Parameters¶
When configuring the Stabilize Mesh stage, keep in mind that the ultimate goal is long segments with acceptable pose quality for your own project’s needs. Depending on your clip and final playback platform, different levels of accuracy and compression may be warranted.
Also, it’s important to pay close attention to your source meshes if you have very strict requirements for your result. Ultimately, the Stabilize Mesh stage is limited by the source data. If your meshes are very noisy from frame to frame, or if your actor is moving very quickly, there may be limits to the length of high quality segments.
Warning
Stabilization is a complex process and can result in some loss of quality in some clips. On many clips, you can run Stabilize Mesh with default settings to produce a desirable result. However, for more complex clips, please read the “Working With Stabilize Mesh” section below.
The parameters provided for the Stabilize Mesh stage can be roughly categorized into three groups:
Deformation Weights, like alphaRigid, alphaSmooth, alphaPoint, and alphaPlane. These settings influence how the keyframe is deformed, to help conform it to the target mesh.
Correspondence Thresholds, like Match Distance Limit and Match Cosine Limit. These settings influence which vertices are chosen as pairs to guide the deformation process to correctly capture the pose.
Node Allocation, like Sample Radius Factor and Hierarchy Node Sample parameters. These settings influence the size and amount of nodes used during deformation.
Process Settings, like Max Error and Max Work Unit Length. These settings configure the initial conditions for stabilization and set boundaries and limits for processing.
Deformation Weights¶
alphaSmooth: Controls the “alpha smooth” regularization weight. Higher values make the surface of the mesh move in more uniform, fluid ways.
alphaPlane: Controls the “alpha plane” regularization weight. This weight tries to accurately capture surface curvature. Higher values preserve more surface direction from the source frame.
alphaPoint: Controls the “alpha point” regularization weight. This weight tries to precisely capture surface point positions. Higher values prioritize individual vertex positions more in the deformation process.
alphaRigid: Controls the contribution of the Node Rigidity weight. Higher settings result in stiffer initial transformations, and can help accurate deformation when the pose varies dramatically between frames.
Correspondence Thresholds¶
Match Cosine Limit: Directional limit for pairs. The ‘Match Cosine’ is calculated by taking the Dot Product of the Normal Vector of each vertex in the potential pair. Potential pairs whose dot product is below this value will be ignored.
Note
The “Dot Product” is a mathematical tool commonly used in 3d graphics to compare the angles of two vectors. If both vectors are unit vectors, like the normals of our mesh, the Dot Product is the cosine of the angle of the two vectors. This means that if the surface normals of the two vertices are oriented exactly the same, the dot product will be “1.0”. A 90 degree difference in surface angle will produce a dot product of 0.0. Surfaces oriented in exactly opposing directions will produce a dot product of -1.0, and so on, with angle differences in-between producing values anywhere between 1.0 and -1.0.
Match Distance Limit: Distance limit in Meters for pairs. Potential pairs whose distance exceeds this value will be ignored.
Node Allocation¶
Use Hierarchy Node Sampler: Enables/Disables Hierarchy Node Sampler. Hierarchical node sampling works by starting with a base level of nodes, then increasing allocation of nodes to areas based off of the degree of movement required for deformation. More information on the Hierarchy Node Sampler can be found in the Deformation Algorithm Document
Hierarchy Node Sampler Max Nodes: Determines the maximum number of starting nodes used for deformation when the Hierarchy Node Sampler is enabled. This value will start node creation as a percentage of the number of vertices in the mesh. This parameter only affects deformation if Use Hierarchy Node Sampler is enabled
Sample Radius Factor: This setting influences how many Deformation Nodes will be created for the Deformation process. This setting is multiplied by the average edge length of the mesh to determine the “Sample Radius”. Nodes are evenly distributed based on Sample Radius, so higher Sample Radius values create less nodes, and lower Sample Radius values create more nodes. Higher values will typically result in faster, less accurate stabilization. Sample Radius Factor will only be used if the Hierarchy Node Sampler is disabled.
Important
Remember that Sample Radius Factor scales with edge length. This means higher polygon meshes will use proportionally more nodes than lower polygon meshes on the default setting. If you’re working with very high polygon count meshes (>100,000 tris), we recommend raising this value significantly for your first pass.
Process Settings¶
alphaRigidBreak: This setting determines the stopping point for deforming a frame. Higher values will quit early, resulting in significantly reduced deformation quality, and speeding up individual deformation processing time. Advanced setting.
Segment Max Error: This setting, in meters, controls maximum Correspondence Error acceptable for a stabilized frame. This setting indirectly controls segment length. When the error for a frame exceeds this maximum a segment break is added before this frame.
Important
Segment Max Error is your first, strongest tool to control the amount of compression. If long segments at any cost are a must for your project, you can increase this setting to accept less accurate results.
Initial Non Rigid Align: Performs a low accuracy initial alignment step before deformation. May produce slightly more accurate results with some data. EXPERIMENTAL
Carry Frame: The “Carry Frame” toggle allows you to use the result of each correspondence between two frames as the source for the next correspondence. This means that when building a segment of several frames, HoloEdit will start on the first frame, deform the result of that frame to the next, and so on, for the entire segment. While this setting will not affect your compute time, it reduces parallel processing and can result in significantly slower results. EXPERIMENTAL
Use Hierarchy Node Sampler: This setting provides faster stabilization by performing initial deformation iterations with fewer deformation nodes than would be generated by your mesh and Sample Radius Factor setting, and gradually increasing the number of nodes to the regular count. Additionally, enabling this setting may improve the success of deformation between significantly different Poses if the Match Distance Limit and Match Cosine Limit are relaxed enough to permit correspondences. Recommended for all clips
Progressive Vertex Pairs: This setting provides faster stabilization by performing initial deformations with fewer potential Correspondence Pairs. Recommended for all clips
Double Precision: Use 64 bit math if enabled. Can result in a slight increase in quality, but slower computation.
Merge Vertices: Fuses duplicate verts before running stabilization.
Warning
Merge Vertices should always be enabled unless your mesh is intentionally split up. Non-fused verts result in holes in the mesh and significantly slower stabilization.
Max Iterations: This setting limits the total number of iterations used during each deformation. This setting is mostly used for debugging – typically the deformation process will successfully converge on the limit provided by the alphaRigidBreak parameter well before the default 100 iterations are reached.
Max Work Unit Length: Stabilize intervals are automatically broken up into multiple Work Units for faster parallel processing on the Remote Compute Server. This parameter sets the maximum number of consecutive frames in each Work unit. Lower values complete processing faster, but decrease the length of stabilized segments. Has a minimal impact on compute hours used.
Note
If your clip contains very little motion and maximum compression is a goal for your project, you may want to raise this setting well above the default value of 25. However, expect a much longer wait for your Stabilize results.
Working with Stabilize Mesh¶
Sometimes Stabilize Mesh and Match Template Mesh are referred to as ‘Deformation Stages’. This is because of the core process used in both: ‘Deformation’. Deformation is a complex process with many settings and moving parts.
The deformation process is applied for each keyframe to all neighboring frames. After a deformation, a new result mesh is produced.
In Stabilize Mesh, result meshes are automatically filtered for accuracy. HoloEdit compares the result mesh to the target mesh, and if it accurately captures the pose within an error threshold (supplied with the Segment Max Error parameter) that result mesh is used as the new mesh for that frame.
If the error is too high, that result mesh is rejected, and next HoloEdit tries to use that frame in a subsequent segment instead. The more subsequent frames that each deformed keyframe can accurately capture, the larger the resulting segments.
There are two methods for node allocation, Heirachary Node Sampler and Sample Radius Factor. Hierarchy Node Sampler works by generating a heirachary of node levels based off of a starting percentage of the meshes vertices and adding new nodes in areas of higher movement, for faster processing reduce the Number of Starting Nodes parameter. Sample Radius Factor will create evenly sized and distributed nodes over the surface of the mesh and should be used if you are seeing unsatisfactory results with the Hierachry Node Sampler.
Correspondence Pairs are determined by the closest point method, where the closest point pairs are identified for each vertex on the source mesh based on the Match Distance Limit and Match Cosine Limit settings. The closest match within the distance and cosine limits on the target mesh is used to complete the ‘pair’. If no matches exist, no pair is created from that vertex.
Please refer to the Deformation algorithm page for background information on the Stabilize Mesh process.