Stabilize Mesh¶
Overview¶
Note
Stabilize Mesh uses a complex ‘Deformation’ algorithm to produce new poses. For in depth information on deformation, see the Deformation algorithm page.
The Stabilize Mesh stage automatically converts your clip into one or more “Stabilized Segments”, a kind of geometry that empowers fast editing and powerful SSDR compression.
When you execute a Stabilize interval, HoloEdit chooses one or more Keyframes and begins deforming them to produce Segments, with the goal of all frames in the Interval being captured in the fewest number of Segments as possible.
A Stabilized Segment is a series of frames with a single consistent topology for every individual frame, or pose.
Most clips in HoloEdit should be stabilized. Without stabilization, advanced compression and editing features inside and outside of HoloEdit are unavailable.
Note
This Stage should be run on a cleaned mesh stream, with degenerate triangles fixed and a polygon count between 10,000 and 30,000 triangles. Higher triangle counts are supported, but compute hour costs may increase dramatically. For higher triangle count meshes, start with a higher Sample Radius Factor.
Parameters¶
When Configuring the Stabilize Mesh stage, keep in mind that the ultimate goal is long segments with acceptable pose quality for your own project’s needs. Depending on your clip and final playback platform, different levels of accuracy and compression may be warranted.
Also, it’s important to pay close attention to your source meshes if you have very strict requirements for your result. Ultimately, the Stabilize Mesh stage is limited by the source data. If your meshes are very noisy from frame to frame, or if your actor is moving very quickly, there may be limits to the length of high quality segments.
Warning
Stabilization is a complex process and can result in some loss of quality in some clips. On many clips, you can run Stabilize Mesh with default settings to produce a desirable result. However, for more complex clips, please read the “Working With Stabilize Mesh” section below.
The Parameters provided for the Stabilize Mesh stage can be roughly categorized into three groups:
Deformation Weights, like AphaRigid, AlphaSmooth, AlphaPoint, and alphaPlane. These settings influence how the keyframe is deformed, to help conform it to the target mesh.
Correspondence Thresholds, like Match Distance Limit and Match Cosine Limit. These settings influence which vertices are chosen as pairs to guide the deformation process to correctly capture the pose.
Process Settings, like Max Error and Use Hiearchy Node Sampler. These settings configure the initial conditions for Stabilization and set boundaries and limits for processing.
Deformation Weights
alphaPlane: Controls the “alpha plane” regularization weight. This weight tries to accurately capture surface curvature. Higher values preserve more surface direction from the source frame.
alphaPoint: Controls the “alpha point” regularization weight. This weight tries to precisely capture surface point positions. Higher values prioritize individual vertex positions more in the deformation process.
alphaRigid: Controls the contribution of the Node Rigidity weight. Higher settings result in stiffer initial transformations, and can help accurate deformation when the pose varies dramatically between frames.
alphaSmooth: Controls the “alpha smooth” regularization weight. Higher values make the surface of the mesh move in more uniform, fluid ways.
Correspondence Thresholds
Match Cosine Limit: Directional limit for pairs. The ‘Match Cosine’ is calculated by taking the Dot Product of the Normal Vector of each vertex in the potential pair. Potential pairs whose dot product is below this value will be ignored.
Note
The “Dot Product” is a mathematical tool commonly used in 3d graphics to compare the angles of two vectors. If both Vectors are Unit Vectors, like the Normals of our mesh, the Dot Product is the cosine of the angle of the two vectors. This means that if the surface normals of the two vertices are oriented exactly the same, the dot product will be “1.0”. a 90 degree diference in surface angle will produce a dot product of 0.0. Surfaces oriented in exactly opposing directions will produce a dot product of -1.0, and so on, with angle differences inbetween producing values anywhere between 1.0 and -1.0.
Match Distance Limit: Distance limit in Meters for pairs. Potential pairs whose distance exceeds this value will be ignored.
Process Settings
alphaRigidBreak: This setting determines the stopping point for deforming a frame. Higher values will quit early, resulting in significantly reduced deformation quality, and speeding up individual deformation processing time. Advanced setting.
Segment Max Error: This setting, in meters, controls maximum Correspondence Error acceptable for a stabilized frame. This setting indirectly controls segment length. When the error for a frame exceeds this maximum a segment break is added before this frame.
Important
Segment Max Error is your first, strongest tool to control the amount of compression. If long segments at any cost are a must for your project, you can increase this setting to accept less accurate results.
Initial Non Rigid Align: Performs a low accuracy initial alignment step before deformation. May produce slightly more accurate results with some data. EXPERIMENTAL as of HoloEdit 2021.1
Sample Radius Factor: This setting influences how many Deformation Nodes will be created for the Deformation process. This setting is multiplied by the average edge length of the mesh to determine the “Sample Radius”. Nodes are evenly distributed based on Sample Radius, so higher Sample Radius values create less nodes, and lower Sample Radius values create more nodes. Higher values will typically result in faster, less accurate stabilization.
Important
Remeber that Sample Radius Factor scales with edge length. This means higher polygon meshes will use proportionally more nodes than lower polygon meshes on the default setting. If you’re working with very high polygon count meshes (>100,000 tris), we recommend raising this value significantly for your first pass.
Carry Frame: The “Carry Frame” toggle allows you to use the result of each correspondence between two frames as the source for the next correspondence. This means that when building a segment of several frames, HoloEdit will start on the first frame, deform the result of that frame to the next, and so on, for the entire segment. While this setting will not affect your compute time, it reduces paralell processing and can result in significantly slower results. EXPERIMENTAL as of HoloEdit 2021.1
Use Hierarchy Node Sampler: This setting added in HoloEdit 2021.1 provides faster stabilization by performing initial deformation iterations with fewer Deformation Nodes than would be generated by your mesh and Sample Radius Factor setting, and gradually increasing the number of nodes to the regular count. Additionally, enabling this setting may improve the success of deformation between significantly different Poses if the Match Distance Limit and Match Cosine Limit are relaxed enough to permit correspondences. Recommended for all clips
Progressive Vertex Pairs: This setting added in HoloEdit 20201.1 provides faster stabilization by performing initial deformations with fewer potential Correspondence Pairs. Recommended for all clips
Double Precision: Use 64 bit math if enabled. Can result in a slight increase in quality, but slower computation.
Merge Vertices: Fuses duplicate verts before running stabilization.
Warning
Merge Vertices should always be enabled unless your mesh is intentionally split up. Non-fused verts result in holes in the mesh and significantly slower stabilization.
Max Iterations: This setting limits the total number of iterations used during each Deformation. This setting is mostly used for debugging – typically the deformation process will successfully converge on the limit provided by the alphaRigidBreak parameter well before the default 100 iterations are reached.
Max Work Unit Length: In HoloEdit 2021.1, stabilize intervals are automatically broken up into multiple Work Units for faster parallel processing on the Remote Compute Server. This parameter sets the maximum number of consecutive frames in each Work unit. Lower values complete processing faster, but decrease the length of Stabilized Segments. Has a minimal impact on compute hours used.
Note
If your clip contains very little motion and maximum compression is a goal for your project, you may want to raise this setting well above the default value of 25. However, expect a much longer wait for your Stabilize results.
Working with Stabilize Mesh¶
Sometimes Stabilize Mesh and Template Match are referred to as ‘Deformation Stages’. This is because of the core process used in both: ‘Deformation’. Deformation is a complex process with many settings and moving parts.
The deformation process is applied for each keyframe to all neighboring frames. After a deformation, a new result mesh is produced.
In Stabilize Mesh, result meshes are automatically filtered for accuracy. HoloEdit compares the result mesh to the target mesh, and if it accurately captures the pose within an error threshold (supplied with the Segment Max Error parameter) that result mesh is used as the new mesh for that frame.
If the error is too high, that result mesh is rejected, and next HoloEdit tries to use that frame in a subsequent segment instead. The more subsequent frames that each deformed Keyframe can accurately capture, the larger the resulting segments.
Please refer to the Deformation algorithm page for background information on the Stabilize Mesh process.