Composition Contents¶
Compositions are made up of tracks, stages, and intervals.
Pictured here is a composition including three tracks (“Ewan”, “Example_Clip”, and “Ewan”), with each track containing one or more “stages” (the rows labeled “Load:” or “Clean Mesh”), plus one interval per stage (the long grey blocks of time to the right of the stage name).
Tracks¶
Tracks are the basic building blocks of volumetric compositions, representing individual clips of volumetric video and any editing applied within HoloEdit.
Each track is made up of one or more stages, which contain individual operations for adding or transforming data in the track, such as loading data from a clip, adding rigging data, or performing compression.
In the HoloEdit UI, tracks are displayed along with their individual stages in the Track View (see Track View on the left side of the Composition Panel. Tracks are aligned to the leftmost side, and the individual stages making up those tracks are nested downwards and to the right.
Tracks are displayed in the the order that they were added to the composition, but the order doesn’t affect any HoloEdit processing or export.
Stages¶
Stages are procedural processes applied to tracks to load or manipulate volumetric video. Stages take in data streams from earlier in the track, and replace or alter those data streams going forward down the track. Each kind of stage performs a specific operation on one or more of their track’s data streams, then outputs one or more new or modified streams.
The most fundamental stage is the “Load Asset” stage, which loads new samples into the track from a clip in your HoloEdit workspace.
Stages are applied in order from top to bottom, with each stage taking in the closest preceding stage with appropriate data as its input for processing. Each stage is applied to the track through intervals, which are spans of time in the timeline with individual stage settings configured for their specified portion of the clip. For details on the stages and intervals UI, see the Track View section. See the Stage Types section for more information on the function and settings of individual stages.
Intervals¶
Processing from stages is applied through Intervals. An arbitrary number of intervals can be created per stage, each representing a contiguous stretch of time containing one or more frames for processing. Where stages represent the kind of work that will be done, and to what data streams, intervals represent what portion of the clip’s samples will be processed, and with what settings.
With multiple intervals, a single process that occurs at a specific point in the track (such as stabilization) can have varying settings throughout the clip, to best suit the samples throughout the stream.
Three stages, with different intervals. The first stage has one long interval, while the subsequent two each have three smaller intervals. The second interval in the third stage is selected.¶
Loading Clips¶
Clips are added to a HoloEdit composition using a special stage, called Load Asset. Load Asset intervals directly apply the samples from your clip to your track.
For most data types, this means overwriting any existing data of the same type supplied by stages earlier in the track. If you use a Load Asset interval to add new mesh data to a HoloEdit track with existing meshes, the meshes from the new Load Asset stage will replace any existing meshes in the Track Result.
Audio streams behave differently. Audio is additive, so subsequent Load Asset stages containing audio will layer multiple audio streams over eachother.
To control this behavior, you can use the Excluded Streams setting on any Load Asset interval to exclude streams that you may not wish to apply.
Segments and Stabilization¶
Temporal Stabilization, or Stabilization, is a special form that mesh streams can take which empowers compression and editing of volumetric video.
In stabilized data, a mesh stream is broken up into one or more segments. Each mesh sample within a given segment shares the same mesh topology and UV Layout with its neighbors.
Segments¶
Samples of stabilized data in HoloEdit are grouped into portions called Segments. Segments are introduced to a HoloEdit track using stages such as Stabilize Mesh and Match Template Mesh, or by adding a clip including segment data in a Load Asset stage.
In the Composition Panel, segments are displayed as dashed lines dividing segment borders in a stabilized interval.
Some processes and workflows only work with stabilized segments, such as Compress Mesh Segments or Mari Export.
Track and Composition Results¶
To understand your HoloEdit composition, it helps to understand the Track Result and Composition Result. HoloEdit is a non-linear editor, which means that rather than having a single, edited version of your volumetric clip, the state of the track at each individual stage is available at any time. In HoloEdit, a Track Result is the state of your entire clip at any given point in a track. This contains any new edits to your data applied by the current stage, in addition to any non-edited data inherited from all previous stages. In HoloEdit this is sometimes called the Waterfall, because un-edited data flows down from through previous stages like a waterfall.
The track result is used whenever a clip is exported, displayed in the HoloEdit viewport, or dispatched for processing in a stage. This is the product of each stage that has been processed in your track. The final product of every visible track combined is called the composition result.
Samples contained in hidden stages or that have not been completely processed are excluded from the track result displayed in the viewport and in exports.
When executing a stage, it helps to understand the result up to the point in the track where that stage appears. If the required data for your stage is not present in the current result it will not be possible to execute. To help understand the concept of the track result, please read the example below.
For example, imagine a track with three stages
The first stage is a Load Asset stage with a single interval from frames 0 - 200.
The next is a Stabilize Mesh stage with intervals from 0-100, 101-150, and 151-200. The first and third intervals are complete, but the second has failed.
The final stage is a Compress Mesh Segments stage, with corresponding intervals at 0-100, 101-150, and 151-200. The first and third have been successfully processed, but the second is blocked by the preceding stage.
The track result, in this case, will be frames 0-100 from stage 3, frames 101-150 from stage 1, and 151-200 from stage 3. This track result is what is visible in the viewport with the track selected, and is also what would be output from holoedit when exported.
Note
When an interval is blocked by an incomplete interval in a previous stage, the entire interval will fail to compute. In the previous example, if there had been two intervals in the third stage – 0-100 and 101-200, the entire 101-200 interval would be blocked by the failed 101-150 interval in the second stage.
Compositions and Job Data¶
Processing in HoloEdit is done via Jobs run on the Local or Remote Job Server. Each job is made up of individual Work Units, and as work units complete processing they incrementally send finished data back to HoloEdit. If jobs corresponding to an open composition are being run, results from that job will be fetched as they are completed. If that composition is closed, the corresponding data will all be fetched as soon as the composition is re-opened. In some cases, such as after deleting the relevant stage or intervals from a composition, jobs may be considered abandoned within one composition but not another.
See Processing & Jobs for more details on the job server.
Saving and Loading Compositions¶
Compositions can be saved and loaded using the “Save” and “Save As” options in the File Menu. Saved compositions should be stored in the current workspace, and will be saved as composition .json.lz files. Saved compositions can be loaded by double-clicking on the entry in the Project Panel.
Along with saving the current stage and interval configurations, saved compositions store job status, such as active, waiting to download files from the server, and so on. A saved composition can be closed while individual intervals are processing on the Remote Job Server, and will fetch the current job status and download necessary files when it’s opened back up.
Autosave¶
HoloEdit will periodically automatically save your current composition to the “autosave” folder located in the root directory of your Workspace. Autosaves include your current composition name plus the date and time.
Autosaves will store composition and job result status like any other save file, and may be deleted periodically.