Player Reference¶
The Unreal HoloSuite plugin uses the HoloSuitePlayer actor class, which is capable of decode and playback of volumetric data from OMS and AVV files encoded in HoloEdit. The player leverages standard Materials and Shaders (External Links) for rendering, exposes functionality to Blueprints and the Sequencer (External Links), and follows the Media Framework (External Link) for OMS texture video playback.
Note
This is the plugin reference documentation for working with the HoloSuite player inside Unreal Engine.
For a step by step guide to setting up a HoloSuite Player in Unreal, refer to the Player Configuration example document.
HoloSuitePlayer Class¶
The HoloSuitePlayer class is used to create instances of HoloSuite volumetric video players in your scene. It drives necessary components for the decoding of files generated using Arcturus software and can be configured to customize playback.
Properties:¶
The HoloSuitePlayer actor class contains the following plugin-specific properties:
Source:
HoloSuite Volumetric File: A volumetric video file (OMS or AVV) encoded using HoloEdit.
Mesh Material: A Material for the volumetric mesh. This material should include an ArcturusOMS or ArcturusAVV Material Function for proper playback.
When a source volumetric file is assigned other properties become available, depending on the file type.
If the player is configured for OMS playback, these are the additional properties:
Source:
Texture Source: An MP4 video or an image sequence for texture playback.
Media Player: An automatically generated Media Player configured with a corresponding Media Texture.
Media Player Material: An automatically generated Material for the Actor’s Media Player.
Playback:
Play on Open: Toggle whether the source should play automatically.
Loop: Toggle whether the source should loop, i.e. should restart once it finishes playback.
Ping Pong: Toggle to switch the playback direction each time the clip loops.
Reverse: Toggle playing the clip in reverse. Note that reverse playback does not support motion vectors.
Loop: Toggle whether the source should loop, i.e. should restart once it finishes playback.
Mute: Toggle to disable audio playback.
Frame Rate: Set the Frame Rate. If your MP4 file was exported with a custom frame rate, set this parameter to match your exported frame rate.
Current Frame: Set the current frame of the AVV clip.
Decoder:
Max Buffered Sequences: Maximum number of sequences from the source OMS file to buffer prior to playback. It is recommended that this parameter is left with the default value, as it will automatically change in accordance to the provided source file. However, if for any reason you wish to reduce the amount of memory used during gameplay, you can reduce this value in accordance to your needs.
Rendering:
Responsive AA: Toggle to enable responsive anti-aliasing. This feature addresses playback motion blur.
Receive Decals: Toggle to enable mesh decals. Find out more about Decals here (External Link).
Skeleton:
Enable Skeleton: Enable to use skeletal data, and reveal additional retargeting playback options for your OMS.
Enable Retargeting: Enable to use skeletons and skin weights for OMS source files that have been exported with rigging data. Further details provided in the Retargeting section.
Player Skeletal Mesh: A SkeletalMesh generated from the source OMS file that drives bone transforms used for retargeting or to attach objects onto skeleton bones.
Retargeting Animation: An Animation Blueprint generated for the source OMS file to drive skeleton animation during playback.
If the player is configured for AVV playback, these are the additional properties:
Playback:
External Timing: Toggle to enable playback control from an external controller. Used for Sequencer-driven playback.
Play on Open: Toggle whether the source should play automatically.
Loop: Toggle whether the source should loop, i.e. should restart once it finishes playback.
Ping-pong: Causes the clip to play through, then cause to play in reverse once reaching the last frame.
Reverse: Causes the clip to start at its end frame and play in reverse.
Frame Rate: Set the Frame Rate. If your MP4 file was exported with a custom frame rate, set this parameter to match your exported frame rate.
Current Frame: Set the current frame of the source AVV file. Used in-editor only, to preview the content.
Level of Detail:
LOD 0/1/2 Screen Size: Used to configure the levels of detail. This should be configured to match the export settings of the AVV file.
Minimum LOD: Set the minimum LOD to use. Useful for when the player is expected to always be far away. Default is 0.
Force LOD: Set the LOD to always use. Ignored if set to -1. Find out more about Levels of Details here (External Link).
Decoder:
Load in Editor: Toggle whether the AVV content should be decoded and loaded into the scene while working in the Editor and not playing.
Playback Delay: Set the amount of time in milliseconds that the player should wait for before starting playback.
Use CPU Decoder: Toggle whether the AVV decoding should be done in the CPU. This is recommended for mobile target platforms such as the Quest.
Rendering:
Motion Vectors: Toggle to enable the use of motion vectors. This will only have an effect if your AVV has been exported from HoloEdit with completed Generate Motion Vectors stage results spanning the entire clip.
Responsive AA: Toggle to enable responsive anti-aliasing. This feature addresses playback motion blur.
Receive Decals: Toggle whether the volumetric video should receive decals.
Skeleton
Enable Skeleton: Enables skeletons for the use of skeletons for object/particle attachment. Applicable only to source files that have been exported with rigging data.
Arcturus Material Functions¶
HoloSuitePlayer actors drive the Arcturus OMS Material function in an assigned Material for OMS Playback and the Arcturus AVV Material function for AVV. In order to play back correctly, you will need to configure the properties of your actor to the corresponding OMS or AVV and material.
The Arcturus Material functions are used to animate and texture volumetric files for playback. The ArcturusAVV and ArcturusOMS nodes each have two Output Pins:
Texture: The Color texture sampled from the Media Player assigned to the object’s OMSMeshActor class.
Normals: The Mesh Normals produced from SSDR deformation.
The ArcturusOMS node has one additional Output Pin:
World Position Offset: The World Position Offset required to deform the mesh for playback.
For a simple player, these pins can be connected to the Emissive (or base) color, Normal, and World Position Offset outputs, respectively. In your Custom Materials, apply additional nodes as needed between the Arcturus function and your output node.
Media Player for OMS Playback¶
A Media Player, Media Texture, and a Media Player Material are automatically generated during HoloSuitePlayer configuration for OMS playback.
One Media Player configured to control Playback.
One Media Texture configured to select the corresponding Media Player that was generated.
One Material to shade the Media Player playback. This Material contains one Texture Sample node sampling the Media Texture, connected to the Emissive Color output pin.
One Media Sound Component to playback audio content from the source MP4 texture file. This component isn’t exposed to the user.
Retargeting¶
One key feature of OMS Playback is Retargeting. Retargeting allows you to dynamically re-pose your volumetric capture like any other Skinned Mesh in Unreal without disrupting playback.
Using the AnimGraph, you may re-animate any bones embedded in the OMS via the Generate Skeleton stage. If Skin Weights have been provided, either manually or via the Generate Skin Weights for Head Retargeting stage in HoloEdit, the weighted vertices will be transformed accordingly.
To enable retargeting, you must configure a Skeletal Mesh, Skeleton, and Animation Blueprint, as shown above. The components of Retargeting are as follows:
Skeletal Mesh: A mesh bound to a hierarchical skeleton of bones which can be animated for the purpose of deforming the mesh. This should be created automatically from any OMS containing rigging data.
Skeleton: A component that links between mesh and animation. It must be created via right-clicking on the SkeletalMesh component and choosing “Create Skeleton” from the “Skeleton” category.
Retargeting Blueprint: Retargeting uses an Animation Blueprint to drive animation during playback.
See Retargeting Configuration to learn how to configure a HoloSuitePlayer actor with retargeting enabled.
Blueprints and Scripting¶
HoloSuitePlayer Components:
AVVPlayer: Allows for playback and control of AVV files.
OMSPlayer: Allows for playback and control of OMS files.
Configuring HoloSuite Player Components¶
To configure a HoloSuite Player component you will need to start by creating a blueprint in your content browser. When creating, select the Actor parent class. You can now open the blueprint and click the Add button in the components tab. Select either the AVVPlayer or OMSPlayer component depending on your desired volumetric format. Once added you will need to configure the Source settings in the details panel. For more information on configuring sources, please read the HoloSuitePlayer Class section of the docs. Now that you have configured the source, you can compile and save. The HoloSuite Player is now ready to be used in your scene.
HoloMesh Actions:
GetAverageIOTime: Retrieve the average IO time in milliseconds for all of the HoloSuitePlayers in the scene.
GetFPS: Retrieve global average frames per second of HoloSuitePlayers. Useful for profiling multiple volumetric actors.
GetLastBreakTime: Retrieve amount of time in milliseconds exceeded by the HoloSuitePlayers in the previous frames.
GetVisableMeshCount: Retrieve the number of visable meshes for all of the HoloSuitePlayers in the scene.
HoloSuitePlayer Actions:
Getters and Setters: Most HoloSuitePlayer actor properties have corresponding getter and setter functions to modify them on runtime through Blueprint logic.
Decoder:
Set Decoder Parameters: Sets the parameters used to decode the volumetric file.
Level Of Detail:
Set LODParameters: Sets LOD parameters for the volumetric actor.
Playback:
Pause: Pauses playback.
Play: Starts playback.
Seek: Seeks to the specified playback time. This is looking for a timespan as an input. This can be generated by the make timespan node.
Set Playback Parameters: Sets the parameters used to playback the volumetric file.
Rendering:
Set Rendering Parameters: Sets the parameters used to render the volumetric file.
Skeleton:
Attach Actor To Skeleton: Attaches an Actor to one of the SkeletalMesh’s bones (if EnableSkeleton is true).
Get Skeletal Mesh Component: Returns the child SkeletalMeshComponent, generated from the SkeletalMesh provided, if there is any (if EnableSkeleton is true).
Has Skeleton Data: Returns true if the volumetric asset has a skeleton.
Set Skeleton Parameters: Sets the parameters used for the skeleton of the volumetric actor.
Source:
Close: Stops playback and clears the volumetric file from the HoloSuite Player.
Open AVVFile: Assigns a new volumetric file encoded using HoloEdit and opens it. Meant for AVV playback. Automatically starts playback after completion if PlayOnOpen is enabled.
Open OMSFile: Assigns a new volumetric file and corresponding texture file encoded using HoloEdit and opens it. Meant for OMS playback. Automatically starts playback after completion if PlayOnOpen is enabled.
Set Mesh Material: Assigns a new HoloSuite Player material to the HoloSuite Player.
HoloSuitePlayer Events:
OnEndReached: Invoked when playback has finished.
OnLoop: Invoked when loop point is reached and playback restarts from the beginning. Currently only available for AVV playback.
OnMediaReady: Invoked when the volumetric source file(s) are ready for playback. Only applicable to OMS playback.
OnPlaybackResumed: Invoked when playback starts or is resumed.
OnPlaybackSuspended: Invoked when playback is stopped / suspended / paused.
OnMediaOpenFailed: Invoked when the source volumetric source file(s) failed to be loaded. Currently only available for AVV playback.
OnMediaOpened: Invoked when the volumetric source file(s) has been successfully loaded.