• home
  • Player Reference

Player Reference

The Unreal HoloSuite plugin uses the HoloSuitePlayer actor class, which is capable of decode and playback of volumetric data from OMS and AVV files encoded in HoloEdit. The player leverages standard Materials and Shaders (External Links) for rendering, exposes functionality to Blueprints and the Sequencer (External Links), and follows the Media Framework (External Link) for OMS texture video playback.

Note

This is the plugin reference documentation for working with the HoloSuite player inside Unreal Engine.

For a step by step guide to setting up a HoloSuite Player in Unreal, refer to the Player Configuration example document.

HoloSuitePlayer Class

The HoloSuitePlayer class is used to create instances of HoloSuite volumetric video players in your scene. It drives necessary components for the decoding of files generated using Arcturus software and can be configured to customize playback.

Properties:

The HoloSuitePlayer actor class contains the following plugin-specific properties:

../_images/hsp-4.0.0-ue5-holosuiteplayer-default-properties.png

Source:

  • HoloSuite Volumetric File: A volumetric video file (OMS or AVV) encoded using HoloEdit.

  • Mesh Material: A Material for the volumetric mesh. This material should include an ArcturusOMS or ArcturusAVV Material Function for proper playback.

Playback:

  • Play on Open: Toggle whether the source should play automatically.

  • Loop: Toggle whether the source should loop, i.e. should restart once it finishes playback.

  • Frame Rate: Set the Frame Rate. If your mp4 file was exported with a custom frame rate, set this parameter to match your exported frame rate.

Rendering:

  • Responsive AA: Toggle to enable responsive anti-aliasing. This feature addresses playback motion blur.

  • Receive Decals: Toggle to enable mesh decals. Find out more about Decals here (External Link).

Skeleton:

  • Enable Skeleton: Enable the use of skeletons for source files that have been exported with skeleton data.

When a source volumetric file is assigned, other properties become available, depending on the file type.

../_images/hsp-4.0.0-ue5-holosuiteplayer-oms-properties.png

If the player is configured for OMS playback, these are the additional properties:

Source:

  • Texture Source: An MP4 video or an image sequence for texture playback.

  • Media Player: An automatically generated Media Player configured with a corresponding Media Texture.

  • Media Player Material: An automatically generated Material for the Actor’s Media Player.

Playback:

  • Mute: Toggle to disable audio playback.

Decoder:

  • Max Buffered Sequences: Maximum number of sequences from the source OMS file to buffer prior to playback. It is recommended that this parameter is left with the default value, as it will automatically change in accordance to the provided source file. However, if for any reason you wish to reduce the amount of memory used during gameplay, you can reduce this value in accordance to your needs.

Skeleton:

  • Enable Retargeting: Enable to use of skeletons and skin weights for OMS source files that have been exported with rigging data. Further details provided in the Retargeting section.

  • Player Skeletal Mesh: A SkeletalMesh generated from the source OMS file that drives bone transforms used for retargeting or to attach objects onto skeleton bones.

  • Retargeting Animation: An Animation Blueprint generated for the source OMS file to drive skeleton animation during playback.

../_images/hsp-4.0.0-ue5-holosuiteplayer-avv-properties.png

If the player is configured for AVV playback, these are the additional properties:

Playback:

  • External Timing: Toggle to enable playback control from an external controller. Used for Sequencer-driven playback.

  • Current Frame: Set the current frame of the source AVV file. Used in-editor only, to preview the content.

Level of Detail:

  • LOD 0/1/2 Screen Size: Used to configure the levels of detail. This should be configured to match the export settings of the AVV file.

  • Minimum LOD: Set the minimum LOD to use. Useful for when the player is expected to always be far away. Default is 0.

  • Force LOD: Set the LOD to always use. Ignored if set to -1. Find out more about Levels of Details here (External Link).

Decoder:

  • Load in Editor: Toggle whether the AVV content should be decoded and loaded into the scene while working in the Editor and not playing.

  • Playback Delay: Set the amount of time in milliseconds that the player should wait for before starting playback.

  • Use CPU Decoder: Toggle whether the AVV decoding should be done in the CPU. This is recommended for mobile target platforms such as the Quest.

Skeleton:

  • Player Skeletal Mesh: A SkeletalMesh generated from the source OMS file that drives bone transforms used to attach objects onto skeleton bones. Note that retargeting is not yet possible for AVV playback.

Arcturus Material Functions

HoloSuitePlayer actors drive the Arcturus OMS Material function in an assigned Material for OMS Playback and the Arcturus AVV Material function for AVV. In order to play back correctly, you will need to configure the properties of your actor to the corresponding OMS or AVV and material.

The Arcturus Material functions are used to animate and texture volumetric files for playback. The ArcturusAVV and ArcturusOMS nodes each have two Output Pins:

  • Texture: The Color texture sampled from the Media Player assigned to the object’s OMSMeshActor class.

  • Normals: The Mesh Normals produced from SSDR deformation.

The ArcturusOMS node has one additional Output Pin:

  • World Position Offset: The World Position Offset required to deform the mesh for playback.

For a simple player, these pins can be connected to the Emissive (or base) color, Normal, and World Position Offset outputs, respectively. In your Custom Materials, apply additional nodes as needed between the Arcturus function and your output node.

Media Player for OMS Playback

A Media Player, Media Texture, and a Media Player Material are automatically generated during HoloSuitePlayer configuration for OMS playback.

  • One Media Player configured to control Playback.

  • One Media Texture configured to select the corresponding Media Player that was generated.

  • One Material to shade the Media Player playback. This Material contains one Texture Sample node sampling the Media Texture, connected to the Emissive Color output pin.

  • One Media Sound Component to playback audio content from the source MP4 texture file. This component isn’t exposed to the user.

Retargeting

One key feature of OMS Playback is Retargeting. Retargeting allows you to dynamically re-pose your volumetric capture like any other Skinned Mesh in Unreal without disrupting playback.

Using the AnimGraph, you may re-animate any bones embedded in the OMS via the Generate Skeleton stage. If Skin Weights have been provided, either manually or via the Generate Skin Weights for Head Retargeting stage in HoloEdit, the weighted vertices will be transformed accordingly.

To enable retargeting, you must configure a Skeletal Mesh, Skeleton, and Animation Blueprint, as shown above. The components of Retargeting are as follows:

  • Skeletal Mesh: A mesh bound to a hierarchical skeleton of bones which can be animated for the purpose of deforming the mesh. This should be created automatically from any OMS containing rigging data.

  • Skeleton: A component that links between mesh and animation. It must be created via right-clicking on the SkeletalMesh component and choosing “Create Skeleton” from the “Skeleton” category.

  • Retargeting Blueprint: Retargeting uses an Animation Blueprint to drive animation during playback.

See Retargeting Configuration to learn how to configure a HoloSuitePlayer actor with retargeting enabled.

Blueprints and Scripting

../_images/hsp-4.0.0-ue5-holomesh-blueprint-functions.png

HoloMesh Actions:

  • GetFPS: Retrieve global average frames per second of HoloSuitePlayers. Useful for profiling multiple volumetric actors.

  • GetLastBreakTime: Retrieve amount of time in milliseconds exceeded by the HoloSuitePlayers in the previous frames.

../_images/hsp-4.0.0-ue5-holosuiteplayer-blueprint-functions.png

HoloSuitePlayer Actions:

  • Getters and Setters: Most HoloSuitePlayer actor properties have corresponding getter and setter functions to modify them on runtime through Blueprint logic.

  • Pause: Pauses playback.

  • Play: Starts playback.

  • Seek: Seeks to the specified playback time.

  • OpenVolumetricFile: Assigns a new volumetric file encoded using HoloEdit and opens it. Meant for AVV playback.

  • OpenVolumetricFile: Assigns a new volumetric file and corresponding texture file encoded using HoloEdit and opens it. Meant for OMS playback. Automatically starts playback after completion if PlayOnOpen is enabled.

  • GetSkeletalMeshComponent: Returns the child SkeletalMeshComponent, generated from the SkeletalMesh provided, if there is any (if EnableSkeleton is true).

  • AttachActorToSkeleton: Attaches an Actor to one of the SkeletalMesh’s bones (if EnableSkeleton is true).

../_images/hsp-4.0.0-ue5-holosuiteplayer-blueprint-events.png

HoloSuitePlayer Events:

  • OnEndReached: Invoked when playback has finished.

  • OnLoop: Invoked when loop point is reached and playback restarts from the beginning. Currently only available for AVV playback.

  • OnMediaReady: Invoked when the volumetric source file(s) are ready for playback. Only applicable to OMS playback.

  • OnPlaybackResumed: Invoked when playback starts or is resumed.

  • OnPlaybackSuspended: Invoked when playback is stopped / suspended / paused.

  • OnMediaOpenFailed: Invoked when the source volumetric source file(s) failed to be loaded. Currently only available for AVV playback.

  • OnMediaOpened: Invoked when the volumetric source file(s) has been successfully loaded.