Unreal: OMS Player Reference

Note

This is the reference documentation for the Unreal OMS Player Plugin.

For a step by step guide to setting up an OMS Player in Unreal Engine, refer to the OMS Player Configuration example document.

HoloSuite Plugin Install

Unreal Engine version 4.27 is recommended for OMS playback.

Your HoloSuite Install Directory contains Unreal Engine plugins 4.25, 4.26, and 4.27.

To Install the HoloSuite UE4 plugin locate the “HoloSuitePlayer 2021.2.0 UE4 v4.27 Plugin.zip” file in the Tools folder of your HoloSuite 2021.2 file directory. Create a new folder called HoloSuitePlugin and unzip the contents of the zip file into the HoloSuitePlugin directory.

Drag and drop the HoloSuitePlugin folder into your Unreal Engine Plugins folder. By default, this should be: “C:\Program Files\Epic Games\UE_4.27\Engine\Plugins”. See the Plugins page on the Unreal Engine Documentation (external link) for more information on installing plugins in Unreal Engine 4.

To finish installing the plugin, create a new UE4 project. From the main Unreal Engine interface, click on Settings in the top menu, and select Plugins. In the Plugin editor, a HoloSuite Player item will be visible. Check ‘Enabled’ and click the “Restart Now” button in the bottom right of the window. When Unreal opens again, you’ll be ready to start configuring the OMS Player.

Getting started

Each OMS player requires the following files created in Unreal Engine in addition to your mp4 and OMS:

Read on below to learn about the contents of the HoloSuite Plugin. To get started on using the HoloSuite Plugin right away, jump to the OMS Player Configuration example

Build Targets

The Unreal OMS Player can be built for Windows 10 and Mac OS X.

Overview

The Unreal OMS Player is included in Plugin called “HoloSuite Player”. The contents of this plugin can be used to play back compressed Volumetric Video created in HoloEdit. The Plugin package contains c++ classes, blueprint functions, and material functions.

OMS Playback uses the OMSMeshactor class, which follows the Media Framework (External Link) for video playback, and regular UE4 materials using a custom Material Function for rendering. Users can integrate the OMS player with any material graph or blueprint scene logic.

This documentation page describes the contents of the HoloSuite Player Plugin, along with a brief example of configuring an OMS Player.

The HoloSuite Player supports most OMS Playback features. These include compressed OMS playback and retargeting.

Overview: Retargeting

One key feature of OMS Playback is Retargeting. Retargeting allows you to dynamically re-pose your volumetric capture like any other Skinned Mesh in Unreal without disrupting playback. Using the AnimGraph, you may re-animate any bones embedded in the OMS via the Generate Skeleton stage.

If Skin Weights have been provided, either manually or via the Generate Skin Weights for Head Retargeting stage in HoloEdit, the weighted vertices will be transformed accordingly.

One common usage of Retargeting is “Head Retargeting”, using the head bone to make the volumetric actor look at an animated object or camera in the scene during playback. Clips can be prepared for Head Retargeting in HoloEdit using the Generate Skeleton and Generate Skin Weights For Head Retargeting stages without any manual editing.

See the Retargeting Setup section below for a step by step guide on setting up Head Retargeting in Unreal Engine.

Plugin OMS File Support

With the HoloSuite Player plugin installed, OMS files added to your project are automatically detected as OMS files. In addition to playback with the OMSMeshActor class, OMS files have the following functionality:

Right click on an .oms file in the Content Browser

  • SkeletalMesh: A SkeletalMesh component for the embedded skeleton can be created automatically from any OMS containing rigging data. This will appear above the default functionality in the context menu, under the “OMS Actions” header.

  • Create Lit Material: Generate a new default Lit material for your OMSMeshActor. You can use this material as the Mesh Material on an OMSMeshActor Instance.

  • Create Unlit Material: Generate a new default Unlit material for your OMSMeshActor. You can use this material as the Mesh Material on an OMSMeshActor Instance.

The OMSMeshActor Class

The OMSMeshActor class is used to create OMS players in your scene. It drives necessary components of the Arcturus OMS Material function and can be configured to specify an OMS file and enable or disable Retargeting. This class drives OMS playback in conjunction with an Arcturus OMS Material Function.

The OMSMeshActor class contains the following properties:

Default:

OMS Settings:

  • OMS: Specify an OMS file for playback.

  • Texture Source: Specify an .mp4 video for playback.

  • Mesh material: Specify a Material for OMS Playback. This material should include an ArcturusOms Material Function for proper playback.

Advanced settings:

  • Media Player: An automatically generated Media Player configured with a corresponding Media Texture. See Media Player for OMS Playback for more information.

  • Media Player Material: An automatically generated Material for the Actor’s Media Player. See Media Player for OMS Playback for more information.

  • Play on Open: Toggle whether the OMS should play automatically.

  • Frame Rate: Set the Frame Rate. If your mp4 file was exported with a custom frame rate, set this parameter to match your exported frame rate.

Retargeting:

These properties allow you to configure Retargeting. See Retargeting Configuration For more information.

  • Retarget Enabled: Enable Retargeting.

  • Retarget Skeletal Mesh: Specify a Skeletal Mesh for Retargeting.

Understanding the OMSMeshActor class

OMSMeshActors drive the Arcturus OMS Material function in an assigned Material for OMS Playback.

In order to play back correctly, you will need to configure the properties of your OmsMeshActor to the corresponding OMS and materialr.

The Arcturus OMS Material Function

The Arcturus OMS Material function is used to animate and texture OMS files for playback. Used in combination with an OMSMeshActor Class this Material function drives OMS playback. The ArcturusOMS node has three Output Pins:

  • Texture: The Color texture sampled from the Media Player assigned to the object’s OMSMeshActor class.

  • Normals: The Mesh Normals produced from SSDR deformation.

  • World Position Offset: The World Position Offset required to deform the mesh for playback.

For a simple OMS Player, these pins can be connected to the Emissive (or base) color, Normal, and World Position Offset outputs, respectively. In your Custom Materials, apply additional nodes as needed between the ArcturusOMS function and your output node.

Media Player for OMS Playback

A Media Player, media texture, and a media player material are automatically generated during OMSMeshActor configuration.

  • One Media Player configured to control Playback.

  • One Media Texture configured to select the corresponding Media Player that was generated.

  • One Material to shade the Media Player playback. This Material contains one Texture Sample node sampling the Media Texture, connected to the Emissive Color output pin.

Retargeting

Retargeting allows real time animation during playback of an OMS file containing rigging data.

To enable retargeting, you must configure a Skeletal Mesh, Skeleton, and Animation Blueprint.

The components of Retargeting are as follows:

  • SkeletalMesh: The SkeletalMesh component must be created automatically from any OMS containing rigging data. To create an OMS Skeletal mesh, right click on an OMS file in the Content Browser and choose “Create Retarget SkeletalMesh” from the OMSActions section.

  • Skeleton: The Skeleton component must be created via right clicking on the SkeletalMesh component and choosing “Create Skeleton” from the “Skeleton” category.

  • Retargeting Blueprint: Retargeting uses an Animation Blueprint to drive animation during playback. To create a Retargeting Blueprint, create a new Animation Blueprint.

    • After configuring the Animation Blueprint as necessary for your project, add an instance of the blueprint to your scene and configure the “Retarget Skeletal Mesh Actor” property on your corresponding OMSMeshActor to point to the Animation Actor. During playback, the Blueprint logic specified by the Animation Blueprint will be applied to the OMS Player.

Appendix: Additional Plugin Contents

Load OMSFromPath Blueprint function

The LoadOMSFromPath node loads an OMS file from the specified path. This Node is not necessary for standard OMS Player workflows.

HoloSuite Player Class

Not Implemented

OMSMedia Player

Not Implemented

OMS Media Sound

Not Implemented