Unreal: OMS Player Reference

Note

This is the reference documentation for the Unreal OMS Player Plugin.

For a step by step guide to setting up an OMS Player in Unreal Engine, refer to the OMS Player Configuration example document.

HoloStream Plugin Install

Unreal Engine version 4.25 is supported for OMS playback.

To Install the HoloStream UE4 plugin locate the HoloStream UE4 Plugin 2021.1.zip file in the Tools folder of your HoloSuite 2021.1 file directory. Drag and drop the HoloStreamPlugin folder into your Unreal Engine Plugins folder. By default, this should be: “C:\Program Files\Epic Games\UE_4.25\Engine\Plugins”. See the Plugins page on the Unreal Engine Documentation (external link) for more information on installing plugins in Unreal Engine 4.

To finish installing the plugin, create a new UE4 project. From the main Unreal Engine interface, click on Settings in the top menu, and select Plugins. In the Plugin editor, a HoloStream Player item will be visible. Check ‘Enabled’ and click the “Restart Now” button in the bottom right of the window. When Unreal opens again, you’ll be ready to start configuring the OMS Player.

Getting started

Each OMS player requires the following files created in Unreal Engine in addition to your mp4 and OMS:

Read on below to learn about the contents of the HoloStream Plugin. To get started on using the HoloStream Plugin right away, jump to the OMS Player Configuration example

Overview

The Unreal OMS Player is included in Plugin called “HoloStream Player”. The contents of this plugin can be used to play back compressed Volumetric Video created in HoloEdit. The Plugin package contains c++ classes, blueprint functions, and material functions.

OMS Playback uses the OMSMeshactor class, which follows the Media Framework (External Link) for video playback, and regular UE4 materials using a custom Material Function for rendering.Users can integrate the OMS player with any material graph or blueprint scene logic.

This documentation page describes the contents of the HoloStream Player Plugin, along with a brief example of configuring an OMS Player.

The HoloStream Player supports most OMS Playback features. These include compressed OMS playback and retargeting.

Overview: Retargeting

One key feature of OMS Playback is Retargeting. Retargeting allows you to dynamically re-pose your volumetric capture like any other Skinned Mesh in Unreal without disrupting playback. Using the AnimGraph, you may re-animate any bones embedded in the OMS via the Generate Skeleton stage.

If Skin Weights have been provided, either manually or via the Generate Skin Weights for Head Retargeting stage in HoloEdit, the weighted vertices will be transformed accordingly.

One common usage of Retargeting is “Head Retargeting”, using the head bone to make the volumetric actor look at an animated object or camera in the scene during playback. Clips can be prepared for Head Retargeting in HoloEdit using the Generate Skeleton and Generate Skin Weights For Head Retargeting stages without any manual editing.

See the Retargeting Setup section below for a step by step guide on setting up Head Retargeting in Unreal Engine.

Plugin OMS File Support

With the HoloStream Player plugin installed, OMS files added to your project are automatically detected as OMS files. In addition to playback with the OMSMeshActor class, OMS files have the following functionality:

Right click in the Content Browser

  • SkeletalMesh: A SkeletalMesh component for the embedded skeleton can be created automatically from any OMS containing rigging data. This will appear above the default functionality in the context menu, under the “OMS Actions” header.

The OMSMeshActor Class

The OMSMeshActor class is used to create OMS players in your scene. It drives necessary components of the Arcturus OMS Material function and can be configured to specify an OMS file and enable or disable Retargeting. This class drives OMS playback in conjunction with an Arcturus OMS Material Function.

The OMSMeshActor class contains the following properties:

Default:

  • Media Player: Specify a Media Player configured with a corresponding Media Texture. See Media Player configuration for more information.

OMS Settings:

  • OMS: Specify an OMS file for playback.

  • Media Player Material: Specify a Material for the Actor’s Media Player. See Media player configuration for more information.

  • Mesh material: Specify a Material for OMS Playback. This material should include an ArcturusOms Material Function for proper playback.

Retargeting:

These properties allow you to configure Retargeting. See Retargeting Configuration For more information.

  • Retarget Enabled: Enable Retargeting.

  • Retarget Skeletal Mesh: Specify a Skeletal Mesh for Retargeting.

Understanding the OMSMeshActor class

Configured correctly, your OMSMeshActor will drive the Arcturus OMS Material function in an assigned Material for OMS Playback.

In order to play back correctly, you will need to configure the properties of your OmsMeshActor to the corresponding OMS, along with providing the blueprint nodes necessary to drive your media player.

The OMSMeshActor class conforms to the MediaPlayer framework, so it uses the Event Graph to drive an Open Source node with a Media Player and a Media Source.

../_images/mediaplayersetupinblueprint.PNG

A correctly configured OMS player event graph

To play the OMS selected in the properties, add an Open Source Blueprint Node and connect the BeginPlay event to the Execution Pin.

  • The Target should be set with a Get Media Player node configured with the same Media Player assigned in this class’s Properties.

  • The Media Source property should be configured with the Mp4 file corresponding to your OMS.

The Arcturus OMS Material Function

The Arcturus OMS Material function is used to animate and texture OMS files for playback. Used in combination with an OMSMeshActor Class this Material function drives OMS playback. The ArcturusOMS node has three Output Pins:

  • Texture: The Color texture sampled from the Media Player assigned to the object’s OMSMeshActor class.

  • Normals: The Mesh Normals produced from SSDR deformation.

  • World Position Offset: The World Position Offset required to deform the mesh for playback.

For a simple OMS Player, these pins can be connected to the Emissive (or base) color, Normal, and World Position Offset outputs, respectively. In your Custom Materials, apply additional nodes as needed between the ArcturusOMS function and your output node.

Configuring a Media Player for OMS Playback

Media Player requirements for OMS playback are as follows:

  • One Media Player configured to control Playback. Enable Loop and Play On Open.

  • One Media Texture. Configure the Media Player property to select the corresponding Media Player you created.

  • One Material to shade the Media Player playback. This Material should be configured with the Unlit shading mode, and contain one Texture Sample node connected to the Emissive Color output pin. In the Texture Sample node, set the Sampler Type to External and the Texture to the Media Texture you created.

When this configuration is complete, assign the Material and Media Player to the corresponding properties and nodes in your OMSMeshActor.

Retargeting

Retargeting allows real time animation during playback of an OMS file containing rigging data.

To enable retargeting, you must configure a Skeletal Mesh, Skeleton, and Animation Blueprint.

The components of Retargeting are as follows:

  • SkeletalMesh: The SkeletalMesh component must be created automatically from any OMS containing rigging data. To create an OMS Skeletal mesh, right click on an OMS file in the Content Browser and choose “Create Retarget SkeletalMesh” from the OMSActions section.

  • Skeleton: The Skeleton component must be created via right clicking on the SkeletalMesh component and choosing “Create Skeleton” from the “Skeleton” category.

  • Retargeting Blueprint: Retargeting uses an Animation Blueprint to drive animation during playback. To create a Retargeting Blueprint, create a new Animation Blueprint.

    • After configuring the Animation Blueprint as necessary for your project, add an instance of the blueprint to your scene and configure the “Retarget Skeletal Mesh Actor” property on your corresponding OMSMeshActor to point to the Animation Actor. During playback, the Blueprint logic specified by the Animation Blueprint will be applied to the OMS Player.

Appendix: Additional Plugin Contents

Load OMSFromPath Blueprint function

The LoadOMSFromPath node loads an OMS file from the specified path. This Node is not necessary for standard OMS Player workflows.

HoloStream Player Class

Not Implemented

OMSMedia Player

Not Implemented

OMS Media Sound

Not Implemented