Player Reference¶
Note
This is the reference documentation for the HoloSuite Player Unity Package.
For a step by step guide to setting up a HoloSuite Player in Unity, refer to the Unity: HoloSuite Player Configuration Example document.
Overview¶
HoloSuite Player is a Unity package used to play compressed volumetric video clips created using HoloEdit. This package contains the scripts and supporting assets necessary to play embedded or streaming OMS clips in your Unity project. With further customization users can integrate the HoloSuite Player with their Unity scene logic, configure dynamic content, create new VFX shaders, and more.
This page describes the contents of the HoloSuite Player Package, including an in depth description of the HoloSuite Player, its functionality, and properties. If you need to quickly get up and running, refer to the Unity: HoloSuite Player Configuration Example for a step-by-step guide.
Installing the HoloSuite Player¶
The HoloSuite Player Package officially supports Unity version 2019.4(LTS). To create a new project using the HoloSuite Player Unity package, first create a new unity project. If you will be working with Unity’s Universal Render Pipeline or HDR Pipeline, create a new project using the appropriate template.
After creating your new project, open the Package Manager window. Click the + button and select Add package from tarball. Navigate to the HoloSuite/Plugins folder (located at C:\Program Files (x86)\HoloSuite\HoloEdit 2022.2\Plugins using default installation location*) and select the HoloSuite Player Unity Plugin.tgz file, then import the package.
Upgrading the HoloSuite Player Plugin to a Newer Version¶
From the Unity Package Manger window, follow the same instructions to install the plugin in a new project: Navigate to the new HoloSuite Player Unity Plugin.tgz file and click import. When successful, you will see the plugin version number update in the plugin list.
If you need to update any of the sample content, delete the samples that were already imported and then import them again.
Upgrading Existing OMS Player Projects¶
The OMS Player is our legacy player for playing OMS files in Unity. This has been replaced by the HoloSuite Player. If you are using this legacy player in a existing project, it is not necessary to update this project to the new package. While the new package offers numerous advantages, there is not a seamless migration process to quickly update your project: any custom scripts will need to be updated to reference new package assets, and all OMS players will need to be re-configured with the HoloSuite Player script.
If you want to upgrade a project, first make sure there are no assets you rely upon saved anywhere in the Assets/Arcturus/ subfolders. If there are any assets there, you’ll want to move them into another folder. Next you can import the HoloSuite Player package per the instructions above. Once the package has been imported you’ll need to delete the Arcturus folder within Assets. It’s likely Unity won’t like doing this while it’s open, so we recommend closing the program and deleting the folder via File Explorer instead. Once you have deleted the folder, reopen your project. For each saved scene you’ll need to add a HoloSuite Player component to all former OMS players, and then configure it to match the OMS Player script. Finally, delete or disable the OMS Player script and Playback Speed components on the game object.
If you’re also planning on using one of Unity’s advanced render pipelines (URP or HDRP) you’ll need to refer to Unity’s documentation on updating an existing project.
HoloSuite Player Unity Sample Content¶
After the HoloSuite Player Unity package has been added to a project, you can select it in the Package Manager to find a number of sample asset packs:
None of these packs are necessary for using the HoloSuite Player, but each does provide a helpful starting place to build your project from. To add a sample to your project simply click the Import into Project button beside it. Here’s what you can expect from these sample packs:
HoloSuite Player: Includes two example scenes that will demonstrate standard playback and playback with dynamic retargeting.
HoloSuite Player HDRP: Includes two demo scenes that make use of Unity’s High Definition Render Pipeline and Shader Graph. One standard, and one with dynamic retargeting.
This sample will require you to have also imported the HDRP package and configured your project accordingly. Please see Unity’s HDR Pipeline documentation for more information.
HoloSuite Player URP: Includes two demo scenes that make use of Unity’s Universal Render Pipeline and Shader Graph. One standard, and one with dynamic retargeting.
This sample will require you to have also imported the Universal RP package and configured your project accordingly. Please see Unity’s URP Pipeline documentation for more information.
HoloStream: Includes two example scenes that will demonstrate streaming playback. The minimal scene has no playback controls while the standard streaming scene includes a UI, a clip library to change between a number of clips on the fly, and a debug menu.
Getting Started¶
Using the HoloSuite Player requires that at least one OMS and corresponding MP4 file are imported into your project. You will also need a GameObject that is configured using the HoloSuite Player script. To quickly get started with OMS playback, add the HoloSuite Player sample to your project and modify the Ewan scene to use your OMS and MP4 files. Below you’ll find additional details about the contents of the package and general use of the HoloSuite Player script.
For a more in depth walkthrough of setting up the HoloSuite Player, refer to our Unity: HoloSuite Player Configuration Example page.
Note
The Unity HoloSuite Player offers playback using StreamingAssets and native asset paths, but MP4 files must be exported with frame encoding to be used as native assets. A telltale sign that an MP4 requires frame encoding is that the texture will update on the mesh, but the mesh does not animate. Refer to the MP4 export documentation for more information.
Build Targets¶
HoloSuite Player can be built for Android, IOS, Windows 10/11, and Mac OS X.
The HoloSuite Player Script¶
The HoloSuite Player Script can be applied to any GameObject to enable it to play OMS files.
The HoloSuite Player script contains the following inspector settings:
Source Type¶
The Source Type dropdown enables you to indicate where the OMS and MP4 assets are within your Unity project. Assets should be used when your clips have been imported as native Unity assets. They can be populated using click-and-drag or the target button. This is the default source type, and should be used whenever possible. Streaming Assets(Legacy) should be used when your clips have been imported into the StreamingAssets sub-folder. This is only needed when it is required to make files avaliable to the normal filesystem on the device you will be deploying to. For more information refer to Unity’s Streaming Asset documentation
Warning
As a reminder, if you are using MP4 files that have not been exported using frame encoding, you must use the Streaming Assets import type. Failure to do so will result in clips that do not play correctly. Please re-export your MP4 file(s) with frame encoding enabled if you want to import your files as native assets.
No matter the chosen source type, you will need to provide your clip information for the following fields:
OMS Source/Filename: The OMS asset to be played. When using Streaming Assets, include the relative file path from the StreamingAssets folder and the .oms extension (Eg: MySubfolder/MyClip.oms).
Video Clip/Filename: The MP4 asset to be used. As above, when using Streaming Assets you must include the relative file path and the .mp4 extension.
Material: Specify the material to be used by the renderer.
Playback Settings¶
These settings can be used to modify the playback behavior of your clip.
Play On Awake: Toggles whether or not the clip plays automatically when the scene is run.
Loop: Toggles whether or not the clip loops when it reaches the end of playback. When looping is disabled, the clip will hold on the final frame once it has reached the end.
Playback Events¶
These events are emitted by HoloSuite Player as various internal events happen. Create Listeners to update custom components.
On Load: The clip with the given index has been loaded.
On State Change: Player state has changed to the given state.
On Playtime Changed: The current playback time has changed to the given time (seconds).
On Buffertime Changed: The amount of buffered-ahead content has changed to the given time (seconds). Streaming mode only.
On Error: Playback has failed with the given error message.
Retarget¶
Enable this checkbox for an OMS which is intending to use dynamic retargeting. See the Retargeting Overview for more information on preparing a clip for retargeting.
When Retarget is enabled, additional configuration options become available:
Skeleton Root: Select the skeleton generated using the clip’s OMS file after adding it to the scene hierarchy.
Retarget Events
On IK Update: HoloSuite Player fires this event when it is time for IK solvers to modify the pose of the retargeting skeleton.
HoloSuite Player Materials and Shaders¶
For the HoloSuite Player to correctly play the animated mesh content of an OMS file it must use one of the Arcturus custom shaders.
Unity Built-in Render Pipeline Materials¶
Generate a new HoloSuite Player material by right-clicking in the project window and selecting Create > Arcturus > (Current Render engine) > (Material Type). There are two material types, Lit and Unlit. Both of these types are avaliable in the built-in, URP, and HDRP render pipelines. In the built-in pipeline these material’s shaders are written in HLSL. In the URP and HDRP pipelines, these materials use Shader Graph.
Built-in¶
Lit Material The HoloSuite Lit Material uses a SkinnedSurface surface shader that exposes the albedo texture (driven by the HoloSuite Player), smoothness, and metallic values, in addition to a color tint (driven by the HoloSuite Player per-track).
Unlit Material The HoloSuite Unlit Material is an unlit SkinnedMesh shader that exposes the albedo texture (driven by the HoloSuite player) in addition to a color tint (driven by the HoloSuite player per-track).
URP¶
All URP HoloSuite Player shaders are Shader Graph materials. The URP HoloSuite Player Shader Graph materials are driven by the ArcturusOMS_URP and ArcturusTexture_URP nodes. Instructions for ArcturusOMS_URP setup can be found within the node in the Shader Graph.
Lit Material The HoloSuite Lit URP Material is a prebuilt Shader Graph material for playback within the Universal Render Pipeline. This Shader Graph is using the PBR Master output node.
Unlit Material The HoloSuite Unlit URP Material is a prebuilt Shader Graph material for playback within the Universal Render Pipeline. This Shader Graph is using the Unlit Master output node.
HDRP¶
All HDRP HoloSuite Player shaders are Shader Graph materials. The HDRP HoloSuite Player Shader Graph materials are driven by the ArcturusOMS_HDRP and ArcturusTexture_HDRP nodes.
Lit Material The HoloSuite lit HDRP Material is a prebuilt Shader Graph material for playback within the High Definition Render Pipeline. This Shader Graph is using the Lit Master output node.
Unlit Material The HoloSuite Unlit HDRP Material is a prebuilt Shader Graph material for playback within the High Definition Render Pipeline. This Shader Graph is using the Unlit Master output node.
Editing HoloSuite Shader Graph Materials¶
HoloSuite Shader Graph materials can be edited to fine tune the look of your OMS clip. To do this, you will need to input your changes in-between the Arcturus custom nodes and the output node.
Take a look at the two examples below to get an idea of how to do this. Any time you would use a Position or Normal node in a standard Shader Graph, just take the Position, or Normal outputs from the ArcturusOMS(_URP, or _HDRP) node. Just be sure that there is a path for the Position and Normal data from the ArcturusOMS_URP node to reach the output node. If that path is broken the Mesh will stop animating correctly.
Inflate¶
In this example the vertex positions are being offset in the direction of their normals, by their distance from a vector position. This causes the OMS to inflate.
Display Normals¶
This example shows adding the normal value to the texture color. This displays the normals, and can be useful for troubleshooting.
Extensions¶
Extensions are additional components that can be added to the HoloSuite player to extend its functionality. For more information on the HoloSuite Player extensions, take a look at the Unity: HoloSuite Player Extensions page of our docs.
Retargeting Overview¶
One key feature of OMS playback is Retargeting. Retargeting allows you to dynamically re-pose your volumetric capture like any other Skinned Mesh in Unity without disrupting playback. Using procedural animation code such as IK look-at scripts, you may re-animate any bones embedded in the OMS via the Generate Skeleton stage.
If Skin Weights have been provided, either manually or via the Generate Skin Weights for Head Retargeting stage in HoloEdit, the weighted vertices will be transformed accordingly.
One common usage of Retargeting is “Head Retargeting,” using the head bone to make the volumetric actor look at an animated object or camera in the scene during playback. Clips can be prepared for Head Retargeting in HoloEdit using the Generate Skeleton and Generate Skin Weights For Head Retargeting stages without any manual editing.
See the Retargeting Setup guide for a step by step walkthrough for setting up head retargeting in Unity.
The HSP Look At Controller¶
The HSP Look At Controller extension is provided with the HoloSuite Player package and can be used to orient a transform to track another at runtime. Typically this script is used for the Head Retargeting workflow, to enable a volumetric actor to track the scene camera by turning their head. For more information on the HSP Look At Controller, please refer to the HSP Look At Controller portion of the HoloSuite Player Extensions document.
Generating OMS Skeleton Assets¶
You can quickly generate the skeleton asset required for retargeting by right-clicking on an OMS asset and selecting Create > Arcturus > Create Retarget Skeleton option.
When selected, an <OMS filename> Skeleton prefab will be created adjacent to the target OMS file. A console error will be thrown if you try using the function on a non-OMS asset, or if the targeted OMS file does not contain retargeting data.
Note
.oms files will not be recognized as OMS-type assets while they are imported into the StreamingAssets folder. You can move a .oms file from StreamingAssets into the main asset tree to use this operation and generate a skeleton prefab, then move it back into the StreamingAssets folder afterwards.
When working with your own custom retargeting scripts, you will need to trigger it using the DoUpdate() function. To configure this, just add your retargeting scripts retarget function to the HSPLookAtController.DoUpdate() event. You will find this under the Retargeting Events drop down on the HoloSuite Player component.
Appendix: Unity Package Contents¶
Importing the HoloSuite Player Package will add its contents to the Unity Packages folder tree. Contents of the Packages folder are both read and write enabled. This means that all assets are visible to you, even if you have not imported the sample packages they belong to, and you are free to edit any of the contents. When you import a sample using the Unity Package Manager it will create a local copy of the assets within your project folder.
HoloSuite Player¶
The following content is included when you import the HoloSuite Player Package:
Editor: Scripts and assets used for example UI.
Plugins: Plugin dependencies.
Runtime: Holds most HoloSuite Player core assets.
Extensions: Extension Components to increase the functionality of the HoloSuite Player.
Materials: Materials used in examples or for OMS Playback.
Prefabs: OMS Player Prefabs for use in user created scenes. See example scenes for examples of prefab usage.
Scripts: Various scripts related to OMS Playback and OMS Player configuration.
Shaders: Shaders used for OMS Playback.
Textures: Data textures used for some OMS Playback features.
Samples: Contains all sample contents.