HoloSuite Player Unity Configuration Example¶
Note
This is a step by step guide to setting up a HoloSuite Player in Unity.
For the reference documentation for the Unity HoloSuite Player Plugin, refer to the Unity HoloSuite Player Reference Documentation.
The HoloSuite Player supports playing back your OMS in Unity’s Built-in render pipeline, URP and HDRP.
HoloSuite Player Configuration example¶
The following is a walkthrough of a typical HoloSuite Player configuration:
To begin, ensure all necessary packages are included in your project. For instructions on how to import the HoloSuite Player Package please refer to the HoloSuite Player Unity Installation documentation.
If you have imported any samples from the Package Manager, you can preview completed Example Scenes found in Assets>Samples>HoloSuite Player>2022.2.0>HoloSuiteplayer(importversion)>Scenes. The second HoloSuitePlayer folder will be named to reflect which sample project you imported into your project.
Sample Volumetric Data will also be provided in the sample assets.
For more information on the HoloSuite Player sample content review HoloSuite Player Unity Sample Content.
The Unity HoloSuite Player supports importing OMS and MP4 as though they are native assets as long as frame encoding is present in the MP4 file. You can drag and drop OMS and MP4 files into any folder in your unity project. OMS and MP4 data that do not include frame encoding must be located inside of the StreamingAssets directory, located in the Assets project folder. (Assets/Streaming assets)
HoloSuite Player Materials¶
The Core function of the Holo Suite Player component is the same for each render pipeline, but there are some differences in the materials and shaders used for each render pipeline. Please create the material specific to the render pipeline used in your project:
Built-in Render Pipeline: To create a material for the Built-in pipeline right click in an empty space in the Project Window, then select Create > Arcturus > Holosuite Lit/Unlit Material (select which material fits your needs)
URP: To create a default material for the URP right click in an empty space in the Project Window, then select Create > Material. Select the new material. In the inspector select the shader dropdown menu. You will find the default URP shaders at Arcturus>URP. Select the shader type that suits your project needs.
To create a Shader Graph material for the URP right click in an empty space in the Project Window, then select Create > Material. Select the new material. In the inspector select the shader dropdown menu. You will find the HoloSuite URP Shadergraph shader at Shader Graphs>HoloSuitePlayerURPShaderGraph.
HDRP: Only Shader Graph materials are supported for the HDRP. Right click in an empty space in the Project Window, then select Create > Material. Select the new material. In the inspector select the shader dropdown menu. You will find the HoloSuite HDRP premade Shadergraph shaders at Shader Graphs>HoloSuitePlayerHDRPLit or Shader Graphs>HoloSuitePlayerHDRPLit.
HoloSuite Player Configuration¶
Player Configuration is provided by the ‘Holo Suite Player’ script. To use a HoloSuite player, add the script to a new GameObject in your scene.
Several fields will need to be updated in the Holo Suite Player (Script) component:
Source Type:
If your MP4 contains frame encoding leave the source type at the default ‘Assets’ option.
If your MP4 does not contain frame encoding select ‘Streaming Assets’ From the drop down menu.
Navigate to the folder containing your OMS file. Click and drag the OMS from the directory into the OMS Source field.
Navigate to the folder containing your MP4 file. Click and drag the MP4 from the directory into the Video Clip field.
Assign the material that matches your render pipeline requirements. If you are unsure, review the details of the HoloSuite Player Materials Setup.
Enable the Shader Graph Material if working with the HoloSuitePlayerShaderGraph material. Otherwise leave it disabled.
Expand the Playback Settings options.
Ensure that Play On Awake and Loop are enabled unless otherwise required based on your project needs.
You can now hit Play in the Unity editor and preview your HoloSuite Player.
Configuration example: Retargeting Setup¶
After configuring your OMS clip for playback, you may want to add Retargeting functionality.
For reference information on OMS retargeting refer to the Retargeting Overview from the Unity HoloSuite Player reference documentation.
If your OMS contains the necessary retargeting data (a skeleton, a skin binding, and skin weights for one or more bones) you can perform the following steps to configure Retargeting playback:
Navigate to your OMS that contains retargeting info in the project directory.
Right click on The OMS and select ‘Create Retarget Skeleton’.
Select the new skeleton prefab and add it to your scene hierarchy.
Enable The Retarget option in the Holo Suite Player script.
Select the HoloSuite Player object, from the hierarchy click and drag the Skeleton into the Skeleton Root field.
After following these steps, your clip will be configured for Retargeting. Play your unity scene to review your clip and ensure it is configured properly: if it looks like it did before you started configuring for retargeting, it’s correct.
Once retargeting has been configured you can use any runtime unity scripts of your choice to animate the skeleton during playback. Changes made to the skeleton transformations will be applied to your clip during playback.
To set up Head Retargeting, we provide the SwingTwistLookIK script in the HoloSuite Player package. You can configure it according to the following steps:
Add a Swing Twist Look IK component to your skeleton object. This is the component you created in step 2 above.
Expand the “Bones” dropdown in the Swing Twist Look inspector, and set the size to “1”. This will expose an “element 0” field.
Assign your head bone in the Element 0 field. To do this, click and drag the “Head” bone (a child of the skeleton prefab: > hips > back > shoulders) onto the element 0 field. This will specify the head bone transform as the gameobject for the script to act on.
Assign your head bone in the “Effector” field. Optionally, you can create a new child object for the head (such an empty gameobject, or a sphere) and supply that object as the effector. The effector will be used to determine the relative rotation of the selected bone to the look at target. Supplying a different object than the head lets you rotate that object to provide a rotational offset, which can be useful if the head is in an extreme pose, or the performer’s eyes are not looking straight forward.
Provide a Target object. Drag any object from your scene into the “Target” field. This is the object your Head Bone will be oriented to look at. This object is often the Main Camera in a scene, so that the volumetric actor can look directly at the viewer in a VR or AR app.
Configure the look at settings according to your needs. The default settings quickly track the target object within a narrow range in front of the actor. Several settings are provided for in depth configuration, with unity tooltips available on mouse over. For more details, see the Swing Twist Look Ik script reference.
Call SwingTwistLookIK.DoUpdate From the On IK Update () event on your Holo Suite Player:
Scroll down to the bottom of your Holo Suite Player inspector to configure the On IK Update () event.
Press the “+” button to add a new listener.
Drag your skeleton prefab object (the object containing the swing twist IK script) onto the “None (Object)” field.
Using the “No Function” dropdown, Select “SwingTwistLookIK -> DoUpdate()”. This will enable the HoloSuite Player to call the SwingTwistLookIK script to animate the skeleton.