Retargeting¶
Note
This is a step by step guide to setting up OMS playback in Unreal with retargeting.
For the reference documentation for the Unreal HoloSuite Player Plugin, refer to the Player Reference Documentation.
One key feature of OMS Playback is Retargeting. Retargeting allows you to dynamically re-pose your volumetric capture like any other Skinned Mesh in Unreal without disrupting playback.
Configuration¶
One common usage of Retargeting is “Head Retargeting”, using the head bone to make the volumetric actor look at an animated object or camera in the scene during playback. Clips can be prepared for Head Retargeting in HoloEdit using the Generate Skeleton and Generate Skin Weights For Head Retargeting stages without any manual editing.
Inside Unreal Engine, you must configure a Skeletal Mesh, Skeleton, and Animation Blueprint to enable retargeting. The components of Retargeting are as follows:
SkeletalMesh: The SkeletalMesh component must be created automatically from any OMS containing rigging data. To create an OMS Skeletal mesh, right click on an OMS file in the Content Browser and choose “Create Retarget SkeletalMesh” from the OMSActions section.
Skeleton: The Skeleton component must be created via right clicking on the SkeletalMesh component and choosing “Create Skeleton” from the “Skeleton” category.
Retargeting Blueprint: Retargeting uses an Animation Blueprint to drive animation during playback. To create a Retargeting Blueprint, create a new Animation Blueprint.
After configuring the Animation Blueprint as necessary for your project, add an instance of the blueprint to your scene and configure the “Retarget Skeletal Mesh Actor” property on your corresponding OMSMeshActor to point to the Animation Actor. During playback, the Blueprint logic specified by the Animation Blueprint will be applied to the OMS Player.
Head Retargeting¶
To add head retargeting data to your player, first configure a standard OMSMeshActor as described in section Getting Started and ensure it’s playing correctly in the viewport.
Next, ensure that Retargeting Enable is selected in your HoloSuite player’s Blueprint Details panel and follow the steps below.
Adding skeletal mesh and bones:¶
OMS playback with retargeting requires a skinned mesh and skeleton data to control your actor’s motions. These can be generated from an OMS file containing embedded rigging data.
To generate a new SkeletalMesh, right click on your OMS file and select “Create Retarget SkeletalMesh”.
Next, right-click on the newly created SkeletalMesh and under Skeleton, click Create Skeleton.
You should rename these appropriately, as both the mesh and skeleton assets require specific configurations.
Retargeting Blueprint:¶
Adding an Animation Blueprint is what enables head retargeting to lock onto a target object and control your actor’s head motion during playback.
For example, you can add a Look At blueprint using the following steps:
In your OMS_Player folder, right-click to create a new asset, and select the Animation Blueprint from the Animation dropdown menu. Animation Blueprints must be provided a target skeleton. Select the skeleton you generated in the previous steps and press the “ok” button.
Open your Animation Blueprint to expose the AnimGraph window.
In the AnimGraph, you will see an empty Output Pose node.
First, Create a Look At node and connect its output to the Result of the Output Pose. With the Look At node selected, update its Details as follows:
Update its Bone to Modify to ‘head’
Expand the Look At Axis category, and set the Look at Axis values to 0.0, 0.0, 1.0
Next, Compile and Save the Animation Blueprint.
After Compiling the Animation Blueprint, you can drag it into a level containing your OMSMeshActor.
If you’ve transformed your player actor in any way, ensure that your animation blueprint Actor has the exact same rotation, scale, and position as your OMS player Actor.
Adding a look at object¶
In order to get a useful result, you’ll also need a target for your ‘Look At’ script to point the head towards.
Place a new object in the scene. For example, a primitive such as a sphere.
In the Animation Blueprint, select the Look At node and in its Details panel, check the “Expose” checkbox next to the Look At Location setting in the Target tab.
This will expose the Look At Location pin in the AnimGraph.
Next, add a variable to use as the look at location, and in the Details tab set its variable type to Vector.
Drag your new Vector variable into the AnimGraph as a Get node and connect it to the Look At Location input of the Look At node.
Next, switch over to the Event Graph of the Animation Blueprint. Add the following assets and connections between the nodes:
Pull from the “Event Blueprint Update Animation” exec pin and Create a Get All Actors with Tag function; in the node’s Tag field, specify any name for a tag. For this example, use “LookAtTarget”.
Click on the Get All Actors with Tag Out Actors output and create a Get (a ref) node.
Click on the Get node’s output pin and create a GetActorLocation node
Drag out from the GetActorLocation node’s Return pin and create a Set node for your vector variable.
Connect the Get All Actors with Tag node’s Exec output to the Set node’s Exec input.
Compile and Save your Animation Blueprint
In the viewport, select the primitive ‘look at’ object that you added to your scene.
In the object’s Details panel, navigate down to the Actor tab and add a Tag element. In the new field that is created below, add the ‘LookAtTarget’ tag that you entered in the Get All Actors with Tag node.
You can hide your skeletal mesh in the viewport by assigning a custom material to it. In your OMS_Player folder, create a new material. Open the material, set the blending mode in its Details panel to Translucent, create a Constant node in the material editor, and connect it to the material’s Opacity channel. Back in your OMS_Player folder, open your skeletal mesh object and assign the translucent material to the skeletal mesh’s Material Slot.
Select ‘Play’ in the viewport toolbar, moving your target object will now cause the actor’s head to follow the target.