HoloEdit 2023.1.0 Release Notes¶
New Features - HoloEdit¶
New Feature - AVV File Type¶
Support has been added to HoloEdit and our Unreal and Unity Players for our new player format: Accelerated Volumetric Video (or AVV). Note that due to the potential of this new format, it has become HoloEdit’s first premium feature and will require an additional licese in order to export AVV files.
For additional details on when you’ll want to use OMS -VS- AVV, we’ve put together a short blog post discussing their key strengths and differences. To support this new format you’ll find the following additions to HoloEdit:
New Stage: Analyze Texture¶
The new Analyze Texture stage evaluates the textures on a mesh to determine where the areas of highest detail are. This detail evaluation is then used to assign priority and texture space to the AVV export’s textures. This stage currently has no uses beyond preparing an AVV file.
New Stage: Transfer Normals¶
The new Transfer Normals stage is able to apply per-frame normals to a post-SSDR mesh. This allows AVV to retain smooth normals during playback while still benefiting from our compression pipeline. This stage currently has no uses beyond increasing the quality of an AVV file.
New Stage: Detect Face¶
The new Detect Face stage is able to identify an actor’s face. This can be used with the updated Clean Mesh or Generate Skeleton stages, or to give the face priority texture space in an AVV export’s textures.
New Export Type: AVV¶
AVV has been added as a new export configuration for the composition export manager, which will output a single .AVV file with embedded mesh, textures and audio, as well as a skeleton if desired. At this time, AVV files can be played using the HoloSuite Player for Unreal or Unity.
Updated Stage: Clean Mesh¶
The Clean Mesh stage has received several major updates:
Data returned from a Detect Face stage can be used to decimate meshes while maintaining the geometric fidelity of the face
Clean Mesh preserves stable mesh segments, enabling you to fix geometric errors and decimate meshes without having to re-stabilize them after doing so
Data returned from a Generate Skeleton stage can be used with the new Skeleton Mask operation during decimation. This allows you to specify regions of the body to exclude from the decimation algorithm
A new Preserve UVs option (defaults to enabled) has been added to the decimation parameters. Disabling this option will allow the mesh to be decimated without regard for the existing UVs and allowing the mesh to be decimated more uniformly and to a higher degree (lowers the floor of how far a mesh can be decimated). Generate UVs and Transfer Texture stages will need to be used afterwards to restore the texture to the mesh
New Stage: Optimize Texture¶
The new Optimize Texture stage allows for a texture to be scaled down to a less costly resolution, and is able to use data returned from a Detect Face stage to intelligently give facial geometry a higher resolution in the resulting texture. Even without utilizing Detect Face this stage allows you to resize a texture within HoloEdit, or to add texture padding around your UVs.
Proxy Texture Shading Mode¶
When a clip is created in HoloEdit, if it uses textures that exceed a 2880x2880 resolution, hidden proxy textures are automatically generated at a resolution of 1440x1440. These new textures can be used in the HoloEdit viewport in order to reduce asset load times and increase the overall responsiveness of the application. These proxy textures can be displayed by selecting the Shading menu at the top of the viewport and enabling the Proxy Textures option at the bottom of the list. Even when Proxy Textures are enabled, they will never be used for any processing by HoloEdit - the original textures will be submitted for all work in jobs and used for exports.
Improvements¶
Improvements To Data Dirtiness Evaluation¶
Dirtiness is displayed as a yellow caution bar on your interval when HoloEdit has evaluated that it should be re-run to account for changes to the composition which would impact the result. There were a number of states that a composition could enter which resulted in dirtiness being displayed on intervals which did not need reprocessing and would not be cleared if they were run again, resulting in confusing messaging in the application. We have improved the evaluation process to account for more acceptable situations in an attempt to both remove the visual noise introduced to a composition by a lot of unnecessary dirtiness markers, and to make it so this indicator is useful to the user as often as possible. We hope the new indicator’s accuracy improves your experience with HoloEdit.
Audio Improvements¶
A number of fixes were made to our audio support in this version, including:
Significantly reduced the audio latency during playback in HoloEdit
Fixed an issue with the preview waveform rendering incorrectly in the Timeline
Fixed an issue that caused audio embedded in MP4 exports to not match its frame timing in HoloEdit
Fixed an issue that could cause mono-audio sources to play at incorrect speeds
Fixed an issue that could cause audio clips to play at an incorrect speed in HoloEdit
Together, these should help to significantly improve the experience of using audio files in HoloEdit. We know there is more work to do and will strive to regularly improve our audio support.
Improvements To Cloud Job Processing Responsiveness¶
We’ve included some improvements to our cloud processing resources which should see jobs start faster and processing occur more regularly. Previously jobs were processed in the order they were submitted, but now when multiple users have submitted work for processing the server will rebalance its available resources to allow all parties to receive processing power.
Large Mesh Scale Detection¶
A check and warning have been added to HoloEdit for the clip creation process. If the clip contains a mesh stream with unexpectedly large dimensions, a popup notification will appear warning you to reimport the clip before proceeding with processing. While it is possible to process a clip at an unusually large scale, doing so requires a solid understanding of stage attributes as the default settings for many of our stages will not yield usable results. It is strongly recommended you reimport the clip and adjust the Import Scaling Factor value found in the clip import window to bring the clip to an expected scale.
Improved Behaviour Of Autosaves¶
A number of improvements have been made to the Autosaves folder. The folder has been reogranized to use subfolders based on the composition of origin, and named so that the newest saves will appear at the top of the list. The way autosaves are generated has also changed. Autosave names indicate the action or job type that lead to their creation so that it’s much easier to determine what information a given autosave is for. There is now a “Before” snapshot to check a compostiion without fetching any job results, an “In Progress” snapshot for every result successfully integrated into the compostion, and a “Complete” snapshot for when a job has been finished in full. Autosaves should now be much more reliable for reverting to a previous version or recovering from an application crash.
Improved Behaviour Of Save Command¶
Previously the Save button would be disabled if the composition wasn’t first saved using Save As. This was problematic as there was no indication that nothing was happening if you were using the Ctrl+S hotkey to save, and closing an unsaved composition has no means of recovery. Now if you select the File > Save or Ctrl+S hotkey with an unsaved composition it will prompt the Save As command.
Additionally, using the Save command after opening an autosave will automatically prompt the Save As command instead. This should make it more difficult to lose work accidentally by working from and saving an autosave file, which exists outside of the normal workspace organizational space.
Removed Keyframe Indicator For Single-Frame Segments¶
In an effort to simplify the Timeline and increase performance, single-frame segments will no longer draw a keyframe indicator. This will most significantly impact a clip imported with the Detect Stabilized Sequences option when there are no stable sequences, and the results of a Remesh stage. All segments that are longer than 1-frame will still show a keyframe indicator.
Mesh Optimization Added To OMS Export Configuration¶
An optimization step has been added to the OMS export process which attempts to repair undesirable mesh attributes (primarily duplicate vertices) that can impact performance in the HoloSuite Player. To go along with this, a new option has been added for OMS exports: Preserve Mesh. Enabling Preserve Mesh will skip the optimization step, ensuring that you get exactly the same mesh data in your OMS that you had in HoloEdit.
Pad UV Islands Added To MP4 Export Configuration¶
A new MP4 generation option has been added which - when an MP4 is exported together with an OMS file - performs a small texture bleed to avoid UV seams appearing in the output. While this should have minimal impact on exports using the original texture resolution, this could easily occur if a large resolution texture was being downsized to a smaller resolution, and the negative impact on quality was quite significant. Where we’re seeing higher and higher resolution textures becoming commonplace this option has been added in an attempt to solve the problem before it proliferates.
Improved Tooltips¶
Many stage parameters have had their tooltips revised in an attempt to bring more information directly into HoloEdit and not have users need to rely on the online technical documentation. While the full documentaiton is still an excellent resource, the new tooltips will give you a much clearer indication of what a number of paramerters and values will do.
Reset To Default Added To All Lighting Window Parameters¶
It is now possible to revert all parameters in the Lighting window to their default state after being modified. We encourage you to use the features of this window to personalize your experience working in HoloEdit while remaining comfortable that you can always return to the defaults if desired.
Improved HoloEdit UI For Windows Screen Scaling¶
Several issues had been identified with the HoloEdit user interface if Windows scaling was enabled - this predominantly caused visual issues and did not impact the functionality of the application. Use of scaling is common practice when using a 4k monitor, and still likely when using a resolution higher than 1080p. The UI should now be consistent across all displays and scaling modes.
Fixes¶
Fixed Frame Encoding Bleeding Into Textures¶
We’ve added a set of routines to run on OMS export that validates if UV coordinates overlap with the frame encoding location, and adjust any frames where a collision would occur (UVs and textures are rotated or flipped accordingly, or scaled by 2 pixels if UVs occupy all four of the map’s corners). This fix should fully address the occasional black and white spots that would appear in processed clips.
Execute Missing / Dirty / Failed Functionality Fixed¶
When a job has failed or been cancelled, or the data stack has changed and cased work to be marked as dirty, additional commands appear in the right-click context menu that allow you to execute only a subset of the data present in selected intervals rather than reprocessing all work. Previously this function was resubmitting the full interval and would re-process valid data. This has been fixed and now only the specified type of work will be submitted when choosing these options. Additionally, there was a visual error that would indicate the entire interval was processing even if only a subset of work had been submitted. This visual error has also been resolved and an interval will accurately display only the ranges where work is being done.
Fixed Support For Processing From A Networked Drive¶
Several issues had been identified with how HoloEdit handled working from a workspace that was located on another computer on a local network. The reliablitly of working from a shared local drive should be much more reliable now that these issues have been fixed.
HoloEdit Crash When Closing A Composition With Multiple Tracks¶
A previously reported known issue has been resolved where HoloEdit crashed when closing a composition if it contained multiple tracks.
Fixed Safari (iOS) Streaming Content¶
A bug was identified that was preventing the HoloStream export processing from generating the assets required to play on iOS Safari browsers. Playback would be limited to only the first few seconds of the clip before it would desync completely. This bug has been fixed and iOS content is being correctly generated. A HoloStream clip will need to be re-exported in order to generate the missing assets.
Fixed Compute Time Counter Erroneously Continues To Increase¶
This previously reported known issue has been addressed. The reported compute time for jobs should correctly stop and total the expected amount.
Fixed Compute Time For Failed Jobs Shows Large Negative Number¶
Same as the above, this previously reported known issue has been addressed. The reported compute time for jobs should correctly stop and total the expected amount.
Fixed A Crash Resulting From SSDR Processing¶
There was a rare HoloEdit crash when returning processed SSDR results due to a poorly formed composition state. This state is now correctly handled and the crash will no longer occur.
Compute and Encode Now Display the Correct Balance While In MyMachine Server Mode¶
When HoloEdit was started while the My Machine processing mode was selected, the User Management and Streaming export screens would display a balance of 0 hours available. This behaviour has been corrected and both screens will now display the correct balance available to your account. Using the local processing mode still does not consume your compute balance, and a HoloStream can be exported after switching to a remote job server without restarting HoloEdit.
Corrected Numerous Issues With Program Files Permissions¶
A number of issues had been discovered with the application when running out of the Program Files directory (the default installation path). We have addressed the following:
Log files can be written and have been moved to
C:\Users\<USERNAME>\AppData\Roaming\HoloSuite\holoedit\logsCrash dump files can be written and are found adjacent to the
HoloEdit.exeapplication fileFBX files can once again be exported from HoloEdit
MP4 files can once again be used as the texture source for a clip
Fixed Clip Preview In Inspector¶
An issue had previously been introduced that caused the Inspector’s preview of mesh and texture data to be hidden. This has been addressed and you will now see a preview of any clip you have selected in the Project panel.
Fixed Drag And Drop Inconsistencies¶
Fixed an inconsistency where drag-and-drop anchor positions would differ depending on which window an asset was being dragged from. All items should now be anchored to the cursor from the top-left, and behave as expected with drop targets.
Changes¶
Import Options Are Displayed Per Data Type¶
Previously all import options were displayed at the bottom of the window regardless of the data type(s) that were being used to generate a clip. The interface now displays options contextually based on the selected data stream tab (this defaults to Mesh where the majority of these options exist).
Automatically Detect Segments For OMS Files¶
To enable a speedy conversion for OMS files to AVV, the importer has been updated to automatically detect segment data for .oms files. This will help users skip a potential pitfall and reduce friction with the workflow.
Increased Default Bitrate Value For MP4 Export Configurations¶
With the increased volume of volumetric captures using higher resolution textures, we’ve increased the default bitrate for MP4 exports to compensate.
Removed ASTC Texture Format From Export¶
The ASTC texture formats have been removed from the Texture-type export configuration. This file format was not longer used in any HoloSuite workflows and doesn’t have a general use case, so support has ended.
Known Issues¶
Match Template Mesh Jobs Report Failed Even When Successful¶
This issue will not prevent the Match Template Mesh stage from being used effectively. When jobs from this stage complete they will always show “Failed” for their status in the job viewer window, and the results on the timeline will be marked as dirty. However reviewing the results is necessary as the work may have completed without issue.
Small Audio Skip On Clip Loop In Editor¶
When playing back a composition with the Looping option enabled, audio isn’t buffered for the first 200ms of playback. The audio does exist and will export accordingly, but unfortunately makes it hard to preview if it is at the very beginning of the composition/clip.
Mesh Subtraction and Texture Reconstruction Workflows Do Not Work¶
There was a regression with the Mesh Capture Tool Unity project which now cannot be opened correctly. Similarly, the Texture Reconstruction stage needs an update to function correctly. As these items are intertwined and rarely used we have opted to address other more critical issues ahead of them, and re-evaluate how we will support these workflows in the future.
HoloEdit Performance Drops Significantly When A Large Timeline Is Being Rendered¶
There is an issue with the renderer which will cause HoloEdit’s performance to drop significantly if you try to view a large amount of information in the Timeline window. While we appreciate that this workaround is not the most convenient, zooming in to display a much smaller chunk of your work will dramatically improve the responsiveness of the application.
Adding The Lighting Panel To The Layout Breaks The Layout Of Previous Versions Of HoloEdit¶
If the Lighting panel has been opened in your layout and you try to open a previous version of HoloEdit (2022.1.1 or older) the layout will not properly load and HoloEdit will be rendered unusable. This can be corrected by going into a version of HoloEdit that contains the Lighting panel (2022.2 and newer), then removing it from the layout before closing the application.
HoloStream Export Jobs Can Fail With Large Clips¶
Unfortunately the definition of “large clips” is nebulous, as it depends on the size of the mesh and textures in combination with the clip’s length. You will see this manifest as a failed Finalizing work unit. We apologize for the inconvenience, but know that a fix in development for this issue which will also improve audio handling.
Cancelling An Export Does Not Cancel The Export¶
If you cancel an export task, the task will appear cancelled in HoloEdit, but the export will still continue.
Cannot Modify Multiple Intervals Settings Simultaneously¶
If you select multiple intervals from the same stage or stage type, modifying any value in the Inspector window will only update that value for the last interval that was clicked.
Running the Manage Workspace Disk Cleanup Tool While Any Saved Composition Has Available Job Results Will Cause HoloEdit to Crash¶
If you submitted a job for processing and then saved your composition and closed HoloEdit or opened another composition, the results for that job won’t be fetched until you open the composition again. If you run the disk cleanup tool before opening the composition and saving the downloaded job result, HoloEdit will crash while trying to review the assets in the saved composition. Odds are pretty good you won’t do this, but the easiest way to avoid it anyway is to only run the utility after you’ve finished downloading any outstanding jobs you had running.
Using Texture Reconstruction And Texture Transfer In A Single Job Returns Bad Results¶
An issue was discovered with our work-unit dependency calculation which results in a particular set of inputs and outputs being incorrectly handled for the order they should be processed by the server. Dependencies are calculated for complex jobs with numerous stages submitted for processing at once, so in this case the workaround is to ensure you don’t process both of these stages in one job.
Windows 11 Update Invalidates Held License¶
This issue is the result of the Windows update causing the computer to look like a different one than the license is currently assigned to. Licenses are automatically released after 24 hours, at which point the licenese can be re-acquired and everything will work fine again. Unfortunately to perform this action any sooner requires contacting our support team.
It is recommended that you release your HoloEdit license before updating to Windows 11 as this will sidestep the issue entirely.
Windows 11 Local Job Server additional warnings¶
On Windows 11 when running the local job server, the log will constantly return failed to get access token and failed to get compute time along with another two lines which reference “payment”. Using the local job server does not consume compute time - you can still run jobs and it will not eat into your compute time balance. This notification is effectively harmless and you are still able to run jobs on the local server.
Mesh Capture Tool First MVE Camera Capture Error¶
Unfortunately due to a timing issue with shader compilation in Unity 2019, the first MVE camera capture takes place before the model is correctly textured. This will be addressed in another version, however for now a workaround does exist within Unity:
Open the Mesh Capture Tool project
Edit > Project Settings > Editor
Scroll to the bottom and find the Shader Compilation section
Disable Asynchronous Shader Compilation
Web Player Texture Size Limitation (1440x1440)¶
The streaming content using the web player, textures larger than 1440 are not supported. Thus you cannot play quality profiles that use higher resolution textures. If a streaming clip contains no quality profiles with textures 1440 or lower then playback will be impossible and it will appear as though the clip is broken.
Please note this does not affect playback when streaming using the Unity player.
Large-Scale Clips Generating Bad Skeletons and SSDR Results¶
A “large-scaled volumetric clip” is now identified on clip creation via a warning popup. If you recieve this error the Generate Skeleton process will likely produce no visible result, and SSDR is likely to encounter precision errors resulting in bad visual noise during playback. As per the warning dialogue, you can correct this issue by right-clicking the clip, Reimporting it and adjusting the Import Scale Factor value until you do not receive the warning.
Compare View Mode Poorly Handles There Only Being Visible Data On One Stage¶
There are several display bugs tied to this case, however as you would typically be using Compare to review processed data with its source or an earlier stage, this is an outlier behaviour and you likely will not encounter it.