Disclaimer

Virtual Production is a complex topic. It is mandatory to understand the basic setup to be able to advance to the advanced techniques (for example set extension and multi-pos warp).
Furthermore, it is mandatory to understand all the basic principles of Pixera. 

Here is a list of all the VP related documentation in chronological order:

XR System Design

Unreal Engine (5.2.1) Setup

Unreal Engine Pixera Setup

Pixera Connect (Unreal Engine Plugin)

Studio Camera (Virtual Production)

Set Extension (Virtual Production)

Multi-Pos Warp  

Setting Up an Unreal Scene in Pixera

 In a proper VP setup, a high resolution led volume is used. As we are not able to simulate this effectively using a led volume on smaller demo/work setups, we are going to use generic displays.

For demonstration purposes we are going to use a simple setup. Import a display in the workspace:

In “Screen Groups” set the “Perspective Mode” to “3D”:

Import Unreal Scene to Pixera

Before the scene can be imported, the “Engine Editor Path” must be setup in the Pixera settings. Please add the path to the “UnrealEditor.exe” as shown in the screenshot.

Import the Unreal scene via the import button to your compositing resources:

After following the steps, the Unreal scene is accessible in the “Resource Browser”:

Now after selecting the Unreal scene in the “Resource Browser” some important settings are accessible in the inspector. Set the correct “Unreal Engine Target Version” and “Render API”. In upcoming releases, the default will be set to “5.2" and “DX12”. Please check this setting as it is important:

The “Launch” settings control the launch options. The default for a regular Unreal project is “Dash Game” mode. “Packaged” is used for Unreal exe setups and "Multiuser" is used for Unreal Multiuser Sessions. 

Add the resource to the timeline and select the resource in the browser afterwards as we must access the settings in the inspector.

Select the Unreal layer and in “Setup/Screen Group Presence” select all the screen groups you want the project to run on.

You should then see a red entry in the Unreal resource's "nDisplay Configuration" stating that the nDisplay config needs a refresh. Hit the “Create” button. Pixera will now recreate the nDisplay config files. 

Furthermore, activate the “Copy RxInterop.dll to Configure Nodes” button (this only has to be done once):

Now, click on run in order to start the Unreal Editor in the background with the selection you made. It will then open and show up your project.

The Unreal scene is now running. In the top right corner, select “Perspectives” from the drop down:

Select the “eye”. Now you can move the perspective.

Positioning Unreal Setup

With the “World Position”, the position of the Unreal scene inside “Inner compositing” can be moved.

The “World Rotation” always rotates the world through the root 0/0/0 position in the “Inner compositing” 3D space.

To execute the “World Rotation” from the changed “World Position”, use the “Relative Camera Controls” in the inspector:

Custom Command Line Arguments

In the resource inspector it is possible to set up “Custom Command Line Arguments”. The options you are given depend on if you are using a Pixera Server and image vs a personal laptop.
 

Laptop

Pixera Machine/Image

Command Line Arguement Options:

1. -noSplash: Show no Unreal splash window on opening.
2. -RenderOffScreen, Will render the project without a visible window.
3. disablescreenmessages: Will disable any messages from Blueprints.
4. stat fps: Will display a fps statistic of the Unreal Level.
5. stat unitgraph: Will display a graph of the Level’s asset usage.
6. r.PostProcessing.PropagateAlpha=2: Will allow layers to appear above and below the Unreal Level when on the timeline.
    

Clipping Plane

By default, the static clipping plane for Unreal setups is set to five hundred (meters):

By changing this setting, the render distance is going to be increased. As seen in the screenshot the sky is now rendered:

Output Assignment

For now, we only worked on the preview. As soon as we change or add an output configuration, we must kill the nDisplay setup and create a new nDisplay config:

By default, the playback on the local system is deactivated. The settings can be changed in the Unreal resource inspector in the compositing. tab. By activating local we can display the setup locally:

Resolution Factors

Often the Unreal scenes can be very demanding. Especially during preprogramming and planning, the setup might be tested on non-production hardware. Or it is not necessary to display the outer frustum in full resolution (example - Virtual Production with tracking. Outer Frustum only responsible for reflections therefore resolution is not important)

In the inspector, it is possible to scale down the render resolution.

1: Scaling factor for preview

2: Scaling factor for production overall (global resolution scaling for everything)

3: Outer frustum scaling factor

Adding a Second Display

Add a second display to the Workspace:

Kill the current Unreal setup in the inspector and create a new nDisplay config. The scene is now displayed on both screens:

Managing the Render Load

Now, both viewports are being rendered on the same graphics card. Depending on the Unreal scene this can lead to performance problems.

The screenshot below was taken on a Pixera 4 server with 2 RTX A6000 graphic cards. As you can see the Usage on the first GPU is at 71% load, the second is currently at 0%:

On multi-GPU systems, the load can be distributed to the second GPU in the settings. Select the resource and hit the “Edit” button. In here, graphic cards can be assigned to displays / led volumes (viewports). The first GPU Index is always zero. 

Viewport one is now rendered on the first GPU. Viewport two is rendered on the second GPU. The load has been distributed:

Curved LED Setup

By default, the content mapping onto a curved LED is flat. 
 

Change setting by selecting the LED (in screen group). Activate “Create Perspective Screen Texture Coordinates”.
Now pixels are mapped correct onto the curved LED wall from the eyepoint:

Overscan

If the eyepoint is below the physical boundary of the used screen you can utilize overscan to help. By using overscan, the perspective viewport size can be increased. Therefore, more content can be displayed without changing the setup. Without changing this option on some setups there will be a black rendering shown when moving the eyepoint out of the render viewport. 

Changing the overscan increases the viewport to catch more of the Unreal scene.

 

Multiclient Setup

Up until now, only single client setups were displayed. To distribute the setup to multiple clients please follow the following steps:
 

• Create multiclient setup. Refer to Director - Client Setup 
• Create nDisplay config as described earlier
• Distribute settings to additional clients:
  
  

Pixera 2.0.6 | 28. February 2024