Mo-Sys Support / VP Pro 5.4.0 / VP Pro Manual

VP Pro Manual -

Last updated: 

1. Prerequisites

1.1 Hardware

  • StarTracker or other Mo-Sys hardware with optional lens data input. StarTracker requires UDP support:
    • Minimum Firmware: v3105.
    • Recommended Firmware: v3788 or above.
  • PC with following requirements:
    • Minimum: Intel I7 with Nvidia GTX 1080 (RTX 2080TI for ray-tracing) or equivalent.
    • Recommended: Intel I7 with Nvidia RTX 4090 or above.
  • Blackmagic, AJA or Bluefish SDI capture card with at least one SDI input and one SDI output. Recommended Decklink 8K, Corvid 88, Kronos K8.
  • Camera with SDI output and sync (Genlock) input.
  • Sync Generator with multiple outputs (e.g. Mini Converter Sync Generator).
  • Optional:

1.2 Software

Note: If running at 25p, 50i or 50p it is recommended to set your graphics card display(s) to 50fps. (You can do so by right-clicking Desktop > Display Settings > Advanced Display Settings > Display adaptor properties for Display X > List All Modes).

2. Creating New Project

Create a new Blank project under Film > Video & Live Events. The Mo-Sys Plugin provides its own content, so the Starter Content is optional here.

Creating New Project

3. Installation

3.1 Option 1: Per Project

It is possible to drag and drop the plugin folders MoSysVPPro and Compositor into \<ProjectName>/Plugins (create a Plugins folder if it doesn't already exist).

Per project

This can be repeated for every project that requires our plugin.

3.2 Option 2: All Projects

To install MoSysVPPro plugin across all projects, you will need to drag and drop the MoSysVPPro and Compositor folders into the Plugins folder where Unreal Engine is installed. E.g C:\Program Files\Epic Games\UE_5.3\Engine\Plugins.

All Projects

Once placed in the Engine plugins' directory, you can open any project and enable the plugin before using it.

If you want to access the plugin's content in the Editor, go to Content Browser > All > Engine > Plugins > Mo-Sys VP Pro Content. This path will be different than if the plugin is installed per project.

Note: If you wish to remove our plugin from your project, refer to Chapter 17.

4. Usage

4.1 Login

After installing the plugin, the unreal project can be opened and a login window will pop up. Enter your credentials to login and it will restart Unreal.

Vp pro login

If the login was successful, you can load the Video Controller and StarTrackers panels in Unreal. These panels can be located under the Window tab or on the Toolbar under the Mo-Sys logo.

Status

The plugin also provides example layouts with many of these panels already loaded. These can be found in Window > Load Layout, under User Layouts.

Notes: - Opening the project after moving the plugin over may cause the loading sequence to stall when initializing (e.g., at 0%, 1%, 2%, etc). This is normal and can just be waited out as it should not take much longer to open. If it is taking an unreasonable amount of time to load, check Chapter 15 Troubleshooting for possible solutions.

  • If opening a project for the first time and the pop-up window bellow shows, click on Enable Missing. This will enable compositing in a new project. It may also require a restart.

Missing project settings

  • If you use Perforce, it is recommended you set the ...\Saved folder to be ignored as it can cause issues with the login otherwise. To do so, first create an ignore.txt file with \Saved inside and put it in your project folder. Then, type p4 set P4IGNORE=ignore.txt into the command console.

4.1.1 Symlinking Saved Folder

If you have a shared project on Network Attached Storage (NAS) and want to access it with a different license, it can be inconvenient to swap licenses by logging out and logging back in again.

In this case, it is possible to symbolically link (to create a symlink to) a local license to the project on the NAS. - Doing this requires some knowledge of the Windows Command Prompt. - You should be comfortable mounting drives in Windows. If not, this is explained in the VP Pro XR manual under Map Drives or Network Locations. - You need to know the IP address and path to the Unreal Project on the NAS. - If you switch machines, the symlink must point to the same place (exact same path) or you will need to delete it and make another. - You will need to log into the VP Pro Plugin in an Unreal Project first to generate the Saved folder and then make a copy of it to use as a local license.

For the NAS, we will: - Use drive letter Z: - Use IP address 192.168.134.2 - Use Unreal\Projects\TestProject for the project path

Note: Your NAS details are likely to be different.

For the Saved folder: - We will use C:\Users\Public\Documents\Saved as the Public user should be available on all machines. This path must be the same for all local machines.

  1. Map the NAS to a network drive letter (Z:) in Windows. This makes it easier to navigate the drive and will be used when you open Unreal from the NAS.
  2. Open a Command Prompt as Admin (Run as Administrator).
  3. In the command prompt, type: ###### net use Z: \\192.168.134.2\Unreal\Projects\TestProject
  4. You may be asked to overwrite the existing drive (Z:) if you mounted it previously, simply say Yes by typing Y and hitting enter.
  5. Change the directory, from the command prompt, to the newly mounted NAS. To do this, type: ###### cd /D Z:
  6. You should be able to see the contents of the Project folder by typing: ###### dir
  7. Now that you are in the project folder, you need to make a symlink to a folder on your local machine. You can do this by typing: ###### mklink /D "./Saved" "C:\Users\Public\Documents\Saved"
  8. This will create a folder in the Project folder called Saved (exactly the same as the folder you normally see in Unreal Projects).
  9. If you make a mistake when typing the command or want to remove the symlink, you can simply delete it using: ###### rmdir Saved
  10. You should be able to navigate to the symlink and access it, like any other folder.
  11. If, however, you see a message say: ###### The symbolic link cannot be followed because its type is disabled.
  12. You need to enable symbolic link permissions. You can do this by typing: ###### fsutil behavior set SymLinkEvaluation R2L:1
  13. Another message should appear saying: ###### Remote-to-local symbolic link evaluation is: ENABLED
  14. You can now access the symlink normally.

Note: changing the SymLinkEvaluation state only affects the machine you are currently using, not the permissions of the NAS, so if you switch machines, you will need to do this again. This change should persist if the machine is turned off and on again though.

4.2 Modes of Operation

Mode, frame rate and other settings are available in Edit > Project Settings > Mo- Sys VP > Mo-Sys VP (should be at the very bottom).

Vp settings

Changing settings in here will likely require a restart. This includes changing modes, frame rate, Io standard, etc. Unreal will tell you to restart after changing.

Settings relating to VP should be changed in the Mo-Sys Project settings to ensure they are changed everywhere. Changing them individually throughout the editor can cause inconsistencies, such as mismatching frame rates.

There are various modes to choose from, depending on what you want to use the plugin for. Some modes are available regardless of your license tier. However, some modes are only accessible with specific license tiers.

License Tiers: VP (Free) < Live < VFX < CCS < Studio < XR

Modes dropdown

Modes (required License tier):
- VFX (VP)
- AR (VP)
- Pre-Keyer (VFX)
- Post-Keyer (VFX)
- Studio Program (Studio)
- Studio Preview (Studio)
- Recorder (VFX)
- XR (XR)

4.3 Open Example Map

Open the L_MoSysAR_Example map from Plugins > Mo-Sys VP Pro Content > StarterContent > Maps.

If you don't see the MoSysVPPro Content folder, go to the top right of the Content Browser, click Settings > Show Plugin Content.

Plugins

The main difference between this example map and a default map is the presence of a MoSys Camera, controlled via a MoSys Stage. This allows for Video cards to transfer input and output data through the Video Controller panel.

Video Controller

4.4 Adding Live Link

To add a source, open the Live Link window (Window > Virtual Production > Live Link). To add a Mo-Sys UDP Source for example, select + Source > Mo-Sys Hardware Source > UDP Mode > Ok.

Click on added source e.g. Mo-Sys UDP Source, find the Mo-sys Live Link settings, select StarTracker or Encoder and edit the subject name e.g. ACam. Then click Add New Subject. On completion, ensure that under Tracking it shows Network Interface IP Address with the IP address of the PC you are currently working on. If not, this can be edited to your PC's IP address (e.g. 192.168.134.67). This should not be a StarTracker's or Encoder's IP.

Now, a new subject should appear under Subject Name. If you have multiple network interfaces, such as multiple PCs, enter the correct IP for the chosen hardware's communications here.

Adding Livelink

Click on the new subject e.g ACam, edit its port number, if you are receiving data, the colour of the subject status should turn green. If not, continue to the next section to configure the StarTracker.

Livelink

Note: If there are multiple MoSys Cameras in the scene, or if the current camera is not configured properly, you may need to set the Camera's Live Link subject to the newly created one.

This is done by clicking on the MoSysCamera in the Outliner panel, finding the MoSysLiveLink component in the components list and setting the Subject Representation. This should be a dropdown which can be set to the Live Link subject you want to control that camera with.

4.5 Setting Up an Encoder in Unreal

If you are using an Encoder system, it is important to do a couple of steps before moving on with the Unreal side:

  1. Make sure it is online and streaming data first. Keep its initial output as close to zero as possible.
  2. Calculate the scale to be used in Unreal - you can do this by measuring how many ticks ($t$) it takes for your encoder to travel 100cm and then divide by that result (e.g. $100cm = 200 t$, so $scale = 0.5$ for a linear option); or how many ticks to complete a full turn (e.g. encoder is on a rotating platform, $360^o = 1800t$, so $scale = 0.2$).

Now, in your Unreal map, add an Actor to your scene that represents your stage. Move that stage to the desired position. Then, add a child Actor to that stage which represents the Encoder. In this example, we've added a platform to the stage.

EncoderActor

Add the MoSysLiveLinkComponentController to its components through the Add button.

EncoderLiveLinkComp

In its settings, make your LiveLink (e.g. ACam) the Subject Representation.

EncoderLiveLinkCompCam

You can now add a MoSysEncoderComponent in the same way as the previous component, through the Add button.

EncoderComp

It this component's settings, tick Editor Live Tracking to see your changes in Editor without the need to be in Play mode. Set which Mode you would like your encoder to use (Linear, for travelling on a continuous line or Angular, if encoder is rotating itself or on a rotating platform). Finally, choose in which axis the encoder will be moving and add the scale previously calculated.

EncoderCompStats

When you move your encoder, you should now see the actor moving accordingly.

Note: There is an example map you can switch to for our implementation: L_MoSys_Encoder_Example. It is set for two encoders but you can test both objects with one - by simply removing the used live link from the Subject Representation of one of them to the other. There is a LiveLink preset that you can use as well, called EncoderLiveLinkPreset.

4.6 Checking StarTracker in Unreal

If the IP address and port are set on the UDP source and the status light is not green, you may need to connect and auto-configure the StarTracker in the StarTrackers panel.

StarTracker Livelink

If the StarTracker does not appear automatically, it can be added via the IP Address input box followed by selecting Add StarTracker. This should add it to the StarTrackers list when confirmed it is connectable and available.

StarTracker IP

Once you see the StarTracker, it can be linked to the newly created subject (e.g. ACam) by using the Live Link dropdown.

StarTracker Cam

It is possible to save the MoSys Live Link preset so that it stays connected after closing and reopening Unreal.

If the status light does is still not green you can auto-configure the StarTracker. This is done by clicking Auto-configure in the StarTrackers panel, next to the StarTracker you added the ACam Live Link subject to. This will ask if you would like to confirm the configuration process, click Confirm to proceed.

StarTracker Auto-configure StarTracker Auto-configure Confirm

The status panel should also update now.

Update Status

Note:

  • The StarTracker's status can also be seen as Optical Good on the far right of the panel. Problems with the StarTracker such as lack of stars, internal errors and overall visibility will be reflected here for easier debugging.

  • You can also use TightVNC or the VNC button next to the StarTracker to manually configure it if auto-configure does not work. Please see the StarTracker user manual for details, under Network and Data.

4.7 Enable Video In

If the video assets are not created when you restart your project in AR mode, the video controller panel can be used to generate them by clicking on Recreate. You may need to restart your project. When configuring the Video Controller, it may also require a refresh to update changes, click Refresh to do this every now and then.

There should be a viewport named Output with the Mo-Sys logo. If this is not present, open it through Window > Viewports > Output.

Startracker Viewports

Once the viewport is open, you can change its type to Mo-Sys Output via Perspective.

Startracker perspective

You may need to configure the video card input and outputs in case either: - AR graphics and live composite are not seen in Unreal and on the SDI output. - the status panel reports Input video is not playing with a red light in the Video option.

If you use Blackmagic's videohub and video cards, ensure the hub is on and the switcher is sending camera output to a video card on your PC. For example, if the camera was sending to the 3rd (input) connector and the output of your PC is the 7th connector then, in the Video Controller panel, you would set the input to the 3rd one. This is done through Configure > Configuration dropdown, select (in this case, DeckLink Quad) the 3rd option then Apply.

You would also do this for the output.

Startracker input

It is also possible to not see the output of the camera in the thumbnail of the input and/or see 1080p25 in red with a black thumbnail. Make sure all correct options are selected and Refresh the video controller as previously mentioned. The status panel should show all 4 lights as green now.

Note: When the video input/output are changed, the Sync option may turn red again. This usually returns to green after a short period of time.

4.7.1 Enable Video In

The main viewport and the Output viewport should now show the output of the camera. Moving around in the Perspective viewport alters the textures and reflect the output of the camera.

Note: In case there isn't anything in the Output viewport, please make sure the MoSysCamera orientated to view the Composite meshes in the scene.

Enable video in

4.8 VFX Mode and MoSys Keyer

Ensure you switch to VFX mode in the project settings before continuing with this section (see 4.2 Modes of Operation for switching modes). Now open the Keyer via the Mo-Sys Panel.

Note: Panel may look different depending on the license tier applied.

Open keyer

If you've purchased a VFX licence, you can access MoSys Keyer in the keyer widget, available from the M panel at the top of the screen. If you're using the VP license, a Keyer will still be available, however it will be Unreal's internal keyer, with limited functionality.

Once opened you should see the output of the camera. If not, see 4.7 Enable Video In.

Keyer

The keyer will allow you to choose up to 5 different variations of the green background in order to get the best green screen keying. This is done by clicking anywhere on the camera output and setting the greens one by one. Other keying settings can be found alongside other colors to customise the key.

If you want to reset the key colors, click on Clear Colors on the top-right of the panel and re-click the 5 colors. You should immediately see the result in the Output viewport.

Keyer output

Note: In the event there is no output from the keyer, here are some things to check: - You are using a composite mesh in the scene. - The Is CompositeKeyer enabled. option under Composite Keyer is set to true in the Keyer Panel. - Render soft mask is off in the composite mesh.

4.9 Post Keyer Mode

This mode requires an external keyer (e.g., Blackmagic Ultimatte) in order to get the fill and key signal. The compositing is done in Unreal.

Switch to Post-Keyer mode in the project settings before continuing. See 4.2 Modes of Operation for details. For best results, switch to the L_MoSysPostKeyer_Example map as it is pre-configured to show the Fill & Key composite.

This is configured in the same way as section Enable Video In. However, this mode requires physically rewiring the video card to receive input and send output. Check the video card's manual for details.

This example uses the DeckLink 8K Pro card. Desktop Video Setup will be used to soft-patch the card's SDI connectors. It also shows the input/output resolutions and framerate for the connections used in Unreal.

Ensure that each SDI connector is set to a single In or Out. For example, this card has 4 connectors so each needs to read SDI \<number> In or Out.

Note: The connector number may not match the connection name. This is fine as long as there is 1 connection per connector.

Decklink

Now route the output Fill signal into the first connector and the output Key signal into the second. Finally, route the output of the PC into the fourth connector. This can be verified by using the video card's provided software, such as Blackmagic Media Express, and checking each device is outputting correctly. Close this software (or similar) before using Unreal as only 1 program can access the SDI input/output connectors at once.

Open Unreal and open a map with a composite mesh (e.g., the PostKeyer example map). Then open the Video Controller panel and set the first input to the Fill signal (e.g., DeckLink 8K Pro (1)) and set the second input to the Key signal (e.g., DeckLink 8K Pro (2)).

Note: If the framerate is incorrect, you can set the Auto option to off and change the framerate, amongst other settings, for each connection. Decklink setup

The output should be set to the PC output signal (e.g., DeckLink 8K Pro (4)). In the end, the Video Controller Panel should look like this.

post keyer mode

Note: At this point:

  • The resolutions, framerates and project settings need to match (e.g., 1080p24). The Video status light shoud now be green.

  • If one of the Input thumbnails does not update after setting the correct device, saving and restarting Unreal can fix this.

  • You should be able to see compositing in the MoSys Output viewport. If not, ensure you have a MoSys Composite Mesh in the scene and then rescale it to try and capture the newly composited video signal. See 7. MoSys Composite Mesh for details.

Here is the Output window without a composite mesh:

Without a composite mesh

And with a composite mesh:

With a composite mesh

4.10 Pre Keyer Mode

Switch to Pre-Keyer mode in the project settings before continuing. (Details in 4.2 Modes of Operation). The DeckLink 8K Pro is still being used for this example.

For this mode, compositing is done in an external keyer, where Unreal outputs the Fill and Key signals. First, you need to rewire the video card so that SDI 1 and 2 are In and SDI 3 and 4 are Out. This configuration may only appear on the first video card connector. E.g.: in Desktop Video Setup.

Decklink 8K

After, open Unreal (as mentioned, with any card controlling software closed) and open the L_MoSysPreKeyer_Example map. Open the MoSys Video Controller (if not already open) and configure the Output, once more. It should only show 1 device as available, since all 4 connectors are being controlled by the first main connector in this configuration. Choose Fill and Key. Ensure the resolution and framerate match the camera and video card. Then click Apply.

Decklink configure

Finally, click Refresh if the outputs do not automatically change. You should now see the first output as the normal image (Fill) and the second output with black and white signals (Key).

Decklink fill and key

To add actors to the external key (for example foreground objects or other mattes) you need to set the actor's Custom Stencil value to 247. Note this does not work with Nanite meshes and all materials must have Write Custom Stencil selected. You can verify this by previewing the Custom Stencil buffer in the Viewport.

Example output:

Example output

Example output 2

Note: If Media Bundle is being used in the scene - for a video wall for example - re-parent the material on the Plane mesh to M_DefaultMedia_MoSys. This prevents it being garbage-matted out.

4.11 Compositing Controls

It is possible to change the Compositing performed by the camera. To do so, you can do this by selecting the MoSysCamera in the Outliner, finding the VP tab in the details panel and then Compositing.

Compositing controls

From here you can change whether to pull a key, enable shadows or enable reflections. Alongside these options are dials for changing the quality and intensity.

5. HyperDeck Disk Recorders

To use the recorders, go to the Project Settings > On Set VFX > VFX. These will be needed to use the Take Recorder Panel (seen in Chapter 6. Take Recorder).

On set VFX

Now add the total number of recorders first to Number Of Recorders, followed by their individual IP addresses as array entries in Recorder Ips.

On set VFX recorder

After, open the External Recorder Monitor. If you have added IP addresses to the previous array these should show up in the individual recorder panels. If not, you can add them here directly.

External Recorder Monitor

External Recorder Monitor Result

Press each Connect button to connect the recorders. On successful connection you should see a message in the panel stating the recorder type and the port. You should also see the state of the recorder (READY) the the Record and Stop buttons.

External Recorders Ready

You can now record from this panel by pressing the record buttons. The panels will show you if they are recording and allow you to press stop in that case.

External Recorders Recording

Note: Before you start recording, make sure you have a hard drive installed in the recorder, with the correct format, enough space and check that you have SDI input signal.

6. Take Recorder

To record and export camera data, use the Take Recorder. Make sure you have connected all recorders through Chapter 5's instructions and do not close the External Recorder Monitor. In order to use, open take recorder:

Open take recorder

If the Take Recorder does not contain any cameras, it may need to be reset. This is done by clicking Reset Take Recorder or the M logo in the Take Recorder. After this, you should see the MoSysCamera placed in the current scene.

Take recorder camera

Additional settings for the take recorder can be found when clicking the Settings icon, then see under Mo-Sys Recorder category.

Take recorder settings

Take recorder settings 2

Individual camera settings can also be accessed by clicking on MoSysCamera in Take Recorder.

Take recorder camera settings

To start/stop recording you can click on the record button in this panel. This will trigger all hardware recorders.

Take recorder record

Starting a recording will open the Sequencer and show the progression of the current take.

Take recorder progression

Stopping a recording will create a Take and place it in a folder named after the current date (YYYY-MM-DD) under Takes folder in the Content Browser. This new take will be named using the time-stamp of the start of the recording.

Take recorder folder

6.1 Exporting Take Data

The Take Log panel can be found in the same place as Open Take Recorder.

open take log

Open it and click on export for a specific take, or export all for all takes. This will export your data as fbx in addition to a text file containing the metadata to the file C: > UnrealTakes on your machine. Take log

7. MoSys Composite Mesh

MoSys Composite Mesh allows you to composite with shadows and reflections. This would be a 3D object in your scene which can be placed/shaped to exactly where you want video to be. There is the possibility of having CG objects occluding it or being in front of other objects.

To use it, click on Quickly add to the project. in the toolbar menu, then Mo-Sys > Mo Sys Composite Mesh, and drag and drop it to the scene.

Composite mesh

To enable shadows, check the box Cast Shadow in the MoSys Composite mesh details.

Cast shadow

8. Arri, Sony and Red Camera Control

If your production is using a camera that supports CAP (Camera Access Protocol), You can communicate with it directly over Ethernet or Wi-Fi and receive live metadata in the Camera Monitor Panel.

Open the panel from the Window menu or using Mo-Sys Toolbar menu. Connect the camera by providing the camera's IP address and password. The password can be found in the camera's menu (the default for Arri cameras is 'arri').

Open camera monitor

Camera monitor

Once connected, you will see the Arri, Sony or Red interface. This will update live with the camera parameters. If you are using the Take Log, the camera metadata will also be captured at the start of the take and added to the text file that is exported from the Take Log.

Camera monitor full

9. Erizos Integration

This integration displays Erizos widgets on the video output. To enable this, you'll need to install Erizos software in addition to Erizos Unreal plugin.

The image below shows how to set up Erizos widgets with VP Pro. To Event BeginPlay add the Erizos widgets, to Event Tick draw the widgets, and to Event End Play remove all widgets.

It is possible to also control the size and the position where you want to draw the widget.

Erizos blueprint

Once you set the above blueprint you can see the Erizos widgets on the Mo-Sys Fill viewport and on the Video output.

Erizos screen

10. Custom Stencils

Objects may require a custom depth for Pre-Keyer mode. The standard sampling value is 247. This will enable objects to be drawn as white instead of black when using the Pre-Keyer's Fill and Key signals.

To enable an object to appear as white, you must enable Render CustomDepth Pass in the Rendering section of the Details Panel of that object. Then set the CustomDepth Stencil Value to 247.

Note: This can also be achieved by searching for CustomDepth in the Details Panel search box.

Custom depth

After this, you can see the results of the render pass by changing the viewport to show it. Go to Lit.

Lit

Then Buffer Visualization and, finally, Custom Stencil.

Custom stencil

It will change the viewport to show all objects with a custom depth over 0 (as 0 corresponds to black). This is a debugged version of the render pass. The actual results will be shown in the Pre-Keyer mode as white objects.

Pre keyer mode

11. NearTime Rendering

NearTime is a cloud based system to enable automatic re-rendering of recorded takes at higher quality than in real time. When NearTime rendering is enabled, recorded camera data is automatically uploaded to the cloud where the take can be re-rendered across multiple render nodes at higher resolution and quality. This is then delivered into your content management system without manual intervention. As the output produced is timecode locked, shortly after a recording completes a high resolution version of the scene is available for use in your NLE. It will also seamlessly synchronise with the related video plates.

NearTime portal

You can access the list of jobs for a project via Unreal directly. This can be done through the NearTime Panel. Make sure you have CSS enabled. This can be done in the Project Settings, by going to Edit > Project Settings > Mo-Sys VP > CCS NearTime. Under the CCS dropdown ensure CCS is enabled and you have a valid NearTime address.

CCS

After this, you can access the NearTime panel via the M logo.

This panel should automatically connect to the NearTime service and show you the list of jobs and their current status.

Open NearTime

NearTime

NearTime workflow

Once you're connected, you can also submit a Take to NearTime from your Content Browser. To do so, find your Level Sequence in the Content Browser, right-click it and select Submit Take to NearTime.

NearTime Submit

For more information, refer to our NearTime Manual.

12. Multi Engine

The Mo-Sys VP Pro Plugin also includes the ability to control multiple Unreal Engine instances via Multi-Engine. This is available from the Live tab in Project settings under Edit > Project Settings > Mo-Sys VP > Live.

Enabling Multi Engine Master allows a machine to act as the controller (or master) of several other machines. This provides it the ability to broadcast data - such as numbers, characters, bytes, and even actor transforms - across the network to other instances.

All other instances should be listed in the Other Multi Engine Ips array.

This will require a restart of the Engine to take effect.

Other multiple engine IPs

It's important to note that all Unreal Engines (in the Multi-Engines list) must have the Mo-Sys VP Pro Plugin in order to send and receive data.

Broadcasting data can be done via Blueprints. This requires the master to send data and all other engines to receive this broadcast via triggering the appropriate events.

The types of data that can be sent are:

  1. Play / Stop
  2. Start Record / Stop Record
  3. Select Camera / Select Camera XR
  4. Select Level
  5. Trigger Animation
  6. Single byte
  7. Single signed 32 bit integer
  8. Custom 8 bytes (or single unsigned 64 bit integer)
  9. Transform (Position, Rotation, Scale)

To access any broadcasting, and type conversions for broadcasting, you must use the Mo-Sys Multi Engine Subsystem in your Blueprint.

For example, if you wanted to broadcast some data every time the spacebar was interacted with, you could do something like this:

Spacebar blueprint

If you then wanted to receive this data, during play, you might implement something like this:

During play

13. Colour-Correct Compositing

Colour-correct compositing requires a number of settings to be changed, and is not usually required or desired since assets have been designed with the Unreal tone-mapper in mind. However, for test purposes it can be useful to review a colour-correct composition.

The following is prepared in the level StarterContent > Maps > L_MoSysColourCorrectComp_Example

The Mo-Sys Camera has the M_NoTonemapper material applied to the CineCamera component.

In the Compositor post-process material M_Compositor_AfterTonemapping found in Compositor > Materials > PostProcess, hook the colour-correct pin to the Emissive output of the material.

Colour-correct compositing

14. OpenColorIO (OCIO)

Our plugin allows users to Apply OCIO through our Video Controller Panel. To do so, users should first setup their OCIO Configuration Asset. The simplest way to do this is by following Unreal's Quick Start Guide on the subject.

In the following example, the Configuration File is the default provided by Unreal. This can be changed to whichever is needed, however, make sure there are only two Color Spaces in the array - one for source and one for destination.

OCIO Settings Example

Now, this new asset needs to be added to Project Settings > Mo-Sys VP > Color > OCIO Color Configuration through the drop-down menu.

OCIO Project Settings.

It should now be possible to Apply OCIO or Remove OCIO under the Video Controller Panel.

OCIO Apply

OCIO Remove

15. Switchboard

15.1 Installation

The Switchboard plugin is in Beta and there are two ways to install it. The best way is by using the provided installer which attempts to install the plugin into the default Engine path. However, if this is unsuccessful, you will need to install it manually.

Automatic

Go to the switchboard plugin folder provided in the MoSysVP.zip file. Then, go to the setup folder and run setup.exe.

Switchboard Setup folder

Switchboard Installer

To verify this, go to your Engine path and check the switchboard plugin is there and the mosys folder.

(For example, if you installed the Engine using the default C drive path, you would look in: C:\Program Files\Epic Games\UE_5.2\Engine\Plugins\VirtualProduction\Switchboard\Source\Switchboard\switchboard\devices)

Switchboard devices

Manual

For this option you will need to copy-paste the mosys folder yourself.

Switchboard MoSys folder

Make sure the folder is placed in the devices folder of the Switchboard plugin. For example: C:\Program Files\Epic Games\UE_5.2\Engine\Plugins\VirtualProduction\Switchboard\Source\Switchboard\switchboard\devices .

So that it looks the same as if you had automatically installed it.

15.2 Running

Launch the Switchboard batch file. This is found where you installed the Engine. (In previous example: C:\Program Files\Epic Games\UE_5.2\Engine\Plugins\VirtualProduction\Switchboard\Source\Switchboard\switchboard.bat).

Switchboard runner

It is recommended to create a desktop shortcut to this file so you do not need to continuously navigate to the Engine's plugin folder.

Once launched, you should see the Welcome to the Mo-Sys Switchboard Plugin message as well as 2 additional tabs on the right-hand side called Mo-Sys Devices and Mo-Sys File Sync.

Switchboard opened

If there are no error messages in the Switchboard log, at the bottom right related to Mo-Sys, the plugin is correctly installed.

15.3 Adding Devices

At the top-left you should see an Add Device dropdown. Clicking this should show a list of possible device types to add.

Switchboard add device

To add a StarTracker, choose the MoSys option (has the Mo-Sys logo next to it).

Switchboard MoSys device

This opens a Search for MoSys Devices dialog box and starts searching for the StarTracker on the network.

Switchboard waiting

Please ensure the StarTracker is on and the StarTracker software is open (you should be able to see your stars/maps). Two attempts are made to find the StarTracker. First, a query for the StarTracker service and then second, a DNS lookup.

Note: Discovery methods do not work over VPN connections. Please add the device manually.

To speed up searches, you can disable the Use Legacy DNS Method which means the Switchboard will only look for StarTrackers using the first method. After waiting for discovery to complete, you should see the device listed in the dialog table.

Switchboard successful discovery

If not, press Refresh in order to look for the device again or use the + button to manually add the device.

Switchboard manual add

Switchboard manual box

With the device listed, you can change its name by double-clicking the device's name (listed in the Device Name column) and then type your own name to identify it. Some devices may have been added that you do not want to use at this time. To remove those, press the device's include checkbox (listed in the Include column) so that it reads No.

If you want to discard the search completely, press the Cancel button. Otherwise, press the Add button and any devices listed as Yes in the Include column will be added to the Devices section of Switchboard.

Note: if you lose your internet connection during discovery, the Search box may not work properly (e.g. discovery finishes but the buttons do not return). You will need to close and re-open it once the internet connection is back and it should continue as normal. If not, close Switchboard completely and try again.

15.4 Devices Section

Devices added previously will appear on the left-hand side under MoSys Devices when launching Switchboard.

A MoSys device's icon (left of the device row) will appear in the colour of the current status to allow for quick checking. Blank if disconnected.

Switchboard devices list

To connect to the device, press the Connect button on the right of the device row. This will make it greyed out whilst Switchboard attempts to connect to the device.

Switchboard devices connect

If successful, Switchboard will log Connected to \<device IP address> and the device's icon will change to match the current status. If not successful, Switchboard will log that it was unable to get the device's information or update it. This is usually due to a connection error, such as the device not being turned on or reachable.

Devices are also listed on the right-hand side under the Mo-Sys Devices panel. This will show each device in a table and report more details about them such as their device type and current status.

[Before connecting to the device] Switchboard devices panel

[After connecting to the device]

Switchboard devices panel

Devices in this panel can also be inspected (the device must be connected to achieve this). StarTracker devices will show a Device Node reporting the current UDP connections of the StarTracker. To inspect, click anywhere on the device's row. Best place to click is the device's current status in the Status column.

Switchboard inspection node

To stop inspecting, click anywhere on the device's row again or click another device to inspect that one instead.

15.5 StarTracker Device Node

The StarTracker device node can be used to update the UDP connections without manually accessing it through VNC or be physically connected to it. These UDP connections represent where StarTracker will send their data to. The Device Node reports the StarTracker's name and IP address. The colour of the IP address is the colour of the current status. It also shows 3 UDP channels.

Each channel has:

  1. An Active/Inactive button - to enable/disable whether the StarTracker should be sending data to this UDP channel.
  2. IP address input field - the IP address to send the data to.
  3. Port input field - the Port to send the data to.
  4. Lock/unlock button - whether you can edit the current UDP connection or not.

To make a change to one of the channels, press the Unlock button to remove the greyed out status. Edit any of the fields in the channel and press the Lock button to save changes. This will send the changes to the StarTracker and update its channels.

15.6 Mo-Sys File Sync Panel

This panel is used to manage Unreal projects' distribution across an array of Unreal/nDisplay nodes. It is required to map all the nodes as network drives (refer to Share and map drives section in VP Pro XR manual), as well as install Free File Sync - a thrid party file synchronisation software. Free File Sync should be installed in a default location: C: > Program Files > Unreal. The projects on all devices should have the same directory path.

Switchboard file sync

The panel lists all nDisplay and Unreal devices found in the considered Switchboard configuration, so it will require Switchboard restart or config reload after adding new devices. Users can view the device type, name, IP address and VP role. Sync column specifies which devices should be updated. User can tick them between yes or no. First, select the folder which is shared from other nodes in Browse 'Share' directory and then click Sync Files. The system will sychronise the devices depending on the dropdown option. Special caution has to be taken, as the files at the destination project locations are being replaced with the local version of the project. The source is always the local Unreal project.

Sync options:

  1. Content - most common, only synchronises content folder.
  2. Plugins - synchronises Plugins folder in the project.
  3. Config - synchronises Config folder with the exception of DefaultGame.ini.
  4. Everything - synchronises everything, replacing all files at the destination project.

Open Sync UI button opens the Free File Sync UI interface, which can be handy in special cases, or if in doubt what files are being overriden. Click compare to see the file to be synchronised.

The panel creates sync files that are stored in local Unreal project in [...]\Saved\SaveGames\FFS.

Map device folder button automatically maps the shared folder on the local device to the devices listed and checked for sync. The network drive will have the path of \\IP_address\share_folder_name.

16. VP Pro for Artists

If you're an artist using VP Pro there are some things that need to be taken into consideration: - It is important to use our VP Pro sample maps first for testing. - It is also important to use a MoSysCamera rather than Unreal's CineCamera.

17. Removing VP Pro Plugin

If you want to remove our plugin from your Unreal project we recommend you disable it through the editor. To do so, go to Edit > Plugins first.

Open Plugins

Then, search for Mo-Sys VP Pro and Compositor plugins and disable them. The editor will need to be restarted for this to take effect. Only press Restart Now when both are disabled. Open Plugins Open Plugins

All Mo-Sys related components, including UI and scene objects should now disappear from your editor and scene. For example, a project might have looked like this previously:

Open Plugins

And now more like this:

Open Plugins

Another way to remove the plugin is to go to the Plugins folder in your project, and eliminate both Compositor and MosysVPPro folders.

Project Folder Plugins

Once this is done, you can open the project and a pop-up should show warning you about the missing plugin:

Project Warning.

Just press the Yes button and this should disable part of the plugin. We do not recommend this way of disabling it as it often does not disable the Compositor plugin. As a result, when you open your project something like this can happen:

Compositor Still Enabled

For this to disappear, you can either disable the Compositor plugin as mentioned above or you can click Settings in the top right corner, select World Settings and untick Is World Composite Enabled under Composite World Data.

18. Troubleshooting

Source Control

Perforce

If you have (or had) a project under Perforce, you may find that the projects' settings get stuck and attempting to change them does not update them or Unreal.

This is common with Rendering settings as changing the way Unreal performs rendering and graphical updates typically requires the compiling (or recompiling) of shaders.

Rendering Settings

The most common issue is found with Unreal's Allow Alpha Through ToneMapper setting which, when changed, requires hundreds of shaders to be compiled, taking a very long time. In addition, the option may appear ticked but not show the desired results (compositing) or vice versa.

This is solved by deleting the temporary folders of an Unreal project. These are:

  1. Intermediate.
  2. DerivedDataCache.

Deleting both of these, completely removes the compiled shaders (and will require a recompile). However, doing so should override Perforce from protecting the files from being overwritten, whether it has changed their Read-Only state, their checked-out state or their locked state.

Note that both folders do not need checking into Perforce, as they are created and compiled upon first launch of the project. Although it's useful to keep them in your Perforce project for a quicker startup, it clashes with Perforce.

Configuration Settings

If the issue is not related to Rendering settings, but instead stops you from making changes to a configuration file, such as an .ini file in the Saved folder, you will need to make sure the file is checked out in Perfoce and is not exclusively locked by another user.

This is because Unreal does not reliably checkout files under Perforce and attempts to write to a file that is protected. You can check if this is the case by going to Perforce and checking out the file yourself.

Please make sure you have read Epic Games' guide and recommendations when using Perforce here.

AJA

To confirm video input and format, use the provider's tools (Blackmagic Media Express or AJA Control Panel). Here is an example of AJA Control Panel configuration:

AJA control panel

Genlock

If you find that you keep having to adjust the delay, make sure that both the camera and the StarTracker are genlocked correctly.

Tracking Data

To confirm tracking data, look in the Live Link window for the status of your source (e.g Optical Good). If you don't see data (Status is Waiting) this is most commonly a networking issue. Check StarTracker's network settings, confirm you can ping StarTracker and vice versa and disable firewalls. For more information on errors, open the Output Log. You can filter to show Errors, or filter by LogMoSysVirtualProduction or LogMoSysTracking to see plugin-specific issues.

Video

Some camera and video card issues occur when the resolution and framerate do not match the Unreal project settings.

If you lost video in the Editor after you opened the Keyer panel, go to Plugins > Mo-Sys VP Pro Content > MoSysNewKeyer folder, open M_MoSysNewKeyer material. If you see an error in the material, click on the texture Compositor_MediaInputTexture then change the Sampler Type from linear colour to colour, and then from colour to linear colour, then save your material.

Recorders

When opening a project where you have connected to recorders before, it can at times say they are already connected when connection is attempted. It is possible for Unreal not to shutdown completely and have background processes still active. If that is the case, shutdown Unreal and verify there are no background activities in the Task Manager related to Epic. You can now reopen the project and recorders should be available to connect to again.

Project versions and Temporary folders

Upgrading projects can cause bugs when launching, such as projects hanging or the Unreal engine opening in the background but never the foreground. If this occurs, it is likely because of old cached save files and configuration files. This can be solved by deleting the Saved and Config folders in the Unreal Project.

Unreal folder

These folders are recreated when re-opening the project again and should contain the updated project information.

Deleting these folders may also solve issues with logging in and token failures within Unreal when accessing any of the Mo-Sys tools.

Bonjour / Avahi

NOTE: Bonjour and Avahi are used interchangably as they refer to the same thing, only the OS differs. 1. Bonjour -> Windows 2. Avahi -> Linux

As stated in the prerequisites, you need to install Bonjour for discovery to work. However, after installing Bonjour, if you still find you are having issues finding StarTrackers, you can check a couple of things:

  1. You have Bonjour installed and it shows up in the Add or Remove Programs section of your OS.
  2. You can query the Bonjour service and report its version. Running the command dns-sd -V on Windows should return a version number. On Linux, you will need to check the status of the Avahi daemon by running the command sudo systemctl status avahi-daemon and it should return that the daemon is active. If it does not and returns error -65563, this means Bonjour is installed but not enabled as a service. On Windows, this can be solved by going to services.msc and enabling it and on Linux, this is solved by starting the service via sudo systemctl enable --now avahi-daemon. You may need to restart the computer after this and recheck that it is now enabled. If you get a different error code, please submit a support ticket.
  3. You are on the same network as the StarTracker. Bonjour can work over a VPN but requires a network bridge to be setup to work.
  4. you have a stable internet connection, to and from the StarTracker. This can be confirmed by pinging the StarTracker and making sure you get a fairly quick response. A weak connection can result in tracking data loss and/or disconnecting from the StarTracker completely.