Describes the feature set and how to work with the Virtual App
This App is used in conjunction with the Virtual Plugin to transmit and receive data over IP between the device and PC. The only user input required in the app is the IP address of the PC.
Remote Session App
Broadcast to the PC position and
rotation of camera or object
This technology allows you to turn Android devices into a virtual joystick which will be useful for designers and developers who create new game mechanics and work with models.
Game mechanic in Unreal Engine 4
The Virtual App enables a user to drive a Camera and Objects using an Android device in a virtual production environment. With ARCore the position and rotation of the Android devise is broadcast wirelessly to the PC.
This technology allows you to turn Android devices into a virtual camera, which will be useful for designers and developers who create new game mechanics and work with game cameras.
Virtual production in Unreal Engine 4
Virtual Plugin for Unreal Engine 4
Remote Session Plugin
Video striming from PC
Broadcast to the PC position and
rotation of camera or object
The plugin allows to use any objects in the scene, including all blueprints, cameras, etc. and move them in real-time in the game scene using the Virtual App, which transfers the movement and rotation data of an Android device in space.
How to start work with UE4 plugin
- Unreal Engine 4.23
- Windows 10, 64-bit
- GeForce GTX 1060
- Intel i5 7th Generation CPU
- 8 GB RAM
- ARCore-capable Android device for Virtual App
- Wi-Fi physical layer speed according to IEEE 802.11n > 200 Mbps
- GeForce GTX 1070 or above
- Intel i7 7th Generation CPU or above
- 16 GB RAM or above
- Wi-Fi physical layer speed according to IEEE 802.11n > 300 Mbps
1. Run your project, restart it if necessary. When the project has successfully started go to Edit-> Plugins-> Other and make sure the plugin is active.
If you added plugin from the marketplace, you should go to “Content Browser” and select “Show Engine Content”.
After that you should see the folder “Camera C ++ Classes” in “Content Browser”.
2. To find out your IP, press the key combination and type “cmd”.
Console will be open, you should write the command “ipconfig” in it.
3. Right-click on the “MyActor” object and select “Create Blueprint class based on MyActor”.
Select the directory in which you want to save the child object, specify a name and click “Create Blueprint Class”.
Open your new blueprint in “Class Default”, set IP and Port in the “Network” section. In the “Multiplication by number” section, set the value “500”. You can control the scale of the movement of the camera/object by increasing or decreasing the multiplier, the value of “1000” is 1 : 1 -it is suitable for movement, the value of “100” is suitable for working with rotations.
In the “Multiplication by number” section you can control your coordinates “Movement” and “Rotation”. Also you can turn on and turn off all coordinats if you need.
In the “ARPhone/Bode” section you can control “Camera Follow” – mark it if you want to work wih virtual camera and unmark if yoy want to work with virtual joystick, mark and unmark “Debug Message” and choose a camera to display, it can be “Cinematic Camera” (you should add it to the scene before choosing).
In the “Component”, select the “Mesh”.
Add a new “Component” in your Blueprint.
Add a new “Blueprint” to the scene. Now it will be working in the editor and compiled project on Windows and Android platforms. Also, you can add a couple of same “Blueprints” in your game, our plugin is support multi-devices.
4. Now you can click the “Play” button. In order to check if the connection has opened, press the key combination “Win + R” and enter “cmd”. Now, at the command prompt, type “netstat -aon” and see if your connection is open.
If the connection is open, you can start working and launching the application on your smartphone.
How to set video streaming
Prepare Your Unreal Engine Application
In this step, you’ll create a standalone executable file for your Project.
1. Open your Project in the Unreal Editor.
2. From the main menu in the Unreal Editor, select Edit > Plugins.
3. Under the Graphics category, find the Pixel Streaming Plugin and check its Enabled box.
4. Click Yes to confirm.
4. Click Restart Now to restart your Project and apply the change.
5. From the main menu, choose Edit > Editor Preferences…
6. Under the Level Editor > Play category, find the Additional Launch Parameters setting, and set its value to -AudioMixer.
7. Now it is set to work in Standalone Game, if you want to set it for work at compiled project follow the next steps. (look at “Start the Servers ” stage to check how to set server)
8. Package your Project for Windows. From the main menu in the Unreal Editor, choose Files > Package Project > Windows > Windows (64-bit).
9. Browse to the folder on your computer where you want the Unreal Editor to place the packaged version of your Project, and click Select Folder.
10. The Unreal Editor begins the packaging process.
11. When the packaging process is finished, go to the folder that you selected. You’ll find a folder called WindowsNoEditor with contents similar to the following:
12. Every time you start your packaged application, you need to pass it the same command-line flags set in step 6 above. One way to do this is to set up a shortcut:
Press Alt and drag your .exe file to create a new shortcut in the same folder (or anywhere else you like on your computer).
Right-click the shortcut and choose Properties from the contextual menu.
On the Shortcut tab of the Shortcut Properties window, append the text -AudioMixer -PixelStreamingIP=127.0.0.1 -PixelStreamingPort=8124 at the end of the Target field, and click OK.
You now have a packaged, standalone Unreal Engine application that has the Pixel Streaming Plugin enabled, ready to stream its rendered frames and audio.
Start the Servers
In this step, you’ll start the web services that will help to establish peer-to-peer connections between your Unreal Engine application and clients’ browsers.
2. In your Unreal Engine installation folder, find the location of the Signaling Server under Engine/Source/Programs/PixelStreaming/WebRTCProxy/bin/WebRTCProxy
3. In your Unreal Engine installation folder, find the location of the file “Package.json” and change “bcrypt 2.0.1” to “bcrypt 4.0.1”
4. In your Unreal Engine installation folder, find the location of the Signaling Server under Engine/Source/Programs/PixelStreaming/WebServers/SignallingWebServer
4. Start the Signaling Server by running the run.bat file. The first time you run the server, it will download all the dependencies it needs. When the server has started and is ready to accept connections, you’ll see the following lines in the console window:
When the Unreal Engine application connects to the Signaling and Web Server, you should see the following line of output in the console window opened by the Signaling and Web Server:
Streamer connected: 1
This means that you now have the Unreal Engine application running with the Pixel Streaming Plugin enabled, and the front-end Signaling and Web Server is ready to route connecting clients to the Unreal Engine application.
If necessary, you can stop and restart the Unreal Engine application and the Signaling and Web Server independently. As long as they’re both running at the same time, they should be able to reconnect automatically.
At this point, you have everything you need set up and working on your computer and mobile phone. All that’s left is to connect a browser to test video streaming.
1. Open your Virtual App and add your computer IP.
2. You will find streaming session, press play.
3. Streaming is ready to work.