Virtual Camera / Controller · Free on Fab

CAMERA IN HAND.
UNREAL IN CONTROL.

Turn an Android device into a virtual camera, joystick or custom controller for Unreal Engine. Build real-time interactions in the editor and in packaged Android and PC projects.

Virtual Camera illustrated on a landscape phone
Virtual App + Virtual Plugin

Two components. A live scene.

Use an ARCore-capable Android device with the app and the Unreal Engine plugin. Requirements, connection steps and controls are covered in the documentation.

Device → Unreal Engine

Position, rotation & input

Virtual App sends the Android device’s motion to the PC. Virtual Plugin uses that input to drive cameras, objects and Blueprints in the scene.

Unreal Engine → Device

Scene video streaming

The plugin streams the PC scene view to the device, connecting the physical controller with the real-time result.

Virtual Camera & joystick

This technology allows you to turn Android devices into a virtual joystick as a virtual camera which will be useful for designers and developers who create new game mechanics and work with models and cinematics.

Scene objects & Blueprints

The plugin allows to use of any objects in the scene, including all blueprints, cameras, etc., and moving them in real-time in the game scene using the Virtual App, which transfers the movement and rotation data of an Android device in space.

ARCore motion tracking

The Virtual App enables a user to drive a Camera and Objects using an Android device in a virtual production environment. With ARCore the position and rotation of the Android device are broadcast wirelessly to the PC.

Game-design input

This technology allows you to turn Android devices into virtual joystick, which will be useful for game designers.

Original product demonstrations

Camera, controller & nDisplay in use

Android phone as a virtual-production tracker

See how an Android phone’s position and rotation can drive a virtual camera in Unreal Engine.

Virtual joystick in Unreal Engine

Use a mobile device to move scene objects and test controller input in Unreal Engine.

Virtual production in Unreal Engine 4/5

Use an Android device for scene viewing and camera control in Unreal Engine 4/5.

nDisplay In-Camera VFX

See an Android device used with an nDisplay scene for in-camera visual effects.

Original resources / 2019–2020

Technical notes & product history

The original articles and demonstrations are preserved here with their publication dates. Engine workflows, integration plans and availability statements belong to that historical context. For current setup and compatibility, use the documentation and Fab listing above.

Virtual Production in your Android device Unreal Engine 4/5

This technology allows you to turn Android devices into a virtual joystick, as well as into a virtual camera, which will be useful for designers and developers who create new game mechanics and work with models and game cameras. For local Video Streaming, we use our streaming system which works in Play Editor and can be used with Sequence Recorder. For remote Video streaming, we are using Pixel Streaming Plugin and it works only in a Standalone Game and in a compiled project.

Virtual Camera System in your Android device

The plugin allows to use of any objects in the scene, including all blueprints, cameras, etc. and moves them in real-time in the game scene using the Virtual App, which transfers the movement and rotation data of an Android device in space.

How to setup a Virtual Camera in Unreal

How to configure a virtual camera to work with nDisplay

Tutorial on how to work with Virtual Plugin in off-axis projection nDisplay

Virtual App turns your device into a virtual joystick and virtual camera
Virtual App UE4/5

We developed Virtual App to transmit and receive data of device location in the space between the device and PC – it is used in conjunction with our Virtual Plugin. The change of position and rotation of the Android device is broadcasted wirelessly to the PC using ARCore. It is already implemented on Unreal Engine 4 and will also be integrated into Unity, Maya, Blender, and Cinema 4D.

The plugin allows to use any objects in the scene, including all blueprints, cameras, etc. and move them in real-time in the game scene using the Virtual App, which transfers the movement and rotation data of an Android device in space.

This technology allows you to turn Android devices into a virtual joystick, as well as into a virtual camera, which will be useful for designers and developers who create new game mechanics and work with models and game cameras.

We don’t limit ourselves by working only with a virtual camera, although this functionality is also present in our development. We developed the Virtual App for Android – it is the tool, that allows you to move not only the camera but also any other objects inside the scene in real time. Our plugin – Virtual Plugin for UE4 supports the simultaneous use of multiple devices. Based on this tool, it is possible to develop different multiplayer game mechanics – this applies to both the game camera and any other objects in the game.

3d software and engines

Our goal is to create a unified tool for developers, designers, and animators who can animate their objects or record videos with one movement of the phone. Moreover, the most promising way to use it is the ability to create game mechanics that will allow gamers to use the app as a joystick/camera, etc. We plan to implement the product on five platforms: UE4, Unity, Maya, Cinema 4D, and Blender.

Virtual Plugin

The app is planned to be free until we get a sufficient number of users, allowing us to switch to the sales of monthly or annual licenses, both for users and for companies. The paid version will include product updates, as well as a wider range of tools that will be implemented in the app.

POSSIBLE WAYS TO USE THE APP

Working with virtual
cinematic camera

Using it like joystick for single games and simultaneous
use of multiple devices for a collective game

Works like virtual reality helmet

New AR Technology Changes Game Dev Industry

Over the past six months, our team has been working on the implementation of the idea that will allow using the phone as a tool for developers and designers who work on UE4. Soon, possibly it will be available for other game engines, and design programs.

The main idea is to use a smartphone as an additional tool for developers and designers — the user can move any object, camera, blueprint, etc in a game scene in real-time moving the phone. There are no restrictions, you can walk around the room, rotate the phone in all axes, and the selected object will repeat all your movements in the game scene. You only need a smartphone that supports ARCore For the tracking — Forget about sensors and cameras, you don’t need them anymore.

Why we created the App?

We believe that the phone, as a gadget available to each user, can become a valuable tool in the hands of developers and improve existing models of working with animation, tracking, and cinematic scenes, and overall allow developers to create new game mechanics, where the phone is the next interactive step to continue the gameplay.

We decided to offer open access to the plug-in and its source code so that any developer could independently port our App to any other platform and use it for their work purposes.

We did the same with our Virtual App, it will be free until we can offer the widest possible range of functions and features for users to cover all possible needs for using this technology.

https://www.unrealengine.com/marketplace/en-US/product/virtual-plugin

We did the same with our Virtual App, it will be free until we can offer the widest possible range of functions and features for users to cover all possible needs for using this technology.

https://play.google.com/store/apps/details?id=byowls.virtualapp

Soon we will record some video tutorials and will demonstrate the possibilities of using this application. But even now we can say that there can be a huge number of ideas and we gladly let each of you independently test this technology and decide for yourself where and how it will help you in your work and your implementation of a new idea and game mechanics.

We ask everyone who is interested in this technology to try it, leave a review, offer, or to take part in the development of a technology that is aimed at changing game standards for game dev.

You can see how to use the plugin and App following our link — https://byowls.com/virtual-app/

ByOwls studio showreel
ByOwls artwork