Epic Games Introduces MetaHuman Animator

Epic Games recently unveiled the MetaHuman Animator, a remarkable tool that empowers developers to capture incredibly lifelike human faces using only an iPhone and a PC. This groundbreaking technology has the capability to generate astonishingly realistic simulated individuals, as demonstrated in the captivating video.

Epic Games has introduced the MetaHuman Animator, a tool that promises to deliver high-quality facial animation in mere minutes. Initially showcased at GDC 2023, this advanced version of the user-friendly digital human pipeline revolutionizes performance capture for MetaHumans by offering exceptional fidelity.

The MetaHuman Animator boasts remarkable capabilities, allowing users to effortlessly capture an actor’s performance using an iPhone or a stereo head-mounted camera system (HMC) and seamlessly apply it as high-fidelity facial animation to any MetaHuman character, eliminating the need for manual intervention.

Epic Games emphasizes that every nuanced expression, gaze, and emotion is faithfully captured and reproduced on the digital human, making for a truly immersive experience. The best part is that achieving incredible results is both simple and straightforward—accessible to anyone.

For those new to performance capture, MetaHuman Animator provides a convenient means of incorporating facial animation based on real-world performances into MetaHumans.

Meanwhile, for those already familiar with performance capture, this new feature set significantly enhances existing capture workflows, streamlining the process, saving time and effort, and granting greater creative control. By combining MetaHuman Animator with their current vertical stereo head-mounted camera setup, users can attain even greater visual fidelity, elevating the quality of their work.

In-Depth Information

Previously, faithfully replicating every subtle aspect of an actor’s performance on a digital character required a team of experts and several months of meticulous work. However, with the advent of MetaHuman Animator, this arduous process becomes a breeze, consuming only a fraction of the time and effort.

The impressive new feature set leverages a 4D solver that seamlessly integrates video, depth data, and a MetaHuman representation of the performer. Utilizing GPU hardware, the animation is swiftly generated locally, delivering the final result within minutes.

All these intricate operations take place behind the scenes. Developers simply need to point the camera at the actor and hit the record button. After the capture process, MetaHuman Animator impeccably imparts the actor’s unique individuality and subtle nuances onto the chosen MetaHuman character.

What sets this technology apart is its semantically accurate animation data, utilizing appropriate rig controls, and ensuring temporal consistency. Smooth transitions and control enable effortless artistic adjustments, granting users the freedom to refine the animation according to their preferences.

Blue Dot video

Epic Games’ 3Lateral team collaborated with local Serbian artists to create a captivating short film called “Blue Dot,” which serves as a testament to the capabilities of their groundbreaking technology. In the film, actor Radivoje Bukvić delivers a monologue inspired by a poem written by Mika Antic. The performance was captured at Take One studio’s mocap stage, with cinematographer Ivan Šijak taking on the role of director of photography. The astonishing outcome is that the human depicted in the video is actually an animated character.

These nuanced outcomes exemplify the remarkable level of fidelity that artists and filmmakers can achieve by utilizing MetaHuman Animator in conjunction with a stereo head-mounted camera system and traditional filmmaking techniques.

What’s truly remarkable is that the team managed to attain such impressive animation quality with minimal additional adjustments on top of the exceptional results produced by MetaHuman Animator.

Universal Facial Animation for MetaHumans

Epic Games has confirmed that developers can effortlessly apply facial animation, captured using MetaHuman Animator, to any MetaHuman character or any character utilizing the new MetaHuman facial description standard. This streamlined process requires only a few simple clicks.

This remarkable capability grants developers the freedom to design characters according to their creative vision, with the assurance that the facial animation seamlessly integrates with their creations.

Now, let’s delve into the technical details for a moment. This achievement is made possible by the Mesh to MetaHuman feature, which can generate a MetaHuman Identity using just three frames of video and depth data obtained through an iPhone or reconstructed from data obtained via a vertical stereo head-mounted camera.

Through the process of customizing the solver for each actor, MetaHuman Animator demonstrates its proficiency in generating animation that seamlessly integrates with any MetaHuman character, achieving a harmonious alignment. Additionally, it can leverage audio data to convincingly animate the character’s tongue, adding another layer of realism to the overall performance.

Harness the Power of iPhone for Capturing

Epic Games has confirmed that developers can effortlessly apply facial animation, captured using MetaHuman Animator, to any MetaHuman character or any character utilizing the new MetaHuman facial description standard. This streamlined process requires only a few simple clicks.

With the user-friendly MetaHuman Animator, even a simple setup comprising an iPhone (12 or above) and a desktop PC is sufficient. This is made possible by the updated Live Link Face iOS app, which enables the capture of raw video and depth data. The captured data is then seamlessly integrated into Unreal Engine for further processing.

For those seeking even higher fidelity, developers can utilize MetaHuman Animator alongside an existing vertical stereo head-mounted camera system.

Regardless of whether developers opt for an iPhone or a stereo HMC, MetaHuman Animator significantly enhances the speed and user-friendliness of the capture workflow. This flexibility empowers developers to choose the hardware that best suits their specific shooting requirements and desired level of visual fidelity.

Moreover, the captured animation data supports timecode, facilitating easy alignment of facial performance animation with body motion capture and audio, resulting in a cohesive and comprehensive character performance.

Unleash Creative Freedom on Set

MetaHuman Animator is exceptionally well-suited for facilitating creative iteration on set, thanks to its swift processing and seamless transfer of facial animation onto any MetaHuman character.

By reviewing animation data directly within Unreal Engine during the shoot itself, developers have the opportunity to evaluate the capture quality well in advance of the final character animation stage. This real-time assessment empowers them to make informed decisions and adjustments on the spot.

Furthermore, since reshoots can be conducted while the actor is still present on stage, developers can capture the ideal take instantly, eliminating the need for costly and time-consuming arrangements to reconvene at a later date. This efficient workflow enables the team to optimize their resources and ensure that the best possible performance is captured without unnecessary delays.