Transforming Film Production with Cloud-Based Virtual Technology

The Media & Entertainment industry is undergoing a significant transformation, with digital filmmaking at the forefront. Among the key innovations driving this change is virtual production, which combines real-time technology and cloud computing to enhance creativity and efficiency.

Mediapro Virtual Production embarked on a groundbreaking journey to leverage cloud solutions, reducing production time and expanding creative possibilities. This shift highlights the power of cloud computing in digital filmmaking, making high-quality production more accessible and cost-effective.

One of the most significant trends in the evolving media landscape is the increasing adoption of virtual production. Several factors contribute to this surge:

– The rise of streaming platforms

– Shifting viewer preferences

– Growing demand for social media content

– Advancements in hyper-realistic 3D environments and generative AI

Virtual production enables real-time tracking of camera position and orientation in a 3D space on a physical set. This technology integrates real actors into digitally created environments, either through LED walls or green screens. The key benefit? Actors can see and interact naturally with the digital environment, improving performances.

Benefits of Virtual Production in Digital Filmmaking

1. Real-time visualization: Directors, supervisors, and actors can see a blended composition of 3D environments and real elements immediately, reducing the need for reshoots.

2. Enhanced cinematography: Cinematographers can adjust virtual lighting dynamically to complement the physical set.

3. Optimized sound design: Sound directors can improve audio capture by understanding how the virtual environment affects acoustics.

4. Creative flexibility: Directors can iterate and refine scenes quickly, allowing for more experimentation.

With these advantages, virtual production is redefining digital filmmaking, offering an innovative and efficient approach to high-quality content creation.

New Challenges in Digital Filmmaking

While virtual production provides numerous benefits, it also presents technical challenges. The computing power required to create, render, and mix virtual environments with real-time footage is immense.

Cloud-Based Deferred Rendering: A Game Changer?

A crucial question arises: Should every studio maintain expensive on-premise computing power, or can AWS Cloud be leveraged for deferred rendering?

By using cloud computing, studios can produce high-quality visuals without the need for constant on-site computing infrastructure. This approach enables studios to maintain just enough capacity for real-time rendering, while offloading intensive processing to the cloud, optimizing costs.

At the end of 2023, Mediapro Virtual Production, the unit specialized in virtual production within the company, took an important first step. They produced the first European film made entirely using virtual production techniques in its Barcelona studio location. The movie, Parenostre (2024), generated valuable learnings as part of the film was shot using green screen technology and the rest with rear LED screen technology. Both synchronized with a 3D virtual environment. The film featured both indoor and outdoor settings, but all the shooting was done within a virtual production stage in the center of Barcelona.

One of the first challenges that the technical team encountered was the need to re-render most of the film material, as the quality of the rendered scenes didn’t meet the expected bar. To enhance the rendering quality of the green scenes, they needed to improve lighting and add intricate details to reinforce the script intention. The team estimated that, with the available human and technical resources, this would result in losing half a year in post-production. Not wanting to lose so much time, a solution was needed.

The solution was to execute a fast transition to the Amazon Web Services (AWS) Cloud. They deployed parallel 3D environment rendering, which mostly used Amazon Elastic Compute Cloud (Amazon EC2) GPU-based instances.

By leveraging AWS services, like Amazon EC2 GPU-based instances orchestrated by a computing node management service, Mediapro Virtual Production empowered its creative team with streamlined web access to on-demand rendering environments. This shift enabled them to integrate a global creative team, all within a best-in-class security framework.

This way, creatives were able to open and close work environments according to their creation needs. They were able to multiply their parallel work capabilities without the necessary cloud knowledge to deploy, operate, and maintain software-defined infrastructures.

Geographically distributed teams could now work on the same project with parallelized computing on the cloud to accelerate the results of the creative effort. This enabled them to work with more computing power than they could ever have had on a singular workstation alone.

The outcome was an 83 percent reduction in the estimated post-production time, implying a decrease in overall costs of the film and a faster time to market.

The next step was ensuring real-time synchronization between the studio and the 3D environment in the cloud. This required:

1. Sending real-time camera data to the cloud

2. Returning the rendered 3D environment to the studio within less than 10 frames of latency

Overcoming Latency and Quality Issues

Initially, latency was 200 ms, which was promising, but video quality did not meet the required standard for large LED screens.

To address this, Mediapro’s cloud architects and media streaming experts fine-tuned the process by:

1. Testing various video encoders and AWS media services

2. Developing a GPU streaming framework to capture high-quality video directly from cloud GPUs

3. Eliminating unnecessary processing steps to minimize latency

This cloud-based synchronization enabled studios in locations like Miami, Barcelona, New York, and Buenos Aires to seamlessly adopt virtual production.

Beyond improving production efficiency, this cloud-based approach unlocked a global network of creative professionals.

3D engine creatives are in high demand across industries like gaming and advertising. By shifting virtual production workflows to the cloud, Mediapro was able to:

– Access top talent from anywhere in the world

– Enable real-time remote collaboration on digital filmmaking projects

– Scale computing power as needed, without requiring expertise in cloud infrastructure management

In the early 2000s, audiovisual production centers housed massive on-premise computing power for video processing and storage. Today, the industry is shifting towards cloud-based infrastructures that:

– Provide high-performance computing without on-site limitations

– Support real-time collaboration across continents

– Leverage software-defined environments to enhance production flexibility

The digital filmmaking revolution is in full swing, and cloud computing is empowering creatives to push storytelling boundaries like never before.

Conclusion