The Shotty post-production studio is an established company on the market for rendering and special effects computing for the global film industry. The processes of rendering and special effects require a lot of GPU computing power. With the help of the SONM computing power marketplace, Shotty has acquired the resources needed for video rendering, allowing a significant reduction in production costs.
The challenge for Shotty
Even for smaller tasks than feature-length films, Shotty ran into the problem of optimizing computing resources: their own video cards and device RAM were not up to the job, meaning tasks overran deadlines, which violated agreements with the client. In order to complete the order on time, the customer had to rent capacity directly from one of the big vendors: Amazon, Google, and Microsoft. The cost of this rental was prohibitively expensive.
The customer approached SONM with the following task:
Shotty needed to replace the face of a dancing stand-in with the face of an actor in an MP4 video, frame by frame. For this, it was necessary to use the client’s own application and teach a neural network to recognize faces in the video in advance, with minimal error margins.
Solving the problem with SONM Platform
The possibility of choosing the required resources at the required time for the required price on the SONM platform allows computing power for rendering special effects to be rented with a saving of up to 80%. And there is no increase in the time needed to process the task.
SONM developers guided the customer starting their project. As initial data, Shotty used several videos featuring the two actors. For teaching the neural network, the customer created a Docker container from their program and ran it on the SONM platform.
“We assisted the client to run a selection of frames with a specific face in the program. After launching in learning mode it built its own mathematical model based on neural networks, which then allowed the face in the video to be replaced. At the end of this very long process, the program created interior data and launched the replacement process. It takes the video frame by frame and transfigures it, superimposing the new face from the neural network,” says Eugene Manaev, Chief System Architect at SONM.
The solution included two stages:
- Owners of hardware that was connected to SONM provided computing resources for teaching neural networks until a satisfactory result was achieved (a neural network is used for the recognition of a face in motion);
- The program then superimposed the actor’s face upon the face of the stand-in.
“The first phase lasted until the required superimposition error rate was achieved. The neural network identified the face of the actor and the face of the stand-in — everything took place in motion — and its initial error rate was 10 units. After learning, it fell to a single unit, so the case can be considered successful,” explains Alexei Krotov, Head of Business Development for Russia and CIS, at SONM.
Specifications of the hardware made available to Shotty by SONM suppliers
CPU Celeron 2.8, RAM 4Gb, GPU NVIDIA 6*1060 3 Gb (only a GPU NVIDIA was suitable for the given task).
Result
The neural network learning process took a day, reaching the target level of model correctness. That was 3 times faster than using the customer’s PC with one GPU NVidia 1080 Ti card.
Faces were replaced at a rate of two frames per second, though the productivity of the SONM hardware supplied allows 20 frames.
The possibility of this kind of rendering on SONM hardware is proven, and the potential costs of rendering using mining rigs have been reduced by at least five times.
The client received the service free as part of active testing of GPU computing on SONM.
Potential for co-operation
Shotty is ready to continue collaboration on a commercial basis and is currently working on a number of applied tasks with the SONM team.
“This is also an advantage for such clients because the market itself determines the price. Miners put up their price, and the customer determines what volume of computing resources is needed for the video, how much time and money it would cost on Google, Amazon or render farms, and chooses the most advantageous option of those offered on SONM,” explains Alexei Krotov. “As for the Shotty case, we will see its real price on our trading platform in half a year, and in a year, with the presence of today’s alternatives like SONM, the market will balance out.”
Originally published at sonm.com on October 25, 2018.
