UltraFlo Beta – for flawless 8K studio workflow

Make working with 8K files on NAS feel like they’re local.

8K Today

The world has changed. 8K is coming to cameras¹, editors and even to broadcasters². This is great news for consumers, but there is a downside. As well as the number of pixels, the requirements on workflow are about four times larger than in the case of 4K.
The first issue is the size of uncompressed 8K video: it’s huge! Just consider the size of one single frame in a broadcaster’s 4:2:2 10bit format:

7680 x 4320 x (1 + 0.5 + 0.5)components x 10 bits = 664 Mb = 83 MBytes per frame

It’s an entire DVD volume in one second of 8K@60FPS.
Or take a lightly compressed 8K VR/360° video – that’s about 450 MBps or 3.6 Gbps. This is more than a 1Gbps network can handle. The same holds true for 4K VR video, as can be seen from Table 1 below.

Table 1: VR/360° video at 8K is about 450 MBps or 3.6 Gbps. The same video at 4K resolution is about 150 MBps or 1.2 Gbps. 8K VR video: 8192×8192, 4:2:2, 8bit, 15:1 compression. 4K VR video: 4096×4096, 4:2:2, 8bit, 10:1 compression. 8K Broadcast – Broadcasters use 4:2:2, 10bit, 60 FPS profile with 400 Mbps for 4K. Hence for 8K it will probably be around 1600 Mbps.

It’s not easy to solve a situation when you’ve only got a 1GbE network in a smallish studio and want to work with hi-res videos in real-time simultaneously. Mainly, this is because, if you want to solve it using hardware, you’d need to update to 10GbE or even 40GbE infrastructure (storages, switches, and network cards), which may cost you a dozens of thousands of dollars. So, we decided to go the software way, and offer a solution with a minimum of hardware updates.

UltraFlo Beta

We want to address these problems with our storage accelerator UltraFlo. UltraFlo aims at saving space for uncompressed video and at removing the network bottleneck for real-time preview/working in sub-resolution. We’ve tried to eliminate differences between local and network video editing for groups of people in studios.
We made UltraPix plugins the ideal solution for individuals or very small teams that need to work with hi-res videos on local workstations. Now imagine UltraPixes on multiple workstations fighting for the same files on one storage with limited read/write disk speeds – it can’t end well. So, that’s why we’ve come up with a central point that controls access and given it caching and load-balancing capabilities. And that’s UltraFlo.

Autoproxy feature

UltraFlo also saves storage space using an auto-proxy feature. This means there are multiple resolutions in the master J2K/MXF file without the necessity of generating and managing proxies. It contains them all – but only virtually (technical details can be found in our previous blog post). It’s like the structure of an onion. The master file sits in storage and we can easily peel off the higher resolutions and send Premiere/Nuke just the core layers it needs in real time. Which means you can work with proxies without having to generate and manage them.

Illustration 1: Screenshot of network storage usage with/without UltraFlo. With UltraFlo there is no need to use proxy files and regenerate them in the case of an edit, there is only a master file (second screenshot). UltraFlo can extract the required resolution from the master on the fly and save network bandwidth.

How it works

The UltraFlo storage accelerator server(s) sits between workstations and network storages. When the UltraPix plugin in Adobe Premiere or Foundry NUKE needs a frame with, for example, the 2K resolution from an 8K master (J2K image or MXF), then the plugin asks for it from our UltraFlo accelerator server. The accelerator has a database of all supported files saved on storage, so it instantly knows where the requested 2K data lies in the image file, and redirects them to Premiere/NUKE.
Without UltraFlo, the client would need to decode the video file and process it to know where the desired frames lay. UltraFlo knows where all frames lie, so it saves the CPU time of the workstations and limited bandwidth between UltraFlo and workstations.

Illustration 2: Example of UltraFlo usage. A reviewer wants to watch an 8K video file from the network drive in Adobe Premiere. But 2K would be sufficient for them. The network drive is managed by Comprimato’s UltraFlo accelerator, so it is able to transfer to them just the 2K resolution, which saves ~93.75 % of bandwidth.

Performance can be scaled up using multiple UltraFlo servers and multiple storages which are paired together to minimize lags and frame drop even in big studio workflows of dozens of people.
When network capacity from UltraFlo server to storages isn’t completely utilized, it can be used for predictive preloading. This pre-caching feature tries to guess which files/frames the UltraPix plugin will need, loads them in advance and has them ready when the plugin actually asks for them. This minimizes frame drops in playback and waiting time for thumbnails.

Input files not in JPEG2000 or MXF?

Just to restate, UltraFlo works with JPEG2000 as the main format. We understand it is still not yet quite standard comparing to EXR or other uncompressed formats. For this case, we prepared GPU accelerated video conversion/ingest.
Of course, you can use Adobe Media Encoder, but this may not be the best solution for encoding/decoding files saved on network storage. It can be time-consuming, and copying them there and back takes bandwidth, so we added a standalone video ingester feature to UltraFlo. Ingester is a tool that runs on a storage server and looks after one directory. It independently transcodes every file you put there to J2K or MXF and, once finished, the files are ready to be picked up by everyone with access.
The main advantage compared to Media Encoder with UltraPix is that, unlike UltraPix, the Ingester can use the full potential of multiple GPU cards in a storage server and utilize them completely thanks to our fast UltraJ2K video codec. Our current alpha version of Ingester can transcode TIFF, PNG, and EXR. Support for other formats will follow.

Illustration 3: Chart showing Ingester FPS performance. Input files were 4K TIFF images (8bit) and the output format was as close as possible to UltraPix export profile (lossy 10bit 4K J2K with 7-level DWT and quantization of 0.76). AMD results came from our current OpenCL alpha, Nvidia CUDA results are expected numbers based on the performance of our UltraJ2K codec.

Resolutions in a JPEG2000 file can be ordered from the lowest to highest. So the requested data can be read sequentially as a whole in the most efficient way. The JPEG2000 standard also specifies images in other resolution orders which can’t be accelerated in this way. UltraFlo supports them, but it won’t be as efficient as it could be. Of course, you can use our export profile in Adobe Media Encoder or our Ingester to transcode them.

UltraFlo

UltraFlo helps you work with 8K videos on your local network. It saves space on your storage because of JPEG2000 compression and reduces network requirements by using our autoproxy feature to send only the resolutions needed. UltraFlo makes it possible to use 10GbE or 1GbE networks at 8K studio workflow and enables you to work with hi-res files on your network just like you do with the local ones.
We are also working on integrating our storage accelerator into the existing solutions of storage vendors. For example, we are working with Qumulo storages.
So, tell us the problems that you’re struggling with in your current 8K or 4K workflow. And if you want to know more about our solution, please contact us at ultrapix@comprimato.