Bitrate control and quality layers in JPEG2000

A lot of high-quality digital content providers struggle with network traffic control. Sending a high-resolution video or high-quality still images is a demanding task where network connection may easily become the bottleneck of your solution. In this post, we will try to provide ways to alleviate network traffic with unique JPEG2000 properties and encoding/decoding mechanisms. These mechanisms might be used to achieve constant video bitrate (and thus fulfill various QoS goals), to get most of the connections with very limited bandwidth (e.g. drones) or simply to limit data transmitted while viewing still images. We will pay particular interest to so-called quality layers, which allow progressive video/image quality improvement and we will let you assess their usability in your project.

Bitrate control with JPEG2000

JPEG2000 is based on discrete wavelet transformation  (DWT) which creates so-called “resolutions”, therefore the simplest possibility for bitrate control is discarding resolutions. The idea is that only several of the smallest DWT resolution levels get transmitted to the client in such a case. This option provides a fast way of progressive quality improvement with no special demands/settings during image encoding but may be a poor choice if you’re in desperate need of constant video bitrate because the size of individual resolutions in a JPEG2000 file varies significantly with image content. For example, a black image would be compressed more effectively than images with lots of visual content (see our example below for the reference).
Users of JPEG2000 may be familiar with so-called rate control that allows you to specify a limit on image size during encoding to JPEG2000. Moreover, only the data that contributes the most to visual quality is included in such a rate-limited image. Applying a rate control limit on each frame of the video leaves you with an upper bounded maximal bitrate of the video. For example DCI video standard  (p.33) defines that the size of each frame can’t exceed 1,302,083 bytes at 24 FPS, meaning that the video bitrate won’t exceed 250 Mb/sec. However, we can’t use rate control for lossless storage of data.
Quality layers in JPEG2000 allow you to specify multiple image size limits and thus provide the possibility of progressive image/video quality improvement especially useful in a networked environment. The user can specify several quality layers and their sizes while encoding the JPEG2000 file. When sent over the network each received quality layer contributes to the quality of the image/video that can be displayed on the client side.
Moreover, quality layers can be used for lossless JPEG2000 images where the very last quality layer contains all the image data. In such a scenario, all the original data is preserved (on the server side) and you’re still able to save your network bandwidth by sending only several small quality layers to the client. The killer feature here is again that only the data contributing the most to visual quality (for the given bitrate limit) is transmitted.

How does it work?

Let’s look at a real-life example.
We encoded two sample images with 3 DWT resolutions and specified 2 quality layers (0.5 MB for fast previews and 15 MB). The following graphic depicts how data distribution between resolutions depends on image content and how the data is selected from quality layers.

Only the data that contributes the most to the quality of the image are included in quality layer 0 for image A. Almost all the data in quality layer 0 comes from the very first resolution of the image.

Image B is clearly “simpler”. Lots of similar background results in less data for a compressed image as well as smaller first DWT resolution which in turn drives more data from the second resolution to be included in quality layer 0 than in the case of image A.
In the context of resolution discarding we see that discarding two resolutions would leave us with 1.04 MB in image A, whereas the size of image B would be 0.34 MB – that’s three times the difference!
Images were taken from ASC/DCI Standard Evaluation Material (StEM) Mini-Movie.  Note that bit depth was reduced to 8 bits to reduce image size for educative purposes.

How can I create quality layers?

Quality layers are created during image encoding. Therefore you need a JPEG2000 encoder.
Comprimato UltraJ2K  allows you to create your JPEG2000 encoder on CPU, CUDA or AMD graphics+OpenCL with just a few simple and unified API calls.
Specifying 2 quality layers in your C/C++ program may be as simple as including the following two UltraJ2K API calls in your encoder configuration:

cmpto_j2k_enc_cfg_set_layer_rate_limit(
      enc_cfg,
      0,
      CMPTO_J2K_ENC_COMP_MASK_ALL,
      CMPTO_J2K_ENC_RES_MASK_ALL,
      524288); // 0.5 MB - quality layer 0
cmpto_j2k_enc_cfg_set_layer_rate_limit(
      enc_cfg,
      1,
      CMPTO_J2K_ENC_COMP_MASK_ALL,
      CMPTO_J2K_ENC_RES_MASK_ALL,
      15728640); // full quality (max. 15 MB) - quality layer 1

With this encoder configuration properly set the encoder is fed raw image data and produces your JPEG2000 with the quality layers.

Example application – progressive quality improvement

A common standard seen on well-known websites is that a low-quality image is displayed first and then the quality is progressively improved by sending several smaller versions of the image first followed by the original file; whereas receiving a single JPEG2000 file encoded with quality layers, the client application is able to progressively improve the image quality with no need to transmit any additional data.
With JPEG2000 there is no need to generate multiple versions (sizes) of the image/video. You simply create one master file (JPEG2000 with the quality layers) and you get bitrate control out of the box!
Even better – you can create this master file with Comprimato UltraJ2K SDK and leverage the power of your GPU to achieve state-of-the-art performance – yes it’s possible to do that real-time, even for 4K video.
Let’s do some simple maths. If your server (camera, drone etc.) creates/hosts JPEG2000 video where quality layer sizes are 0.5 MB, 1 MB, and 2 MB and your video is at 30FPS, you can switch instantly between 123 Mb/sec, 245 Mb/sec, and 490 Mb/sec constant bitrates. The choice can be driven for example by the momentary state of your server-client connection or by the client’s ability to process received data.

Example application – digital image storage

Another good application for quality layers may be in the domain of large images where smaller/lower quality previews are often needed. The quality layer mechanism allows the client to display an image without the need to transmit all the data, decreasing thus response latency as well as saving the connection’s bandwidth. The successive quality layers may be sent, later on, increasing image quality progressively.
Images can be even stored in a mathematically lossless mode which does not discard any of your valuable data and saves you storage space anyway. This may be highly desirable for usage in the medical field or even for photos from high-quality digital cameras.

Quality layer encoding with Comprimato UltraJ2K in real-time and even faster

If you’re willing to use GPU platforms and your image resolution is up to 4K, you are good to go with our online speed calculator  (for lossy encoding). Interested in 8K UHD or lossless encoding? Contact our support for performance details.
And of course, Comprimato UltraJ2K allows you to decode JPEG2000 with the quality layers at the client side with either GPU or CPU.
Moreover, Comprimato UltraJ2K is a software solution that is getting faster and faster with new generations of graphic cards and processors!
If you think that the image quality layer and the world’s fastest GPU-accelerated JPEG2000 Comprimato UltraJ2K SDK  may be the right thing for your project or if you have specific questions regarding UltraJ2K integration, please contact our support. We will be happy to help you!