Decoder API Tutorial

This tutorial provides an introduction to writing a h265 video decoding using libde265 using the C language API.

A look around the source tree

When checking out the source repository, the source is organized into these subdirectories:

• libde265 - main library source code. Everything below libde265/encoder is required only for h265 encoding and can be omitted for a decoder-only use.
• dec265 - example h265 video decoder
• enc265 - example h265 video encoder
• sherlock265 - h265 video analysis tool (shows motion-vectors, CTB subdivisions, coding modes)
• tools - miscellaneous tools mainly for encoder development

The code inside 'libde265' is usually compiled into a link-library, while the other directories contain executable programs. We include build scripts based on autoconf and/or CMake.

libde265 is written in C++11, but provides a plain C interface for easy integration. All decoder functions are defined in the header de265.h. This file also is the reference documentation. The header en265.h includes the encoder API and is not needed for a decoder-only software.

This documentation refers to the new API of libde265, which is currently developed in branch frame-parallel. The old API for branch master is very similar, though, and most of what is written here also applies.

Writing a simple video decoder

First, we need to create a decoder context instance using

de265_decoder_context* de265_new_decoder(void)

The decoder uses background threads to do the actual decoding. Before using the decoder, we have to start these background threads with

de265_error de265_start_worker_threads(de265_decoder_context*, int number_of_threads)

Decoding without background threads is currently not supported. Hence, this function must be called. When the decoder is not needed any more, the decoder context must be freed again after use by

de265_error de265_free_decoder(de265_decoder_context*)

Decoding loop

Decoding the video consists of asynchronously pushing compressed video data into the decoder context and pulling decoded frames out of the decoder. You can push in as much data as you want and pull out frames whenever you want. There exist several functions for pushing data into the decoder. The most frequently used might be

de265_error de265_push_data(de265_decoder_context*, const void* data, int length,
                            de265_PTS pts, void* user_data);

This pushes length bytes of data into the decoder. The data will be copied into the decoder context. You can also provide a pts timestamp and user_data. These are not actually used by the decoder, but returned again in the decoded image decoded from this data for your use. The data provided to this function should be a h265 only elementary bitstream with startcode markers. At the end of the file, you should call

de265_error de265_flush_data(de265_decoder_context*);

to tell the decoder that no more data will follow this call. It will then flush all its output queues.

If your data consists of packetized data without startcodes, you can also push complete NAL units into the decoder context using

de265_error de265_push_NAL(de265_decoder_context*, const void* data, int length,
                           de265_PTS pts, void* user_data);

In order to decode the pushed data, call

de265_decode(de265_decoder_context*, int* more);

This does some decoding work, but you might have to call it several times to proceed further in the stream. It may stop either because the output picture buffer is full and you have to get these from the decoder before it will continue, or because there is not enough input data to proceed decoding.

When more is true, the decoding is still in the middle of the stream, so you will have to call de265_decode() again. The returned error indicates whether you have to provide more input data (DE265_ERROR_WAITING_FOR_INPUT_DATA) or if the output buffer is full (DE265_ERROR_IMAGE_BUFFER_FULL). If there is no error (DE265_OK), simply call the function again to proceed.

When more is false, the end of the stream is reached, or an unrecoverable error has occurred.

Decoded images can be extracted from the decoder context using

const struct de265_image* de265_get_next_picture(de265_decoder_context*);

If there are no images available, NULL is returned. After use, you have to free the images with

void de265_release_picture(const de265_image*);

You can keep hold of several images, if you like, but at some time, you must release them.

Decoding loop example

This is an example for a decoding loop. For another example, see dec265/dec265.cc. Note that we are not checking for DE265_ERROR_IMAGE_BUFFER_FULL since we remove all images from the output buffer after each call to de265_decode().

int more = 1;
while (more) {

  de265_error err = de265_decode(context, &more);

  if (err == DE265_ERROR_WAITING_FOR_INPUT_DATA) {
    // ... load more video data into 'buf'

    de265_error err = de265_push_data(context, buf, n, 0, NULL);
    ...
  }

  const de265_image* img;

  do {
    // show available images

    img = de265_get_next_picture(ctx);
    if (img) {
      // ... display image

      de265_release_picture(img);
    }
  } while (img);
}

Accessing image data

The de265_image* is an opaque pointer. To access the image data, you have to use the following functions:

int de265_get_image_width(const struct de265_image*,int channel);
int de265_get_image_height(const struct de265_image*,int channel);
const uint8_t* de265_get_image_plane(const struct de265_image*, int channel, int* out_stride);

enum de265_chroma de265_get_chroma_format(const struct de265_image*);
int de265_get_bits_per_pixel(const struct de265_image*,int channel);
de265_PTS de265_get_image_PTS(const struct de265_image*);
void* de265_get_image_user_data(const struct de265_image*);
void* de265_get_image_plane_user_data(const struct de265_image*, int channel);

Channels are 0,1,2 for Y,Cb,Cr, respectively.

Advanced memory management

For high-performance decoding, you might want the decoder to decode directly into user-supplied image buffers to avoid copying of images. This is possible by writing your own stub functions for allocation and deallocation of image buffers and passing them to the decoder context.

Specifically, you have to fill in this data structure:

struct de265_image_allocation
{
  int  (*get_buffer)(struct de265_image_intern* img,
                     const struct de265_image_spec* spec,
                     void* alloc_userdata);
  void (*release_buffer)(struct de265_image_intern* img,
                         void* alloc_userdata);

  void* alloc_userdata;
};

The de265_image_spec defines the properties of the image to be allocated (width, height, chroma format, alignment). The alloc_userdata variable is passed to each call of get_buffer() and release_buffer() but not used otherwise by the decoder.

The get_buffer() allocation function also received a de265_image_intern for which the image buffers should be set. Call de265_set_image_plane_intern() for each of the planes (Y,Cb,Cr) to set the memory buffers:

void de265_set_image_plane_intern(struct de265_image_intern* img,
                       int cIdx,
                           void* mem, int stride,
                           void *userdata);

The element userdata is again optional, copied to the respective image plane of de265_image_intern and not used by the decoder itself. You can use it, for example, when releasing the image.

Register your custom allocation function with

void de265_set_image_allocation_functions(de265_decoder_context*,
                                          struct de265_image_allocation*);

For a complete reference to the decoder API, check the file de265.h.