Hardware acceleration with VAAPI not working for H.264 decoding

[zsozark]

I am trying to use hardware acceleration with VAAPI for decoding H.264 videos using JavaCV on Linux for weeks now. However, it does not seem to work, while the same setup works perfectly with the FFmpeg command line.

Environment:

• OS: Debian 12
• CPU: 12th Gen Intel(R) Core(TM) i5-12500
• GPU: UHD Graphics 770
• JavaCV version: 1.5.11

Steps to Reproduce:

1. Create a new FFmpegFrameGrabber with the following options:

FFmpegFrameGrabber grabber = new FFmpegFrameGrabber(videoPath);
grabber.setOption("hwaccel", "vaapi");
grabber.setOption("hwaccel_device", "/dev/dri/renderD128");
grabber.setOption("hwaccel_output_format", "vaapi");
grabber.setVideoCodec(avcodec.AV_CODEC_ID_H264);
grabber.start();

2. Try to start the grabber.

(I have tried simple setup as well, and numerous other, but neither of them worked)

grabber.setOption("hwaccel", "auto");

Expected Behavior:

The video should be decoded using VAAPI hardware acceleration.

Actual Behavior:

The decoding falls back to software decoding.

Additional Information:

The command ffmpeg -hwaccel auto -i video.mp4 -f null - works fine and uses VAAPI on the same hw.
I believe I have installed all the necessary dependencies and drivers.

gradle dependencies

implementation('org.bytedeco:javacv-platform:1.5.11')
implementation('org.bytedeco:opencv-platform-gpu:4.10.0-1.5.11')
implementation('org.bytedeco:ffmpeg:1.5.11')
implementation('org.bytedeco:ffmpeg-platform:1.5.11')
implementation('org.bytedeco:ffmpeg-platform-gpl:7.1-1.5.11')

Thank you for your help!

[saudet]

Please try again with the snapshots: http://bytedeco.org/builds/

[zsozark]

Thank you for your prompt answer! I've tested with the latest 1.5.12-SNAPSHOT, but the problem still persists.

Edit: Hardware acceleration seems to working with nvidia GPU by the way (on a different hw ofc).

[silenium-dev]

@zsozark could you enable ffmpeg debug logs and post them here? I suspect that something goes wrong when ffmpeg tries to initialize the VAAPI device.

[zsozark]

Sure, @silenium-dev !

Here is with:

grabber.setOption("hwaccel", "auto");

ffmpeg-auto.log

And this is with other settings (not sure if that's matter):

grabber.setOption("hwaccel", "vaapi");
grabber.setOption("vaapi_device", "/dev/dri/renderD128");
grabber.setOption("hwaccel_output_format", "vaapi");

ffmpeg-vaapi.log

I saw changes because I've extracted ffmpeg from ffmpeg-7.1.1-1.5.12-SNAPSHOT-linux-x86_64.jar and it did play my video with hw acceleration, while the one in ffmpeg-7.1.1-1.5.11-linux-x86_64.jar did not, but my code still not using hwacceleration and I'm certain that my classpath contains the right jar.

Here is my code as well just in case. The devil is in the details...

Java code

private static void grabber(String videoPath) {
    avutil.av_log_set_level(avutil.AV_LOG_TRACE);

    try (FFmpegFrameGrabber grabber = new FFmpegFrameGrabber(videoPath)) {
        grabber.setOption("hwaccel", "auto");
        grabber.start();

        while (true) {
            Frame frame = grabber.grabImage();
            if (frame == null) {
                break;
            }
        }

        grabber.stop();
    } catch (Exception e) {
        e.printStackTrace();
    }
}

[silenium-dev]

The options for using VAAPI seem to be passed to the wrong function:

Info: avformat_open_input rejected some options: Option: pixel_format, value: bgr24 Option: vaapi_device, value: /dev/dri/renderD128 Option: hwaccel, value: vaapi Option: hwaccel_output_format, value: vaapi

And iirc, VAAPI doesn't support bgr24 as pixel_format, only rgb0, but I don't know much about opencv, so I can't tell you how to configure it this way.

I'm currently on the phone, I'll have a more detailed look at this when I'm on my computer.

Regarding the changes between 1.5.11 and 1.5.12: 1.5.11 wasn't compiled with VAAPI support, which was fixed in #1608, but only available in snapshot until the next release

[silenium-dev]

I've looked into FFmpegFrameGrabber, it doesn't support any hardware accelerated decoding. The options that are configurable via setOption are only passed to AVFormatContext, which is the demuxer, and not the same ones that you could pass to the ffmpeg CLI.
There is a setVideoOption, which is passed to the AVCodecContext, but this is not enough to configure hardware acceleration.
You'd need to create a hw device context and pass it to the decoder, like in the ffmpeg hw_decode.c example, but this is currently not implemented in FFmpegFrameGrabber

[zsozark]

Thank you so much for your detailed explanation about hardware decoding in FFmpegFrameGrabber. I really appreciate the help.

Just to confirm my understanding: Currently, there's no straightforward way to enable hardware acceleration in the library, correct? If I want to pursue this, I'd likely need to contribute to the project or explore alternative libraries.

Thanks again for your time and expertise!

[silenium-dev]

Yep, no support unless someone implements it in code

[wheelerkft]

Hi Guys! Thanks for this useful help! We also need hardware acceleration ... :(

@saudet: Can you help us what we need to modify or implement for this? It is not clear to me what @silenium-dev's last sentence means. "You'd need to create a hw device context and pass it to the decoder, like in the [ffmpeg hw_decode.c example]" Thx!

[wheelerkft]

@silenium-dev

Thank you for the useful example! I tried to implement it, but during the run I get the following error.

Log:

[AVHWDeviceContext @ 0x7fe9842fa240] libva: /usr/lib/x86_64-linux-gnu/dri/iHD_drv_video.so has no function __vaDriverInit_1_0
[AVHWDeviceContext @ 0x7fe9842fa240] libva: /usr/lib/x86_64-linux-gnu/dri/i965_drv_video.so init failed
[AVHWDeviceContext @ 0x7fe9842fa240] Failed to initialise VAAPI connection: -1 (unknown libva error).

Code:

AVBufferRef hw_device_ctx = new AVBufferRef();
av_hwdevice_ctx_create(hw_device_ctx, AV_HWDEVICE_TYPE_VAAPI, "/dev/dri/renderD128", null, 0);

VA-API version:

vainfo: VA-API version: 1.17 (libva 2.12.0)

[wheelerkft]

I've worked magic with libraries and now I have hardware acceleration. :) But now another bug has appeared ...

Error: [swscaler @ 0x7fb8579fe840] vaapi is not supported as input pixel format

[silenium-dev]

After VAAPI accelerated decoding, the frame is in GPU memory (vaapi pixel format), but swscale operates on frames in CPU memory (e.g. yuv420p, nv12, rgb24). So you need to transfer it to CPU memory first.

[wheelerkft]

Thank you for this useful help!