Does it work in hololens?

[ZhengzhongSun]

I want to use ffmpeg in hololens, so I need the dll which can be used in hololens. How could I build it with this repo?

[khouzam]

Hi @ZhengzhongSun,

Yes, this would work on Hololens as it builds a Universal Windows Platform compatible binaries.

[ZhengzhongSun]

@khouzam hololens should use BuildFFmpeg.bat win10?

[khouzam]

Yes, that will build all architectures for Windows 10. You can specify run BuildFFmpeg.bat win10 win32 to only build the binary for x86 which is what HoloLens should run.

[ZhengzhongSun]

@khouzam Thanks. It works. But I used it to decode a 2048*2048 h264 video in hololens, it seems a little slow. Do you know about the hardware decode in hololens?

[khouzam]

Are you forcing decode? That will decode in software and be much slower and heavy on the CPU than using hardware decoding.

You might also want to take a look at @lukasf fork of the project until we coordinate to integrate their changes. They have done some great optimizations to improve the performance.

[ZhengzhongSun]

@khouzam Does ffmepg support hardware decode in hololens?

[khouzam]

From our implementation, using ffmpeg will not do hardware decoding, only software as we are using memory buffers to pass the information around. Depending on the type of media that you're trying to play, you can combine using FFmpeg and MediaFoundation to achieve much better media support while keeping great performance (this was one of the goals of this project). A lot of the decisions on how to proceed and what to use depends on the media format that you are trying to play. If you're looking to playback an h.264 video, that is usually able to be decoded in hardware by most Windows platforms. You can still leverage ffmpeg to parse the container and playback formats that are not natively supported by Windows (xvid, divx, rtsp for example), while still levaraging the hardware capabilities of the device.

If you have the video playing, have you tried to set the ForceDecode parameter to false for the video stream and see if that works? With details about the videos and protocols that you're trying to play, we can help guide on the best way to achieve a great experience.

[brabebhin]

@ZhengzhongSun You can try out the better optimized fork, where you can actually get some more support. This version offloads some of the video rendering part to the GPU. The performance of video decoding is greatly increased.

We also have some more features, like audio & video effects support and embedded subtitles.

https://github.com/lukasf/FFmpegInterop

[ZhengzhongSun]

@khouzam I didn't find the ForceDecode parameter

[ZhengzhongSun]

@khouzam @mcosmin222 Thank you very much! Since my demand is simple, I decide to use the hardware decoder for h264 directly. I find the ICodecAPI interface which can enable the hardware acceleration. https://msdn.microsoft.com/en-us/library/windows/desktop/dd742712(v=vs.85).aspx
But I'm not familiar about the decode part and there are no sample code about it. Any suggestion would be appreciated!

[brabebhin]

You do not need that interface. In the context of MediaStreamSource, you need

VideoEncodingProperties::CreateH264()

Then you read packages from the stream and send them to the OS for hardware acceleration.
In FFmpegInterop, if you set "videoDecode" to false, it will automatically pass through all the video packages undecoded. This will use directx APIs to do the decoding on the GPU.

[LoopIssuer]

Hi,
I have issues with building dll for Unity for Hololens. Could you please provide this file? Thanks in advance.

[RookieDeveloper-Wang]

Hi,
I have issues with building dll for Unity for Hololens. Could you please provide this file? Thanks in advance.

Hi,
Is your problem solved? I have encountered such a problem.