custom-demux-1

Custom build that removes .MP4 creation and simplifies logic down to only .H264 extraction

3.0.6 - Timecode Metadata on MP4s

This release adds timecode metadata video-only and video-and-audio MP4s #32, thanks to @kris-sum

3.0.4 - Bugfix release

Fixes issue #15 where certain .ubv files coming from recent Unifi Protect firmwares may not have their framerate correctly probed

Support timelapse files

This release is a minor feature release to support the remuxing of timelapse files. These files previously did not work because their framerate was guessed at 0fps

See full installation and usage instructions at https://github.com/petergeneric/unifi-protect-remux/blob/master/README.md

Bugfix release

This release is a minor bugfix release to add support for .ubv files that have been renamed from Unifi Protect's original naming scheme

See full installation and usage instructions at https://github.com/petergeneric/unifi-protect-remux/blob/master/README.md

Rewrite in go

This release supports the same features as 2.0.0, however the tool is now written in Go and as a result the tool is faster and smaller.

This release supports audio, and if FFmpeg is available then it will create MP4 files. Currently it will create one MP4 for each partition of a multi-partition .ubv, naming the files based on the date+time the video starts.

Audio muxing should account for audio/video synchronisation, however this has not been extensively tested (I don't have any camera samples where AV sync is particularly obvious). If you're experiencing issues and can supply a .ubv for me to examine please raise an issue to get in touch.

QUICK START: FOR UBIQUITI HARDWARE

Instructions for Cloud Key Gen 2 Plus (and other Ubiquiti hardware). Due to the relatively slow CPU and IO subsystems, running a remux this way will be somewhat slow. See below for instructions on how to run under Linux x86 below.

1. Go to the "releases" page at https://github.com/petergeneric/unifi-protect-remux/releases and download the latest remux ARM64 binary (N.B. "ARM64", not "x86")
2. Upload this to your Cloud Key home folder using SSH (SCP) with tar -xf remux-arm64.tar.gz && rm remux-arm64.tar.gz && chmod +x remux
3. Download the latest FFmpeg ARM Static Release (available on the releases page, and also from https://johnvansickle.com/ffmpeg/)
4. Upload this to your Cloud Key and extract it with xz -d ffmpeg-release-arm64-static.tar.xz && tar -xf ffmpeg-release-arm64-static.tar && mv ffmpeg*arm64-static/ffmpeg ./ && rm ffmpeg-release-arm64-static.tar.xz && chmod +x ffmpeg
5. Run the following on your cloudkey: export PATH=$HOME:$PATH (you'll either need to run this every time you log in or put it in your .bashrc file)
6. Navigate to where your .ubv video is located (base path: /srv/unifi-protect/video).
7. Run: remux *.ubv
8. By default, only video is extracted. If you need to extract audio too, add "--with-audio=true" to your command

If FFmpeg is not installed (or if the command fails) the remux tool will leave the raw .aac and .h264 bitstream files; these can be combined with a variety of tools.

QUICK START: FOR x86 LINUX (UBUNTU 20.04 LTS)

Dependencies: ubnt_ubvinfo from CloudKey

N.B. If you have the (discontinued, unsupported) x86 Protect installation, you can use the native x86 ubnt_ubvinfo tool from it - just copy that file to /usr/bin/ubnt_ubvinfo. Otherwise, the following will run the ARM binary on your x86 machine at a slight performance penalty using QEMU.

1. On your x86 machine, install qemu-user with: apt install -y qemu-user gcc-aarch64-linux-gnu
2. Copy /usr/share/unifi-protect/app/node_modules/.bin/ubnt_ubvinfo from your CloudKey to /usr/bin/arm-ubnt_ubvinfo on your x86 machine
3. Run the following:

sudo tee /usr/bin/ubnt_ubvinfo <<EOF
#!/bin/sh
export QEMU_LD_PREFIX=/usr/aarch64-linux-gnu
exec qemu-aarch64 /usr/bin/arm-ubnt_ubvinfo $*
EOF
chmod +x /usr/bin/ubnt_ubvinfo

Dependencies: FFmpeg

To install FFmpeg, run:

apt install -y ffmpeg

Extracting video

Once the dependencies are installed, use the following instructions to get the unifi-protect-remux tool working:

1. Go to the "releases" page at https://github.com/petergeneric/unifi-protect-remux/releases and download the latest remux x86_64 binary
2. Upload this to your Linux server and extract with tar -zxf remux-x86_64.tar.gz
3. Transfer .ubv files from your CloudKey to your x86 server (on cloudkey, .ubv files are found under /srv/unifi-protect/video).
4. Run: remux *.ubv
5. By default, only video is extracted. If you need to extract audio too, add "--with-audio=true" to your command

If FFmpeg is not installed (or if the command fails) the remux tool will leave the raw .aac and .h264 bitstream files; these can be combined with a variety of tools.

Command-line arguments

Usage of remux:
  -with-audio
    	If true, extract audio
  -with-video
    	If true, extract video (default true)
  -mp4
    	If true, will create an MP4 as output (default true)
  -output-folder string
    	The path to output remuxed files to. "SRC-FOLDER" to put alongside .ubv files (default "./")
  -version
    	Display version and quit
  -force-rate int
    	If non-zero, adds a -r argument to FFmpeg invocations

NOTE ON x86 WITHOUT QEMU

The quickstart instructions above show how to run the AARCH64 ubnt_ubvinfo tool shipped with Unifi Protect on your x86 hardware. This relies on qemu-user. If this tool is not available on your x86 machine (and you don't have the native x86 version of ubnt_ubvinfo -- for copyright reasons I can't supply this tool) then you will need to run the ubnt_ubvinfo command on your Ubiquiti hardware, then transfer the .ubv files along with a cached output of ubnt_ubvinfo to your x86 machine for final extraction.

See the scripts folder in the repository: these scripts generate a summary .txt file that you can pull back alongside the .ubv and run remux on your x86 machine.

Known Issues

In order to create an MP4 the tool has to detect the framerate of the video; it works this out by looking at the time between the initial video frames. This should work unless the framerate is changed mid-video. In this case some of the video may appear to run faster or slower than realtime. This seems a pretty unusual case so I'm not planning special logic for it any time soon!

2.0.0: Extract Video and Audio into MP4

This release supports audio, and if FFmpeg is available then it will create MP4 files. Currently it will create one MP4 for each partition of a multi-partition .ubv, naming the files based on the date+time the video starts.

Audio muxing is new; it should account for audio/video synchronisation, however this has not been extensively tested (I don't have any camera samples where AV sync is particularly obvious). If you're experiencing issues and can supply a .ubv for me to examine please raise an issue and get in touch.

QUICK START: FOR UBIQUITI HARDWARE

Instructions for Cloud Key Gen 2 Plus (and other Ubiquiti hardware):

1. Go to the "releases" page at https://github.com/petergeneric/unifi-protect-remux/releases and download the latest remux-arm64 binary (N.B. "ARM64", not "AMD64")
2. Upload this to your Cloud Key home folder using SSH (SCP) and extract with tar -xf remux-arm64.tar.gz && rm remux-arm64.tar.gz && chmod +x remux-arm64
3. Download the latest FFmpeg ARM Static Release (available on the releases page, and also from https://johnvansickle.com/ffmpeg/)
4. Upload this to your Cloud Key and extract it with xz -d ffmpeg-release-arm64-static.tar.xz && tar -xf ffmpeg-release-arm64-static.tar && mv ffmpeg*arm64-static/ffmpeg ./ && rm ffmpeg-release-arm64-static.tar.xz && chmod +x ffmpeg
5. Run the following on your cloudkey: export PATH=$HOME:$PATH (you'll either need to run this every time you log in or put it in your .bashrc file)
6. Navigate to where your .ubv video is located (base path: /srv/unifi-protect/video).
7. Run: remux-amd64 *.ubv
8. By default, only video is extracted. If you need to extract audio too, add "--with-audio" to your command

If FFmpeg is not installed (or if the command fails) the remux tool will leave the raw .aac and .h264 bitstream files; these can be combined with a variety of tools.

NOTE ON AMD64 VERSION

If you have the old (unsupported) x86 version of Protect, extract the ubnt_ubvinfo tool from that (see the path referenced in the scripts/ folder of this repo) and put it in your PATH and you can use the AMD64 binary. This will be significantly faster than running on a Cloud Key. For copyright reasons I can't supply this tool.

If you don't have the x86 Protect installer, you can run some key tasks on your Ubiquiti gear (see the scripts folder). These scripts generate a summary .txt file that you can pull back alongside the .ubv and run remux-amd64 on your x86 machine.

Known Issues

In order to create an MP4 the tool has to detect the framerate of the video; it works this out by looking at the time between the initial video frames. This should work unless the framerate is changed mid-video. In this case some of the video may appear to run faster or slower than realtime. This seems a pretty unusual case so I'm not planning special logic for it any time soon!

v1.0.0

This initial release extracts data from multiple partitions, and names output files using the date+time of the first video frame.

Pre-compiled binaries are available for Linux ARM64 and AMD64.

N.B. if you have the x86 version of ubnt_ubvinfo you can execute it entirely on AMD64, but ordinarily it'd be recommended to run the ARM64 version on your CloudKey and then pull back the extracted media from CloudKey and run FFmpeg on your main machine to rewrap the .h264 files into .MP4

QUICK START FOR CLOUD KEY GEN2

1. Go to the "releases" page at https://github.com/petergeneric/unifi-protect-remux/releases and download the latest remux-arm64 binary.
2. Upload this to your Cloud Key, leaving in the home folder
3. Navigate to where your .ubv video is located (/srv/unifi-protect/video)
4. Run ~/remux-amd64 with a single argument, the .ubv file you want to extract from (e.g. ~/remux-amd64 2020/08/01/XXXXXXXXXXXX_0_rotating_1596300058863.ubv)
5. Once the tool completes, you'll find a series of .h264 files in the same directory as the .ubv input
6. Transfer these to your machine. The .h264 files are raw video bitstreams. It will be easier if you remux them into an .MP4 wrapper using FFmpeg
7. Install FFmpeg for your platform (https://ffmpeg.zeranoe.com/builds/ for Windows, apt install ffmpeg for Linux, brew install ffmpeg for MacOS using HomeBrew or https://evermeet.cx/ffmpeg/ otherwise)
8. Use the following command to remux each .h264 into an MP4: ffmpeg -i example.h264 -vcodec copy example.mp4

QUICK START NOTE FOR AMD64 VERSION

The AMD64 version is designed for use with an AMD64 Unifi Protect installation (or at least, the "ubnt_ubvinfo" tool, extracted from such an installation; for copyright reasons I can't supply that tool). It won't work on a stock Linux install without that additional tool. You CAN, however, run some key tasks on your Ubiquiti gear (see scripts/prepare.sh) and then run the remux itself on your AMD64 box (scripts/remux.sh). This script also runs ffmpeg to generate a .mp4

beta1

This beta release extracts data from multiple partitions, and names output files using the date+time of the first video frame.

A pre-compiled Linux AMD64 binary is available for download as part of this release. This can be used with the x86 version of ubnt_ubvinfo if you have access to it, or you can use the README.txt instructions to generate the ubnt_ubvinfo output on your CloudKey and then run the remux program on an AMD64 machine.

I'd like to get a native arm64 version so that everything can just be run on a cloudkey, but it doesn't look to be possible at the minute