Intra Prediction Mode Algorithm
Libde265 currently (17/Sep/2014) implements three algorithms for intra-prediction mode selection:
- brute-force: test all 35 modes and compute exact bitrate. This should practically be the optimum possible.
- min-residual: compute a simple distortion measure on the residual image and select the mode with minimum distortion. See below for supported distortion measures.
- fast-brute: use a simple distortion measure to select the n-best modes with minimum distortion. Additionally, put the 3 candidate modes into the candidate list. For each candidate mode, compute the exact bitrate and select the best mode.
The following distortion measures are implemented in Libde265:
- SAD: sum of absolute differences
- SSD: sum of squared differences
- SATD-H: sum of absolute transformed differences (with Hadamard transform)
- SATD-D: sum of absolute transformed differences (with DCT/DST transform)
Libde265's FastBrute (with 5+3 candidates) is very close to the optimal brute-force computation with a 4x reduction in computation time (see below). SATD-H and SATD-D perform better than SAD/SSD, the differences between SATD-H and SATD-D are negligible, just as the differences between SAD/SSD. MinResidual is usually worst, again with SATD-H and SATD-D better than SAD and SSD.
The HM13 algorithm is not very good at low bitrates. In this range, it is almost as bad as SAD/SSD. In the high bitrate range, it is performing better, but still far from the FastBrute performance (even though at a lower computational cost, but these are hardly comparable at the moment). X265 also lies somewhere in the middle, but generally achieves a better quality than HM13, especially at lower bitrates. At very low bitrates, it drops off sharply, though.
Considering SSIM quality, it can be observed that the HM quality is even below Min-Residual. It should be evaluated in the future if this is the case for other sequences, too. X265 has very good SSIM at high bitrates, but again drops off sharply at low bitrates. For Fast-Brute the results are again very close to optimum over the whole bitrate range.
Computation times (x265 is not included here, but it is much faster because of its optimized routines). Brute-force is by far the most computational expensive and computation time increases with bitrate because the exact bitrate computation takes more time at higher rates. The Fast-Brute algorithm is about a factor 4 faster and computation time does not increase as much. Computation time of Min-Residual is almost constant, because there is no bitrate dependent computation, only the final encoding takes a bit more time.