(work in progress)
Python code for training, evaluating and running a light-weight machine learning based demosaicing model. The concept behind the model architecture is to assume a fixed bayer pattern, use PixelUnshuffle to transform the values of each bayer patch into 4 channels at the same position, thus halfing the width and height. The resulting tensor is the input to 3 Residual-in-Residual Dense Blocks (from the ESRGAN paper). A pixel shuffle operation upscales the image again followed by a convolution. This network should only predict the residual of the result obtained by bilinear interpolation.
As training data about 10.000 downscaled images of the the OpenImages V7 dataset are used.
Preliminary results are very promising and the network seems to beat existing demosaicing algorithms like AHD or VNG by a large margin in terms of PSNR and also in terms of artifacts when doing visual comparisons of the result. The datasets used for testing are the Kodak and McMaster dataset. This does come at the cost of longer computation times, although the model runs at a very reasonable performance (40ms per megapixel on a RTX4070Ti). I will add more information soon.