Keras-MXNet 2.2.4.1

Highlights

1. New Feature: Added support for sparse tensors.
2. Added sparse kernel for Dense layer
3. Added sparse support for dot, concat, sum, mean operators and Embedding layer.
4. Added sparse linear regression benchmark
5. Added documentation on sparse support with mxnet backend

API Change:

1. To use sparse kernel with Dense layer, following API change has been made:
   prediction = Dense(units=1, activation='linear', sparse_weight=True)(input)
   Setting flag sparse_weight=True will enable sparse kernel.
2. To use Embedding layer with sparse data, sparse_grad needs to be added.
   Embedding(max_features, 128, input_length=10, sparse_grad=True)

Performance

1. Added sparse benchmark scripts and results on training and inference using CPU & GPU
2. On higher batch sizes > 128, Keras-MXNet sparse training benchmarks show a significant increase in training speed on GPUs.
3. Keras-MXNet sparse training and inference benchmarks show significantly less amount of memory usage in GPUs.
4. See following documents below for more details:

• Sparse Training Benchmark Results
• Sparse Inference Benchmark Results

Unsupported Functionalities

<< Same as previous release v2.2.4 >>

1. Keras MXNet models does not support pickling #194
2. For using RNN with unrolling=True, Keras-MXNet does not support variable length inputs. You need to pad input sequences to make it static length and provide input length when you set unroll=True. See using RNN with Keras-MXNet tutorial for more details. This limitation does not exist when you set unroll=False (default option)
3. 14 Keras operators are not supported with MXNet backend. Update operators, symbolic gradient, localconv1d, localconv2d, higher order functions, other operator like cumsum, cumprod, stack, ctc and more. See Operators missing with MXNet backend Github Issue for more details.
4. Unsupported keras/examples list.
5. Cross backend models are not supported. Training with TensorFlow backend and loading the Keras model with MXNet backend is not supported.

Known Issues

<< Same as previous release v2.2.4 >>

1. MXNet backend performance significantly drops with channels_last image_data_format. It is highly recommended to use channels_first image_data_format. See performance guide for more details.
2. MXNet backend does not support boolean. For example, in``topk operator with MXNet backend uses 0/1 instead of boolean. Issue
3. depthwise_conv2d supports depth_multiplier=1 only. Issue
4. Models with Custom Loss are not serializable. Issue

Installation

# Install keras-mxnet
pip install keras-mxnet

# Install MXNet - CPU 
pip install mxnet-mkl --pre

# Install MXNet - GPU
pip install mxnet-cu90mkl --pre

Contributions

Thanks to all the contributors for their contributions in this release:
@kalyc, @roywei, @sandeep-krishnamurthy

Other Resources and Getting Started

1. Keras-MXNet installation guide - https://github.com/awslabs/keras-apache-mxnet/blob/master/docs/mxnet_backend/installation.md
2. Using multi-gpu distributed training - https://github.com/awslabs/keras-apache-mxnet/blob/master/docs/mxnet_backend/multi_gpu_training.md
3. Performance tuning guide for Keras with MXNet backend - https://github.com/awslabs/keras-apache-mxnet/blob/master/docs/mxnet_backend/performance_guide.md
4. Export MXNet Model from Keras-MXNet - https://github.com/awslabs/keras-apache-mxnet/blob/master/docs/mxnet_backend/save_mxnet_model.md
5. Checkpointing MXNet Model from Keras-MXNet - https://github.com/awslabs/keras-apache-mxnet/blob/master/docs/mxnet_backend/save_mxnet_model.md
6. Using RNN in Keras-MXNet. Limitations and workarounds - https://github.com/awslabs/keras-apache-mxnet/blob/master/docs/mxnet_backend/using_rnn_with_mxnet_backend.md
7. Use sparse data for training models in Keras-MXNet - https://github.com/awslabs/keras-apache-mxnet/blob/master/docs/mxnet_backend/using_sparse_data_with_mxnet_backend.md
8. Keras-MXNet benchmark results -https://github.com/awslabs/keras-apache-mxnet/tree/master/benchmark