This repository was archived by the owner on Jul 1, 2024. It is now read-only.
Keras-MXNet 2.2.4.1
Highlights
- New Feature: Added support for sparse tensors.
- Added sparse kernel for Dense layer
- Added sparse support for
dot
,concat
,sum
,mean
operators andEmbedding
layer. - Added sparse linear regression benchmark
- Added documentation on sparse support with mxnet backend
API Change:
- To use sparse kernel with Dense layer, following API change has been made:
prediction = Dense(units=1, activation='linear', sparse_weight=True)(input)
Setting flagsparse_weight=True
will enable sparse kernel. - To use Embedding layer with sparse data,
sparse_grad
needs to be added.
Embedding(max_features, 128, input_length=10, sparse_grad=True)
Performance
- Added sparse benchmark scripts and results on training and inference using CPU & GPU
- On higher batch sizes > 128, Keras-MXNet sparse training benchmarks show a significant increase in training speed on GPUs.
- Keras-MXNet sparse training and inference benchmarks show significantly less amount of memory usage in GPUs.
- See following documents below for more details:
Unsupported Functionalities
<< Same as previous release v2.2.4 >>
- Keras MXNet models does not support pickling #194
- For using RNN with unrolling=True, Keras-MXNet does not support
variable length inputs
. You need to pad input sequences to make it static length and provideinput
length
when you setunroll=True
. See using RNN with Keras-MXNet tutorial for more details. This limitation does not exist when you set unroll=False (default option) - 14 Keras operators are not supported with MXNet backend. Update operators, symbolic gradient, localconv1d, localconv2d, higher order functions, other operator like cumsum, cumprod, stack, ctc and more. See Operators missing with MXNet backend Github Issue for more details.
- Unsupported keras/examples list.
- Cross backend models are not supported. Training with TensorFlow backend and loading the Keras model with MXNet backend is not supported.
Known Issues
<< Same as previous release v2.2.4 >>
- MXNet backend performance significantly drops with
channels_last
image_data_format. It is highly recommended to usechannels_first
image_data_format. See performance guide for more details. - MXNet backend does not support
boolean
. For example,in``topk
operator with MXNet backend uses0/1
instead of boolean. Issue depthwise_conv2d
supportsdepth_multiplier=1
only. Issue- Models with Custom Loss are not serializable. Issue
Installation
# Install keras-mxnet
pip install keras-mxnet
# Install MXNet - CPU
pip install mxnet-mkl --pre
# Install MXNet - GPU
pip install mxnet-cu90mkl --pre
Contributions
Thanks to all the contributors for their contributions in this release:
@kalyc, @roywei, @sandeep-krishnamurthy
Other Resources and Getting Started
- Keras-MXNet installation guide - https://github.com/awslabs/keras-apache-mxnet/blob/master/docs/mxnet_backend/installation.md
- Using multi-gpu distributed training - https://github.com/awslabs/keras-apache-mxnet/blob/master/docs/mxnet_backend/multi_gpu_training.md
- Performance tuning guide for Keras with MXNet backend - https://github.com/awslabs/keras-apache-mxnet/blob/master/docs/mxnet_backend/performance_guide.md
- Export MXNet Model from Keras-MXNet - https://github.com/awslabs/keras-apache-mxnet/blob/master/docs/mxnet_backend/save_mxnet_model.md
- Checkpointing MXNet Model from Keras-MXNet - https://github.com/awslabs/keras-apache-mxnet/blob/master/docs/mxnet_backend/save_mxnet_model.md
- Using RNN in Keras-MXNet. Limitations and workarounds - https://github.com/awslabs/keras-apache-mxnet/blob/master/docs/mxnet_backend/using_rnn_with_mxnet_backend.md
- Use sparse data for training models in Keras-MXNet - https://github.com/awslabs/keras-apache-mxnet/blob/master/docs/mxnet_backend/using_sparse_data_with_mxnet_backend.md
- Keras-MXNet benchmark results -https://github.com/awslabs/keras-apache-mxnet/tree/master/benchmark