text classification in tensorflow

Implementing multi models for Text Classification in TensorFlow.

Contents

Data and Preprocess

Data

Models are used to perform sentiment analysis on movie reviews from the Large Movie Review Dataset, which contains 25,000 highly polar movie reviews for training, and 25,000 for testing.
In this task, given a movie review, the model attempts to predict whether it is positive or negative. This is a binary classification task.

Preprocess

1. Load positive and negative sentences from the raw data files.
2. Clean the text data.
3. Pad each sentence to the maximum sentence length.
4. Word vector mapping, Each sentence becomes a bag of word vectors.

Models

1. FastText

• text classification: Bag of Tricks for Efficient Text Classification, 2016.07

2. TextCNN

• Convolutional Neural Networks for Sentence Classification, 2014.08
• A Sensitivity Analysis of (and Practitioners' Guide to) Convolutional Neural Networks for Sentence Classification

3. BiLSTM

• Bidirectional Recurrent Neural Networks, 1997

4. TextRCNN

Learn more contextual information than conventional window-based neural networks.

• Recurrent Convolutional Neural Networks for Text Classification, 2015

5. Hierarchical Attention Networks

• Hierarchical Attention Networks for Document Classification, 2016

6. Seq2seq with Attention

• Neural Machine Translation by Jointly Learning to Align and Translate, 2014.09

Performance

epochs = 10
batch_size = 64
max_learning_rate = 0.001
decay_rate = 0.8
decay_steps = 2000
l2_reg_lambda = 1e-3
embedding_trainable = False

Models	fastText	TextCNN	BiLSTM	TextRCNN	HierAtteNet	Seq2seqAttn	EntityNet	DynamicMemory	Transformer
Accuracy	0.834304	0.878276	0.884974	0.840128	0.889314

References

• Understanding Convolutional Neural Networks for NLP
• Implementing a CNN for Text Classification in TensorFlow
• Github: cnn-text-classification-tf
• 基于 word2vec 和 CNN 的文本分类 ：综述 & 实践
• 自然语言处理中CNN模型几种常见的Max Pooling操作
• LSTM Networks for Sentiment Analysis
• Bag of Tricks for Efficient Text Classification
• Chatbots with Seq2Seq

License

This project is licensed under the terms of the MIT license.