progressive-learning is a package for exploring and using progressive learning algorithms developed by the neurodata group.
The natural process of biological learning involves progressive acquisition of new information developing on past knowledge and experiences, which often leads to a performance improvement on a given task. Learning a second language, for instance, is associated with higher performance in an individual’s native language compared to that of monolinguals. In classical machine learning, the process usually begins from the state of tabula rasa, zero knowledge, and is optimized for a single task. The issues arise when the system is sequentially optimized for multiple tasks exhibiting “catastrophic forgetting,” diminishing performance of previously learned tasks. One of the current limitations of artificial intelligence revolves around this inability to transfer knowledge.
The progressive learning package utilizes representation ensembling algorithms to sequentially learn a representation for each task and ensemble both old and new representations for all future decisions. Here, two complementary representation ensembling algorithms based on decision forests (Lifelong Forest) and deep networks (Lifelong Network) demonstrate forward and backward knowledge transfer of tasks on multiple real datasets, including both vision and language applications.
progressive-learning package requires only a standard computer with enough RAM to support the in-memory operations.
This package is supported for Linux and macOS. The package has been tested on the following systems:
- Linux: Ubuntu 16.04
- macOS: Mojave (10.14.1)
- Windows: 10
This package is written for Python3. Currently, it is supported for Python 3.6 and 3.7.
progressive-learning mainly depends on the Python scientific stack.
keras
tensorflow
numpy
scikit-learn
joblib
git clone https://github.com/neurodata/progressive-learning.git
cd progressive-learning
python3 setup.py install
We welcome contributions from anyone. Please see our contribution guidelines before making a pull request. Our issues page is full of places we could use help! If you have an idea for an improvement not listed there, please make an issue first so you can discuss with the developers.
This project is covered under the Apache 2.0 License.
We appreciate detailed bug reports and feature requests (though we appreciate pull requests even more!). Please visit our issues page if you have questions or ideas.
If you find progressive-learning useful in your work, please cite the package via the progressive-learning paper
Vogelstein JT, Helm HS, Mehta RD, Dey J, Yang W, Tower B, LeVine W, Larson J, White C, Priebe CE. A general approach to progressive learning. arXiv preprint arXiv:2004.12908. 2020 Apr 27.