Skip to content

Commit

Permalink
Merge pull request #120 from NVIDIA/develop
Browse files Browse the repository at this point in the history
[REVIEW] gquant rename to greenflow
  • Loading branch information
avolkov1 authored Feb 22, 2021
2 parents abf9d33 + 26be17b commit 231b5ce
Show file tree
Hide file tree
Showing 333 changed files with 1,505 additions and 1,427 deletions.
17 changes: 14 additions & 3 deletions gQuant/CHANGELOG.md → CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,11 +1,22 @@
# Changelog

## [v1.0.2](https://github.com/NVIDIA/fsi-samples/tree/v1.0.2) (2021-02-11)
## [v1.0.3](https://github.com/NVIDIA/fsi-samples/tree/v1.0.3) (2021-02-13)

[Full Changelog](https://github.com/NVIDIA/fsi-samples/compare/v1.0.2...v1.0.3)

**Merged pull requests:**

- \[REVIEW\]Change "gquant" to "greenflow" [\#119](https://github.com/NVIDIA/fsi-samples/pull/119) ([yidong72](https://github.com/yidong72))

## [v1.0.2](https://github.com/NVIDIA/fsi-samples/tree/v1.0.2) (2021-02-12)

[Full Changelog](https://github.com/NVIDIA/fsi-samples/compare/v1.0.1...v1.0.2)

**Merged pull requests:**

- Merge pull request \#116 from rapidsai/develop [\#118](https://github.com/NVIDIA/fsi-samples/pull/118) ([avolkov1](https://github.com/avolkov1))
- \[REVIEW\]gQuant 1.0.2 [\#117](https://github.com/NVIDIA/fsi-samples/pull/117) ([yidong72](https://github.com/yidong72))
- \[REVIEW\] update the readme file and version number [\#116](https://github.com/NVIDIA/fsi-samples/pull/116) ([yidong72](https://github.com/yidong72))
- \[REVIEW\]Upgrade the plugin to JupyterLab 3 [\#114](https://github.com/NVIDIA/fsi-samples/pull/114) ([yidong72](https://github.com/yidong72))

## [v1.0.1](https://github.com/NVIDIA/fsi-samples/tree/v1.0.1) (2021-01-20)
Expand Down Expand Up @@ -118,6 +129,7 @@
- \[REVIEW\]upgrade to RAPIDS 0.9 [\#57](https://github.com/NVIDIA/fsi-samples/pull/57) ([yidong72](https://github.com/yidong72))
- \[REVIEW\] change the text for notebook 05 [\#55](https://github.com/NVIDIA/fsi-samples/pull/55) ([yidong72](https://github.com/yidong72))
- Fix \#50b - Rename notebook folder to notebooks [\#52](https://github.com/NVIDIA/fsi-samples/pull/52) ([miguelusque](https://github.com/miguelusque))
- Fix \#50 - Rename notebook folder to notebooks [\#51](https://github.com/NVIDIA/fsi-samples/pull/51) ([miguelusque](https://github.com/miguelusque))

## [v0.2](https://github.com/NVIDIA/fsi-samples/tree/v0.2) (2019-08-16)

Expand All @@ -138,7 +150,6 @@

**Merged pull requests:**

- Fix \#50 - Rename notebook folder to notebooks [\#51](https://github.com/NVIDIA/fsi-samples/pull/51) ([miguelusque](https://github.com/miguelusque))
- Fix \#17 - cuIndicator notebook plot widget is too complicated \(WIP\) [\#45](https://github.com/NVIDIA/fsi-samples/pull/45) ([miguelusque](https://github.com/miguelusque))
- Fix \#39 - Refactor 04\_portfolio\_trade.ipynb notebook [\#44](https://github.com/NVIDIA/fsi-samples/pull/44) ([miguelusque](https://github.com/miguelusque))
- Merge develop to master [\#43](https://github.com/NVIDIA/fsi-samples/pull/43) ([yidong72](https://github.com/yidong72))
Expand All @@ -156,6 +167,7 @@
- update the build.sh [\#15](https://github.com/NVIDIA/fsi-samples/pull/15) ([yidong72](https://github.com/yidong72))
- Feature xgb notebook [\#11](https://github.com/NVIDIA/fsi-samples/pull/11) ([yidong72](https://github.com/yidong72))
- CUQ-5: Mortgage example using gQuant. [\#10](https://github.com/NVIDIA/fsi-samples/pull/10) ([avolkov1](https://github.com/avolkov1))
- CUQ-5: Mortgage example using gQuant. [\#9](https://github.com/NVIDIA/fsi-samples/pull/9) ([avolkov1](https://github.com/avolkov1))
- Feature indicator node [\#8](https://github.com/NVIDIA/fsi-samples/pull/8) ([yidong72](https://github.com/yidong72))
- Feature mulit assets indicator [\#7](https://github.com/NVIDIA/fsi-samples/pull/7) ([yidong72](https://github.com/yidong72))
- Update build.sh [\#6](https://github.com/NVIDIA/fsi-samples/pull/6) ([phogan-nvidia](https://github.com/phogan-nvidia))
Expand All @@ -180,7 +192,6 @@

- Revert "gQuant34 - Update build.sh to make use of RAPIDS v0.8 container" [\#33](https://github.com/NVIDIA/fsi-samples/pull/33) ([yidong72](https://github.com/yidong72))
- gQuant34 - Update build.sh to make use of RAPIDS v0.8 container [\#12](https://github.com/NVIDIA/fsi-samples/pull/12) ([miguelusque](https://github.com/miguelusque))
- CUQ-5: Mortgage example using gQuant. [\#9](https://github.com/NVIDIA/fsi-samples/pull/9) ([avolkov1](https://github.com/avolkov1))
- Synch master with develop [\#4](https://github.com/NVIDIA/fsi-samples/pull/4) ([avolkov1](https://github.com/avolkov1))
- added unit tests for the cuindicator [\#3](https://github.com/NVIDIA/fsi-samples/pull/3) ([yidong72](https://github.com/yidong72))
- CUQ-21: Improving tutorials for gQuant [\#2](https://github.com/NVIDIA/fsi-samples/pull/2) ([avolkov1](https://github.com/avolkov1))
Expand Down
4 changes: 3 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,5 +2,7 @@

Repo Index

1. [gGuant](gQuant) - A graph computation toolkit that helps you to organize the workflows in graph computation.
1. [greenflow](greenflow) - A graph computation toolkit that helps you to organize the workflows in graph computation.
2. [greenflowlab](greenflowlab) - A JupyterLab plugin that provides the UI interface for `greenflow`.
3. [gGuant](gQuant) - An external plugin with a set of nodes for quantitative analyst tasks, built on top of the [RAPIDS AI](https://rapids.ai/) project, [Numba](https://numba.pydata.org/), and [Dask](https://dask.org/).

4 changes: 2 additions & 2 deletions gQuant/.gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,8 @@ __pycache__
dask-worker-space/
dist/
build/
gquant.egg-info/
greenflow.egg-info/
*.swp
docker/Dockerfile.Rapids
docker/gQuant
docker/greenflow
node_modules
56 changes: 25 additions & 31 deletions gQuant/README.md
Original file line number Diff line number Diff line change
@@ -1,77 +1,71 @@
# gQuant - Graph Computation Tool

**NOTE:** For the latest stable [README.md](https://github.com/rapidsai/gquant/blob/main/README.md) ensure you are on the `main` branch.
# Greenflow Plugins for FSI Examples


## What's Inside This Repo

There are a few projects inside this repo:

1. [gquant](gquant) - A graph computation toolkit that helps you to organize the workflows in graph computation.
2. [gquantlab](gquantlab) - A JupyterLab plugin that provides the UI interface for `gquant`.
3. [plugins](plugins) - A few gquant plugins with example notebooks.
1. [simple_example](plugins/simple_example) - A simple external plugin example for gQuant.
2. [rapids_plugin](plugins/rapids_plugin) - An external plugin with a set of nodes for quantitative analyst tasks, built on top of the [RAPIDS AI](https://rapids.ai/) project, [Numba](https://numba.pydata.org/), and [Dask](https://dask.org/).
. [plugins](plugins) - A few greenflow plugins with example notebooks.
1. [simple_example](plugins/simple_example) - A simple external plugin example for greenflow.
2. [gquant_plugin](plugins/gQuant_plugin) - An external plugin with a set of nodes for quantitative analyst tasks, built on top of the [RAPIDS AI](https://rapids.ai/) project, [Numba](https://numba.pydata.org/), and [Dask](https://dask.org/).
3. [nemo_plugin](plugins/nemo_plugin) - An external plugin with a set of nodes that wraps the [NeMo library](https://github.com/NVIDIA/NeMo) .

These projects are all released as independent Python projects with their own `setup.py` files.

## Screenshots
![Tuturial](tutorial.gif "Tutorial")
![Quick Demo](gquantlab_demo.gif "Demo")
![Quick Demo](greenflowlab_demo.gif "Demo")


## Binary installation

### Install the gGuant
To install the gQuant graph computation library, run:
To install the greenflow graph computation library, run:
```bash
pip install gquant
pip install greenflow
```
Or install `gquant` at the gQuant directory:
Or install `greenflow` at the greenflow directory:
```bash
pip install .
```

### Install the gQuantLab JupyterLab plugin
To install `gquantlab` JupyterLab plugin, make sure `nodejs` of version [12^14^15] is installed. E.g.:
### Install the greenflowLab JupyterLab plugin
To install `greenflowlab` JupyterLab plugin, make sure `nodejs` of version [12^14^15] is installed. E.g.:
```bash
conda install -c conda-forge nodejs=12.4.0
```
Then install the `gquantlab`:
Then install the `greenflowlab`:
```bash
pip install gquantlab
pip install greenflowlab
```
Or install `gquantlab` at the gquantlab directory:
Or install `greenflowlab` at the greenflowlab directory:
```bash
pip install .
```

### Install the gQuant plugins
### Install the greenflow plugins

Under the plugin root directory, install the plugin as normal python packages.
```bash
pip install .
```

Note, gQuant node plugins can be registered in two ways:
Note, greenflow node plugins can be registered in two ways:

1. (Recommended)Write a external plugin using 'entry point' to register it. Check the `plugins` directory for details
2. Register the plugin in `gquantrc` file. Check the `System environment` for details
2. Register the plugin in `greenflowrc` file. Check the `System environment` for details


## Docker Install

- Build and run the container:

```bash
$ cd gQuant/docker && . build.sh
$ cd greenflow/docker && . build.sh
```
When building the container, you can run gQuant in two modes: dev or prod. In the dev mode, please check the README file in `gquantlab` directory to install the plugins and Python libraries.
When building the container, you can run greenflow in two modes: dev or prod. In the dev mode, please check the README file in `greenflowlab` directory to install the plugins and Python libraries.

In the production mode, you can launch the container by following command and start to use it
```bash
$ docker run --runtime=nvidia --rm -it -p 8888:8888 -p 8787:8787 -p 8786:8786 gquant/gquant:[tag from the build]
$ docker run --runtime=nvidia --rm -it -p 8888:8888 -p 8787:8787 -p 8786:8786 greenflow/greenflow:[tag from the build]
```

## Example notebooks
Expand All @@ -83,18 +77,18 @@ Example notebooks, tutorial showcasing, can be found in __notebooks__ folder in

There are a few system environment that the user can overwrite.

The custom module files are specified in the `gquantrc` file. `GQUANT_CONFIG` enviroment variable points to the location of this file. By default, it points to
`$CWD\gquantrc`.
The custom module files are specified in the `greenflowrc` file. `GREENFLOW_CONFIG` enviroment variable points to the location of this file. By default, it points to
`$CWD\greenflowrc`.

In the example `gquantrc`, system environment variable `MODULEPATH` is used to point to the paths of the module files.
In the example `greenflowrc`, system environment variable `MODULEPATH` is used to point to the paths of the module files.
To start the jupyterlab, please make sure `MODULEPATH` is set properly.

For example, if you want to start the jupyterlab in the gQuant root directory.
For example, if you want to start the jupyterlab in the greenflow root directory.
```bash
MODULEPATH=$PWD/modules jupyter-lab --allow-root --ip=0.0.0.0 --no-browser --NotebookApp.token=''
```

Or, if you want to start the jupyterlab in the gquantlab directory.
Or, if you want to start the jupyterlab in the greenflowlab directory.
```bash
GQUANT_CONFIG=../gquantrc MODULEPATH=$PWD/../modules jupyter-lab --allow-root --ip=0.0.0.0 --no-browser --NotebookApp.token=''
GREENFLOW_CONFIG=../greenflowrc MODULEPATH=$PWD/../modules jupyter-lab --allow-root --ip=0.0.0.0 --no-browser --NotebookApp.token=''
```
2 changes: 1 addition & 1 deletion gQuant/ci/checks/changelog.sh
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
#!/bin/bash
# Copyright (c) 2020, NVIDIA CORPORATION.
###########################
# gQuant CHANGELOG Tester #
# greenflow CHANGELOG Tester #
###########################

# Checkout main for comparison
Expand Down
2 changes: 1 addition & 1 deletion gQuant/ci/checks/style.sh
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
#!/bin/bash
# Copyright (c) 2020, NVIDIA CORPORATION.
################################################################################
# gQuant Style Tester
# greenflow Style Tester
################################################################################

# Ignore errors and set path
Expand Down
4 changes: 2 additions & 2 deletions gQuant/ci/cpu/build.sh
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
#!/usr/bin/env bash
# Copyright (c) 2020, NVIDIA CORPORATION.
################################################################################
# gQuant cpu build
# greenflow cpu build
################################################################################
set -e

Expand Down Expand Up @@ -55,7 +55,7 @@ conda config --set ssl_verify False
# BUILD - Conda package build
################################################################################

gpuci_conda_retry build conda/recipes/gquant --python=${PYTHON}
gpuci_conda_retry build conda/recipes/greenflow --python=${PYTHON}

################################################################################
# UPLOAD - Conda package
Expand Down
6 changes: 3 additions & 3 deletions gQuant/ci/cpu/upload.sh
Original file line number Diff line number Diff line change
Expand Up @@ -41,15 +41,15 @@ fi
################################################################################

gpuci_logger "Get conda file output locations"
export GQAUNT_FILE=`conda build conda/recipes/gquant --python=$PYTHON --output`
export GQAUNT_FILE=`conda build conda/recipes/greenflow --python=$PYTHON --output`

################################################################################
# UPLOAD - Conda packages
################################################################################

if [ "$UPLOAD_GQUANT" == "1" ]; then
if [ "$UPLOAD_GREENFLOW" == "1" ]; then
test -e ${GQAUNT_FILE}
echo "Upload gquant"
echo "Upload greenflow"
echo ${GQAUNT_FILE}
gpuci_retry anaconda -t ${MY_UPLOAD_KEY} upload -u ${CONDA_USERNAME:-rapidsai} ${LABEL_OPTION} --force ${GQAUNT_FILE}
fi
12 changes: 6 additions & 6 deletions gQuant/ci/gpu/build.sh
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
#!/bin/bash
# Copyright (c) 2020, NVIDIA CORPORATION.
###########################################
# gQuant GPU build and test script for CI #
# greenflow GPU build and test script for CI #
###########################################
set -e
NUMARGS=$#
Expand Down Expand Up @@ -56,22 +56,22 @@ conda config --show-sources
conda list --show-channel-urls

################################################################################
# BUILD - Build gQuant
# BUILD - Build greenflow
################################################################################

gpuci_logger "Build gQuant"
gpuci_logger "Build greenflow"
cd $WORKSPACE
python -m pip install -e .


################################################################################
# TEST - Run py.tests for gQuant
# TEST - Run py.tests for greenflow
################################################################################

if hasArg --skip-tests; then
gpuci_logger "Skipping Tests"
else
gpuci_logger "Python py.test for gQuant"
gpuci_logger "Python py.test for greenflow"
cd $WORKSPACE
py.test -vs --cache-clear --junitxml=${WORKSPACE}/junit-gquant.xml --cov-config=.coveragerc --cov=gquant --cov-report=xml:${WORKSPACE}/gquant-coverage.xml --cov-report term tests/
py.test -vs --cache-clear --junitxml=${WORKSPACE}/junit-greenflow.xml --cov-config=.coveragerc --cov=greenflow --cov-report=xml:${WORKSPACE}/greenflow-coverage.xml --cov-report term tests/
fi
3 changes: 0 additions & 3 deletions gQuant/conda/recipes/gquant/build.sh

This file was deleted.

46 changes: 0 additions & 46 deletions gQuant/conda/recipes/gquant/meta.yaml

This file was deleted.

Loading

0 comments on commit 231b5ce

Please sign in to comment.