Merge pull request #33 from BGIResearch/dev

Dev

.gitignore

@@ -6,6 +6,7 @@ __pycache__/
 
 # C extensions
 *.so
+*cy.cpp
 
 # Distribution / packaging
 .Python
@@ -128,3 +129,6 @@ dmypy.json
 
 # Pyre type checker
 .pyre/
+
+h5config.json
+test_data/

docs/source/General/Installation.rst

@@ -1,16 +1,51 @@
 Installation
 ============
 
-install from pypi
+Pre-built installation (recommended)
+------------------------------------
 
 .. code:: bash
 
     pip install stereopy
 
-or install from github
+Source installation
+--------------------------------------------
+
+To install stereopy from source, you need:
+
+- HDF5 1.8.4 or newer with development headers
+- A C compiler
+
+On Unix platforms, you also need pkg-config unless you explicitly specify a path for HDF5 as described below.
+
+You can specify build options for stereopy as environment variables when you build it from source:
 
 .. code:: bash
 
     git clone https://github.com/BGIResearch/stereopy.git
     cd stereopy
-    python setup.py install
+    HDF5_DIR=/path/to/hdf5 python setup.py install
+
+The supported build options are:
+
+ - To specify where to find HDF5, use one of these options:
+    - HDF5_LIBDIR and HDF5_INCLUDEDIR: the directory containing the compiled HDF5 libraries and the directory containing the C header files, respectively.
+    - HDF5_DIR: a shortcut for common installations, a directory with lib and include subdirectories containing compiled libraries and C headers.
+    - HDF5_PKGCONFIG_NAME: A name to query pkg-config for. If none of these options are specified, stereopy will query pkg-config by default for hdf5, or hdf5-openmpi if building with MPI support.
+
+Source installation on OSX/MacOS
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+HDF5 and Python are most likely in your package manager (e.g. Homebrew, Macports, or Fink). Be sure to install the development headers, as sometimes they are not included in the main package.
+
+XCode comes with a C compiler (clang), and your package manager will likely have other C compilers for you to install.
+
+Source installation on Linux/Other Unix
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+HDF5 and Python are most likely in your package manager. A C compiler almost definitely is, usually there is some kind of metapackage to install the default build tools, e.g. build-essential, which should be sufficient for our needs. Make sure that that you have the development headers, as they are usually not installed by default. They can usually be found in python-dev or similar and libhdf5-dev or similar.
+
+Source installation on Windows
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Installing from source on Windows is a much more difficult prospect than installing from source on other OSs, as not only are you likely to need to compile HDF5 from source, everything must be built with the correct version of Visual Studio. Additional patches are also needed to HDF5 to get HDF5 and Python to work together.

docs/source/contributing.rst

@@ -0,0 +1,135 @@
+Contributing guide
+~~~~~~~~~~~~~~~~~~
+
+Contents
+========
+- `Contributing code`_
+- `Code style`_
+- `Project structure`_
+- `Writing documentation`_
+- `Making a release`_
+- `Testing`_
+
+Contributing code
+-----------------
+1. Clone the Stereopy source from github::
+
+    git clone https://github.com/BGIResearch/stereopy.git
+2. Create a new branch for your PR and checkout to the new branch::
+
+    git checkout -b new_branch_name
+
+3. Add the new feature or fit the bugs in your codebase.
+4. After all tests passing, update the related documentation, such as release note, example and so on.
+5. Make a Pull Requests back to the dev branch, We will review the submitted code, and the merge to the main branch after there is no problem.
+
+Code style
+----------
+1. Coding requirements comply with pep8 specification.
+2. The file naming adopts lowercase and underscore uniformly; the name of the class adopts the camel case naming method.
+
+    eg: file name: dim_reduce.py; class name: DimReduce
+
+3. Variable naming should be in lower case and should be as meaningful as possible, avoiding unintentional naming.
+4. The comment information should be perfect, and each file, function, and class should write its comment information.
+   We recommend using restructured Text as the docstring format of comment information.
+
+Project structure
+-----------------
+The stereopy project:
+
+- `stereo <stereo>`_: the root of the package.
+
+  - `stereo/core <stereo/core>`_: the core code of stereo, which contains the base class and data structure of stereopy.
+  - `stereo/algorithm <stereo/algorithm>`_: the algorithm module, main analysis and implementation algorithms, which
+    deals with methodology realization.
+  - `stereo/image <stereo/image>`_: the image module, which deals with the tissue image related analysis, such as cell
+    segmentation, etc.
+  - `stereo/io <stereo/io>`_: the io module, which deals with the reading, writing and format conversion of different
+    data structure, between our StereoExpData and AnnData, etc.
+  - `stereo/plots <stereo/plots>`_: the plotting module, which contains all the plotting functions for visualization.
+  - `stereo/utils <stereo/utils>`_: Some common processing scripts.
+  - `stereo/tests <stereo/tests>`_: the Tests module, which contains all the test scripts.
+
+Writing documentation
+---------------------
+We use Sphinx to auto-generate documentation in multiple formats. Sphinx is built of reStructured text and, when using
+sphinx most of what you type is reStructured text.
+
+Installing Sphinx::
+
+    pip install sphinx
+
+    pio install sphinx_rtd_theme
+
+    pip install recommonmark   # surpport md
+
+Make a doc directory, and start sphinx::
+
+    cd stereo
+
+    mkdir docs
+
+    # start sphinx
+    sphinx-quickstart
+
+Modify the config file:
+    In doc/source directory is now a python file called conf.py. This is the file that controls the basics of how 
+    sphinx runs when you run a build. For more information about the file, see the official 
+    `Sphinx <https://www.sphinx-doc.org/en/master/usage/configuration.html>`_ document.
+
+Write the Tutorials/example what you need to update:
+    Tutorials and examples are hosted on a separate repository called 
+    `Tutorials <https://github.com/BGIResearch/stereopy/tree/main/docs/source/Tutorials>`_.
+
+Make and build html::
+
+    make clean
+
+    make html
+
+    open index.html
+
+docstrings format
+`````````````````
+We use the reStructured text style for writing docstrings. If you’re unfamiliar with the reStructuredText (rst) format,
+see the `link <https://www.sphinx-doc.org/en/master/usage/restructuredtext/basics.html>`_.
+
+Making a release
+----------------
+This part is to tell developers how to publish stereopy to PyPi.
+
+Checking the environment::
+
+    # First, install twine
+    pip install twine
+
+    # make a build
+    python setup.py sdist bdist_wheel
+
+    # check the build
+    twine check dist/*
+
+Making release::
+
+    # Tag the version info
+    git tag {version}
+
+    # Build distributions and wheel
+    python setup.py sdist bdist_wheel
+
+    # Check whether the compilation result can be installed successfully
+    # eg: pip install dist/stereopy-{version}-py3-none-any.whl
+
+    # push the tag to github
+    git push origin {version}
+
+    # Upload wheel and code distribution to PyPi
+    twine upload dist/*
+
+After any release has been made, create a new release notes file for the next feature and bugfix release.
+
+Testing
+-------
+For each functional module, a corresponding test script should be created to ensure that the developed function
+is normal. All our test files are unified in the `tests <stereo/tests>`_ directory.

docs/source/release_note.rst

@@ -3,6 +3,15 @@ Release Notes
 
 .. role:: small
 
+Version 0.2.2
+-----------
+0.2.2 :2021-11-17
+~~~~~~~~~~~~~~~~~~~~~~~
+1. Optimize the performance of find marker.
+2. Add Cython setup_build function and optimize gef io performance.
+3. Add hotspot pipeline for spatial data and squidpy for spatial_neighbor func.
+4. Add polygon selection for interactive scatter plot and simplify the visualization part of the code.
+
 
 Version 0.2.1
 -----------

requirements.txt

@@ -1,20 +1,22 @@
 bokeh==2.3.3
+imageio==2.9.0
 numpy>=1.20.3
 matplotlib>=3.4.2
 pandas>=1.3.3
 scipy>=1.7.1
 seaborn~=0.11.1
 h5py>=3.2.1
+gefpy>=0.1.1
 tqdm>=4.62.1
 setuptools>=41.0.0
 scikit-learn>=0.24.2
 numba~=0.53.1
 statsmodels~=0.12.1
 leidenalg>=0.8.7
-umap-learn>=0.5.1
+umap-learn==0.5.1
 KDEpy>=1.1.0
 Jinja2
-tifffile
+tifffile~=2021.7.2
 shapely~=1.7.1
 packaging~=20.9
 glog==0.3.1
@@ -23,7 +25,7 @@ opencv-contrib-python
 opencv-python-headless
 Pillow==8.2.0
 slideio~=0.5.225
-panel==0.12.0
+panel==0.12.1
 holoviews==1.14.5
 param==1.11.1
 hvplot==0.7.3
@@ -33,10 +35,14 @@ datashader>=0.13.0
 anndata~=0.7.6
 phenograph==1.5.7
 scikit-misc>=0.1.4
-requests
+requests~=2.26.0
 urllib3
 typing_extensions~=3.7.4
 natsort~=7.1.1
 joblib~=1.0.1
 patsy~=0.5.1
 louvain>=0.7.0
+igraph~=0.9.8
+arboreto~=0.1.6
+squidpy
+xarray>=0.20.1

setup.py

@@ -15,15 +15,15 @@
 
 setup(
     name='stereopy',
-    version='0.2.1',
+    version='0.2.2',
     setup_requires=['setuptools_scm', 'numpy', 'panel', 'pytest', 'quilt3', 'scipy', 'phenograph'],
     description='Spatial transcriptomic analysis in python.',
     long_description=Path('README.md').read_text('utf-8'),
     long_description_content_type="text/markdown",
     url='https://github.com/BGIResearch/stereopy',
     author='BGIResearch',
     author_email='[email protected]',
-    python_requires='>=3.6',
+    python_requires='>=3.7',
     install_requires=[
         l.strip() for l in Path('requirements.txt').read_text('utf-8').splitlines()
     ],
@@ -45,3 +45,12 @@
         'Topic :: Scientific/Engineering :: Visualization',
     ],
 )
+
+try:
+    import os
+    print('install hotspot from git, command: pip install git+https://github.com/yoseflab/Hotspot.git')
+    os.system('pip install git+https://github.com/yoseflab/Hotspot.git')
+    # os.system('pip install pyscenic~=0.11.2 --ignore-installed pyarrow')
+except Exception as e:
+    print('Hotspot is not install successly, please check.')
+    print(e)

stereo/algorithm/dim_reduce.py

@@ -40,15 +40,16 @@ def factor_analysis(x, n_pcs):
     return tran_x
 
 
-def pca(x, n_pcs):
+def pca(x, n_pcs, random_state=0):
     """
     Principal component analysis.
 
     :param x: 2D array, shape (M, N)
     :param n_pcs: the number of features for a return array after reducing.
+    :param random_state : int, RandomState instance
     :return:  ndarray of shape (n_samples, n_components) Embedding of the training data in low-dimensional space.
     """
-    pca_obj = PCA(n_components=n_pcs)
+    pca_obj = PCA(n_components=n_pcs, random_state=random_state)
     x_pca = pca_obj.fit_transform(x)
     variance = pca_obj.explained_variance_
     variance_ratio = pca_obj.explained_variance_ratio_