Meson Build + Proper Windows Support (#300)

Co-authored-by: Jack Myers <[email protected]>
Co-authored-by: Neil Wu <[email protected]>
Co-authored-by: Neil Wu <[email protected]>
Co-authored-by: Sabet Seraj <[email protected]>

.github/build_real.sh

@@ -7,4 +7,4 @@ if [[ $IMAGE == "private" ]]; then
     cp -r $HOME/SNOPT/* pyoptsparse/pySNOPT/source
 fi
 
-pip install .[optview,testing]
+pip install .[optview,testing] -v

.github/environment.yml

@@ -0,0 +1,18 @@
+dependencies:
+# build
+  - python >=3.8
+  - numpy >=1.16
+  - ipopt
+  - swig
+  - meson >=0.60
+  - compilers
+  - pkg-config
+  - pip
+  - setuptools
+  - build
+# testing
+  - parameterized
+  - testflo
+  - scipy >1.2
+  - mdolab-baseclasses >=1.3.1
+  - sqlitedict >=1.6

.github/windows.yaml

@@ -0,0 +1,41 @@
+trigger:
+  - main
+
+pr:
+  - main
+
+pool:
+  vmImage: windows-latest
+
+jobs:
+  - job: Windows
+    steps:
+    - powershell: Write-Host "##vso[task.prependpath]$env:CONDA\Scripts"
+      displayName: Add conda to PATH
+
+    - script: conda config --add channels conda-forge && conda config --set channel_priority strict
+      displayName: Set channel priority
+
+    - script: conda create --yes --name pyos-build
+      displayName: Create environment
+
+    - script: |
+        call activate pyos-build
+        call conda install -y mamba
+        call mamba env update --file .github/environment.yml
+        call mamba install -y libpgmath
+      displayName: Install mamba and update environment
+
+    - script: |
+        set IPOPT_DIR=%CONDA_PREFIX%\Library
+        set CC=cl
+        set FC=flang
+        set CC_LD=link
+        python -m build -n -x .
+        pip install --no-deps --no-index --find-links dist pyoptsparse
+      displayName: Build and install pyoptsparse
+
+    - script: |
+        cd tests
+        testflo -n 1 .
+      displayName: Run tests

.github/workflows/windows-build.yml

@@ -0,0 +1,45 @@
+name: PyOptSparse Windows Actions
+
+on:
+  push:
+    branches:
+      - main
+  pull_request:
+    branches:
+      - main
+
+jobs:
+  build-windows:
+    runs-on: windows-latest
+
+    steps:
+      - uses: actions/checkout@v2
+      - uses: conda-incubator/setup-miniconda@v2
+        with:
+          python-version: 3.8
+          mamba-version: "*"
+          channels: conda-forge,defaults
+          channel-priority: strict
+          activate-environment: pyos-build
+          environment-file: .github/environment.yml
+      - name: Install libpgmath
+        shell: bash -l {0}
+        run: |
+          conda activate pyos-build
+          mamba install libpgmath
+      - name: Build and install pyoptsparse
+        shell: cmd /C CALL {0}
+        run: |
+          call conda activate pyos-build
+          set IPOPT_DIR=%CONDA_PREFIX%\Library
+          set CC=cl
+          set FC=flang
+          set CC_LD=link
+          python -m build -n -x .
+          pip install --no-deps --no-index --find-links dist pyoptsparse
+      - name: Run tests
+        shell: bash -l {0}
+        run: |
+          conda activate pyos-build
+          cd tests
+          testflo --pre_announce -v -n 1 .

.gitignore

@@ -13,3 +13,14 @@ env
 pyoptsparse/pySNOPT/source
 pyoptsparse/pyNLPQLP/source
 *.egg-info
+
+.idea/
+/meson_build/
+/dist/
+/staging_dir/
+
+*.lib
+
+*.pdb
+
+*.pyd

doc/advancedFeatures.rst

@@ -6,9 +6,21 @@ Advanced Features
 .. Parallel Execution
 .. ------------------
 
+MPI handling
+------------
+pyOptSparse can optionally run in parallel if a suitable ``mpi4py`` installation exists.
+This will be automatically detected and imported at run-time.
+
+If you only want to run in parallel, you can force pyOptSparse to do so by setting the environment variable
+``PYOPTSPARSE_REQUIRE_MPI`` to any one of these values: ``['always', '1', 'true', 'yes']``
+If a suitable ``mpi4py`` is not available, an exception will be raised and the run terminated.
+
+If you explicitly do not wish to use ``mpi4py``, set the environment variable ``PYOPTSPARSE_REQUIRE_MPI`` to anything other than those values.
+This can come in handy, for example, if your ``MPI`` installation is not functioning properly, but you still need to run serial code.
+
 Storing Optimization History
 ----------------------------
-pyOptSparse includes an :ref:`history` class that stores all the relevant optimization information an SQL database.
+pyOptSparse includes a :ref:`history` class that stores all the relevant optimization information an SQL database.
 This database is updated at every optimization iteration, and can be accessed via both the API described in the linked section, and via :ref:`optview`.
 By default, the history file is NOT written.
 To turn the history recording on, use the ``storeHistory`` attribute when invoking the optimization run, e.g.:

doc/install.rst

@@ -34,10 +34,12 @@ Python dependencies are automatically handled by ``pip``, so they do not need to
 The only exception is ``numpy``, which is required as part of the build process and therefore must be present before installing.
 
 .. note::
-  * In Linux, the python header files (python-dev) are also required.
+  * In Linux, the python header files (``python-dev``) are also required.
   * **We do not support operating systems other than Linux.**
     For macOS users, the conda package may work out of the box if you do not need any non-default optimizers.
-    For Windows users, `this thread <https://github.com/mdolab/pyoptsparse/issues/273>`_ may be helpful.
+    For Windows users, a conda package is on the way, if it's not already in the repos.
+    This comes with the same disclaimer as the macOS conda package.
+    Alternatively, follow the :ref:`conda build instructions<conda build instruction>` below as this will work on any platform.
 
 Installation
 ~~~~~~~~~~~~
@@ -57,13 +59,18 @@ For those not using virtual environments, a user install may be needed
 
 If you plan to modify pyOptSparse, installing with the developer option, i.e. with ``-e``, will save you from re-installing each time you modify the Python code.
 
-It is also possible to install pyOptSparse by calling ``python setup.py install``, but this is not recommended.
-
 .. note::
-  Some optimizers are proprietary and their sources are not distributed with pyOptSparse.
+  Some optimizers are proprietary, and their sources are not distributed with pyOptSparse.
   To use them, please follow the instructions on specific optimizer pages.
-
-For those who intend to use pyOptSparse with IPOPT, OpenMDAO developers provide a `bash script <https://github.com/OpenMDAO/build_pyoptsparse>`_ that simplifies the installation of the optimizer with different linear solvers.
+
+Specifying compilers
+~~~~~~~~~~~~~~~~~~~~
+To specify a non-default compiler (e.g. something other than ``/usr/bin/gcc``), meson recognizes certain `special environment variables <https://mesonbuild.com/Reference-tables.html#compiler-and-linker-selection-variables>`__.
+For example, to specify the Intel compilers, simply run
+
+.. prompt:: bash
+
+  FC=$(which ifort) CC=$(which icc) pip install .
 
 .. _install_optview:
 
@@ -96,7 +103,7 @@ to run all tests.
 
 Update or Uninstall
 -------------------
-To update pyOptSparse, first delete the ``build`` directory, then update the package using ``git``.
+To update pyOptSparse, first delete the ``meson_build`` directory, then update the package using ``git``.
 For stability, users are encouraged to stick to tagged releases.
 Install the package normally via ``pip``.
 
@@ -106,18 +113,74 @@ To uninstall the package, type
 
   pip uninstall pyoptsparse
 
-.. note::
-  pyOptSparse can optionally run in parallel if a suitable ``mpi4py``
-  installation exists. This will be automatically detected and
-  imported at run-time.
+.. _conda build instruction:
+
+Conda Build Instructions
+------------------------
+The following instructions explain how to build and install pyOptSparse in a conda environment.
+This has the advantage that ``conda`` can be used to install all the necessary dependencies in an isolated and reproducible environment.
+Considering how finicky Windows can be with ABI compatibility among various compilers, this is the recommended approach.
+The guide will work on any platform, however.
+
+The only build requirement for the build is a working ``conda`` installation as all compilers and dependencies are pulled from the ``conda-forge`` repos, with the exception of a Windows build, which requires Visual Studio 2017 C++ Build Tools.
+
+First, we need to create the ``conda`` environment.
+An ``environment.yml`` file is provided in the ``pyoptsparse`` repo:
+
+.. tabs::
+
+  .. code-tab:: bash Linux/OSX
+
+    conda create -y -n pyos-build
+    conda activate pyos-build
+    conda config --env --add channels conda-forge
+    conda config --env --set channel_priority strict
+
+    conda env update -f .github/environment.yml
+
+  .. code-tab:: powershell Windows
+
+    conda create -y -n pyos-build
+    conda activate pyos-build
+    conda config --env --add channels conda-forge
+    conda config --env --set channel_priority strict
+
+    conda env update -f .github\environment.yml
+    conda install libpgmath
+
+Next, we need to tell the compiler where to find IPOPT:
+
+.. tabs::
+
+  .. code-tab:: bash Linux/OSX
+
+    export IPOPT_DIR="$CONDA_PREFIX"
+
+  .. code-tab:: powershell Windows
+
+    set IPOPT_DIR=%CONDA_PREFIX%\Library
+
+Finally, build the wheel and install it using pip:
+
+.. tabs::
+
+  .. code-tab:: bash Linux/OSX
+
+    # build wheel
+    python -m build -n -x .
+
+    # install wheel
+    pip install --no-deps --no-index --find-links dist pyoptsparse
+
+  .. code-tab:: powershell Windows
 
-  If you only want to run in parallel, you can
-  force pyOptSparse to do so by setting the environment variable
-  ``PYOPTSPARSE_REQUIRE_MPI`` to anyone of these values: ``['always', '1', 'true', 'yes']``
-  If a suitable ``mpi4py`` is not available, an exception will be raised and the run
-  terminated.
+    # set specific compiler flags
+    set CC=cl
+    set FC=flang
+    set CC_LD=link
 
-  If you explicitly do not wish to use ``mpi4py``, set the environment variable ``PYOPTSPARSE_REQUIRE_MPI``
-  to anything other than those values. This can come in handy, for example, if your ``MPI`` installation
-  is not functioning properly, but you still need to run serial code.
+    # build wheel
+    python -m build -n -x .
 
+    # install wheel
+    pip install --no-deps --no-index --find-links dist pyoptsparse

meson.build

@@ -0,0 +1,45 @@
+# Much of this is from SciPy
+
+project(
+  'pyoptsparse',
+  'c', 'cpp',
+# unnecessary metadata commented out until Meson supports PEP517 and installation with pip
+#  version: 'x.x.x',
+#  license: 'GPL-3',
+  meson_version: '>= 0.60',
+  default_options: [
+    'buildtype=debugoptimized',
+    'c_std=c99',
+    'cpp_std=c++14',
+  ],
+)
+
+fortranobject_c = '../fortranobject.c'
+
+cc = meson.get_compiler('c')
+cpp = meson.get_compiler('cpp')
+
+# We need -lm for all C code (assuming it uses math functions, which is safe to
+# assume for SciPy). For C++ it isn't needed, because libstdc++/libc++ is
+# guaranteed to depend on it. For Fortran code, Meson already adds `-lm`.
+m_dep = cc.find_library('m', required : false)
+if m_dep.found()
+  add_project_link_arguments('-lm', language : 'c')
+endif
+
+# Adding at project level causes many spurious -lgfortran flags.
+add_languages('fortran', native: false)
+
+# https://mesonbuild.com/Python-module.html
+# Here we differentiate from the python used by meson, py3_command, and that python target, py3_target. This is useful
+# when cross compiling like on conda-forge
+py_mod = import('python')
+py3_command = py_mod.find_installation()
+if get_option('python_target') != ''
+    py3_target = py_mod.find_installation(get_option('python_target'))
+else
+    py3_target = py3_command
+endif
+py3_dep = py3_target.dependency()
+
+subdir('pyoptsparse')

meson_options.txt

@@ -0,0 +1,12 @@
+option('ipopt_dir', type: 'string', value: '',
+        description: 'Top-level dir for ipopt')
+
+option('incdir_numpy', type: 'string', value: '',
+    description: 'Include directory for numpy. If left empty Meson will try to find it on its own.')
+
+option('python_target', type: 'string', value: '',
+    description: '''Target python path. This is used in the case that the Python installation that PyOptSparse is intended
+    to be built for is different than the Python installation that is used to run Meson. For example, Meson may be installed
+    on the user's system which is run using the system Python installation, but the user may want build PyOptSparse for
+    a Python installation in a virtual environment. Leave as an empty string to build for Python installation running
+    Meson.''')

pyoptsparse/__init__.py

@@ -1,4 +1,4 @@
-__version__ = "2.8.3"
+__version__ = "2.9.0"
 
 from .pyOpt_history import History
 from .pyOpt_variable import Variable