Skip to content

Commit 7b98186

Browse files
jackm97Jack Myersewu63sseraj
authored
Meson Build + Proper Windows Support (#300)
Co-authored-by: Jack Myers <[email protected]> Co-authored-by: Neil Wu <[email protected]> Co-authored-by: Neil Wu <[email protected]> Co-authored-by: Sabet Seraj <[email protected]>
1 parent 4ae37e6 commit 7b98186

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

45 files changed

+854
-2366
lines changed

‎.github/build_real.sh

+1-1
Original file line numberDiff line numberDiff line change
@@ -7,4 +7,4 @@ if [[ $IMAGE == "private" ]]; then
77
cp -r $HOME/SNOPT/* pyoptsparse/pySNOPT/source
88
fi
99

10-
pip install .[optview,testing]
10+
pip install .[optview,testing] -v

‎.github/environment.yml

+18
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,18 @@
1+
dependencies:
2+
# build
3+
- python >=3.8
4+
- numpy >=1.16
5+
- ipopt
6+
- swig
7+
- meson >=0.60
8+
- compilers
9+
- pkg-config
10+
- pip
11+
- setuptools
12+
- build
13+
# testing
14+
- parameterized
15+
- testflo
16+
- scipy >1.2
17+
- mdolab-baseclasses >=1.3.1
18+
- sqlitedict >=1.6

‎.github/windows.yaml

+41
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,41 @@
1+
trigger:
2+
- main
3+
4+
pr:
5+
- main
6+
7+
pool:
8+
vmImage: windows-latest
9+
10+
jobs:
11+
- job: Windows
12+
steps:
13+
- powershell: Write-Host "##vso[task.prependpath]$env:CONDA\Scripts"
14+
displayName: Add conda to PATH
15+
16+
- script: conda config --add channels conda-forge && conda config --set channel_priority strict
17+
displayName: Set channel priority
18+
19+
- script: conda create --yes --name pyos-build
20+
displayName: Create environment
21+
22+
- script: |
23+
call activate pyos-build
24+
call conda install -y mamba
25+
call mamba env update --file .github/environment.yml
26+
call mamba install -y libpgmath
27+
displayName: Install mamba and update environment
28+
29+
- script: |
30+
set IPOPT_DIR=%CONDA_PREFIX%\Library
31+
set CC=cl
32+
set FC=flang
33+
set CC_LD=link
34+
python -m build -n -x .
35+
pip install --no-deps --no-index --find-links dist pyoptsparse
36+
displayName: Build and install pyoptsparse
37+
38+
- script: |
39+
cd tests
40+
testflo -n 1 .
41+
displayName: Run tests

‎.github/workflows/windows-build.yml

+45
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,45 @@
1+
name: PyOptSparse Windows Actions
2+
3+
on:
4+
push:
5+
branches:
6+
- main
7+
pull_request:
8+
branches:
9+
- main
10+
11+
jobs:
12+
build-windows:
13+
runs-on: windows-latest
14+
15+
steps:
16+
- uses: actions/checkout@v2
17+
- uses: conda-incubator/setup-miniconda@v2
18+
with:
19+
python-version: 3.8
20+
mamba-version: "*"
21+
channels: conda-forge,defaults
22+
channel-priority: strict
23+
activate-environment: pyos-build
24+
environment-file: .github/environment.yml
25+
- name: Install libpgmath
26+
shell: bash -l {0}
27+
run: |
28+
conda activate pyos-build
29+
mamba install libpgmath
30+
- name: Build and install pyoptsparse
31+
shell: cmd /C CALL {0}
32+
run: |
33+
call conda activate pyos-build
34+
set IPOPT_DIR=%CONDA_PREFIX%\Library
35+
set CC=cl
36+
set FC=flang
37+
set CC_LD=link
38+
python -m build -n -x .
39+
pip install --no-deps --no-index --find-links dist pyoptsparse
40+
- name: Run tests
41+
shell: bash -l {0}
42+
run: |
43+
conda activate pyos-build
44+
cd tests
45+
testflo --pre_announce -v -n 1 .

‎.gitignore

+11
Original file line numberDiff line numberDiff line change
@@ -13,3 +13,14 @@ env
1313
pyoptsparse/pySNOPT/source
1414
pyoptsparse/pyNLPQLP/source
1515
*.egg-info
16+
17+
.idea/
18+
/meson_build/
19+
/dist/
20+
/staging_dir/
21+
22+
*.lib
23+
24+
*.pdb
25+
26+
*.pyd

‎doc/advancedFeatures.rst

+13-1
Original file line numberDiff line numberDiff line change
@@ -6,9 +6,21 @@ Advanced Features
66
.. Parallel Execution
77
.. ------------------
88
9+
MPI handling
10+
------------
11+
pyOptSparse can optionally run in parallel if a suitable ``mpi4py`` installation exists.
12+
This will be automatically detected and imported at run-time.
13+
14+
If you only want to run in parallel, you can force pyOptSparse to do so by setting the environment variable
15+
``PYOPTSPARSE_REQUIRE_MPI`` to any one of these values: ``['always', '1', 'true', 'yes']``
16+
If a suitable ``mpi4py`` is not available, an exception will be raised and the run terminated.
17+
18+
If you explicitly do not wish to use ``mpi4py``, set the environment variable ``PYOPTSPARSE_REQUIRE_MPI`` to anything other than those values.
19+
This can come in handy, for example, if your ``MPI`` installation is not functioning properly, but you still need to run serial code.
20+
921
Storing Optimization History
1022
----------------------------
11-
pyOptSparse includes an :ref:`history` class that stores all the relevant optimization information an SQL database.
23+
pyOptSparse includes a :ref:`history` class that stores all the relevant optimization information an SQL database.
1224
This database is updated at every optimization iteration, and can be accessed via both the API described in the linked section, and via :ref:`optview`.
1325
By default, the history file is NOT written.
1426
To turn the history recording on, use the ``storeHistory`` attribute when invoking the optimization run, e.g.:

‎doc/install.rst

+83-20
Original file line numberDiff line numberDiff line change
@@ -34,10 +34,12 @@ Python dependencies are automatically handled by ``pip``, so they do not need to
3434
The only exception is ``numpy``, which is required as part of the build process and therefore must be present before installing.
3535

3636
.. note::
37-
* In Linux, the python header files (python-dev) are also required.
37+
* In Linux, the python header files (``python-dev``) are also required.
3838
* **We do not support operating systems other than Linux.**
3939
For macOS users, the conda package may work out of the box if you do not need any non-default optimizers.
40-
For Windows users, `this thread <https://github.com/mdolab/pyoptsparse/issues/273>`_ may be helpful.
40+
For Windows users, a conda package is on the way, if it's not already in the repos.
41+
This comes with the same disclaimer as the macOS conda package.
42+
Alternatively, follow the :ref:`conda build instructions<conda build instruction>` below as this will work on any platform.
4143

4244
Installation
4345
~~~~~~~~~~~~
@@ -57,13 +59,18 @@ For those not using virtual environments, a user install may be needed
5759

5860
If you plan to modify pyOptSparse, installing with the developer option, i.e. with ``-e``, will save you from re-installing each time you modify the Python code.
5961

60-
It is also possible to install pyOptSparse by calling ``python setup.py install``, but this is not recommended.
61-
6262
.. note::
63-
Some optimizers are proprietary and their sources are not distributed with pyOptSparse.
63+
Some optimizers are proprietary, and their sources are not distributed with pyOptSparse.
6464
To use them, please follow the instructions on specific optimizer pages.
65-
66-
For those who intend to use pyOptSparse with IPOPT, OpenMDAO developers provide a `bash script <https://github.com/OpenMDAO/build_pyoptsparse>`_ that simplifies the installation of the optimizer with different linear solvers.
65+
66+
Specifying compilers
67+
~~~~~~~~~~~~~~~~~~~~
68+
To specify a non-default compiler (e.g. something other than ``/usr/bin/gcc``), meson recognizes certain `special environment variables <https://mesonbuild.com/Reference-tables.html#compiler-and-linker-selection-variables>`__.
69+
For example, to specify the Intel compilers, simply run
70+
71+
.. prompt:: bash
72+
73+
FC=$(which ifort) CC=$(which icc) pip install .
6774

6875
.. _install_optview:
6976

@@ -96,7 +103,7 @@ to run all tests.
96103

97104
Update or Uninstall
98105
-------------------
99-
To update pyOptSparse, first delete the ``build`` directory, then update the package using ``git``.
106+
To update pyOptSparse, first delete the ``meson_build`` directory, then update the package using ``git``.
100107
For stability, users are encouraged to stick to tagged releases.
101108
Install the package normally via ``pip``.
102109

@@ -106,18 +113,74 @@ To uninstall the package, type
106113

107114
pip uninstall pyoptsparse
108115

109-
.. note::
110-
pyOptSparse can optionally run in parallel if a suitable ``mpi4py``
111-
installation exists. This will be automatically detected and
112-
imported at run-time.
116+
.. _conda build instruction:
117+
118+
Conda Build Instructions
119+
------------------------
120+
The following instructions explain how to build and install pyOptSparse in a conda environment.
121+
This has the advantage that ``conda`` can be used to install all the necessary dependencies in an isolated and reproducible environment.
122+
Considering how finicky Windows can be with ABI compatibility among various compilers, this is the recommended approach.
123+
The guide will work on any platform, however.
124+
125+
The only build requirement for the build is a working ``conda`` installation as all compilers and dependencies are pulled from the ``conda-forge`` repos, with the exception of a Windows build, which requires Visual Studio 2017 C++ Build Tools.
126+
127+
First, we need to create the ``conda`` environment.
128+
An ``environment.yml`` file is provided in the ``pyoptsparse`` repo:
129+
130+
.. tabs::
131+
132+
.. code-tab:: bash Linux/OSX
133+
134+
conda create -y -n pyos-build
135+
conda activate pyos-build
136+
conda config --env --add channels conda-forge
137+
conda config --env --set channel_priority strict
138+
139+
conda env update -f .github/environment.yml
140+
141+
.. code-tab:: powershell Windows
142+
143+
conda create -y -n pyos-build
144+
conda activate pyos-build
145+
conda config --env --add channels conda-forge
146+
conda config --env --set channel_priority strict
147+
148+
conda env update -f .github\environment.yml
149+
conda install libpgmath
150+
151+
Next, we need to tell the compiler where to find IPOPT:
152+
153+
.. tabs::
154+
155+
.. code-tab:: bash Linux/OSX
156+
157+
export IPOPT_DIR="$CONDA_PREFIX"
158+
159+
.. code-tab:: powershell Windows
160+
161+
set IPOPT_DIR=%CONDA_PREFIX%\Library
162+
163+
Finally, build the wheel and install it using pip:
164+
165+
.. tabs::
166+
167+
.. code-tab:: bash Linux/OSX
168+
169+
# build wheel
170+
python -m build -n -x .
171+
172+
# install wheel
173+
pip install --no-deps --no-index --find-links dist pyoptsparse
174+
175+
.. code-tab:: powershell Windows
113176

114-
If you only want to run in parallel, you can
115-
force pyOptSparse to do so by setting the environment variable
116-
``PYOPTSPARSE_REQUIRE_MPI`` to anyone of these values: ``['always', '1', 'true', 'yes']``
117-
If a suitable ``mpi4py`` is not available, an exception will be raised and the run
118-
terminated.
177+
# set specific compiler flags
178+
set CC=cl
179+
set FC=flang
180+
set CC_LD=link
119181

120-
If you explicitly do not wish to use ``mpi4py``, set the environment variable ``PYOPTSPARSE_REQUIRE_MPI``
121-
to anything other than those values. This can come in handy, for example, if your ``MPI`` installation
122-
is not functioning properly, but you still need to run serial code.
182+
# build wheel
183+
python -m build -n -x .
123184

185+
# install wheel
186+
pip install --no-deps --no-index --find-links dist pyoptsparse

‎meson.build

+45
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,45 @@
1+
# Much of this is from SciPy
2+
3+
project(
4+
'pyoptsparse',
5+
'c', 'cpp',
6+
# unnecessary metadata commented out until Meson supports PEP517 and installation with pip
7+
# version: 'x.x.x',
8+
# license: 'GPL-3',
9+
meson_version: '>= 0.60',
10+
default_options: [
11+
'buildtype=debugoptimized',
12+
'c_std=c99',
13+
'cpp_std=c++14',
14+
],
15+
)
16+
17+
fortranobject_c = '../fortranobject.c'
18+
19+
cc = meson.get_compiler('c')
20+
cpp = meson.get_compiler('cpp')
21+
22+
# We need -lm for all C code (assuming it uses math functions, which is safe to
23+
# assume for SciPy). For C++ it isn't needed, because libstdc++/libc++ is
24+
# guaranteed to depend on it. For Fortran code, Meson already adds `-lm`.
25+
m_dep = cc.find_library('m', required : false)
26+
if m_dep.found()
27+
add_project_link_arguments('-lm', language : 'c')
28+
endif
29+
30+
# Adding at project level causes many spurious -lgfortran flags.
31+
add_languages('fortran', native: false)
32+
33+
# https://mesonbuild.com/Python-module.html
34+
# Here we differentiate from the python used by meson, py3_command, and that python target, py3_target. This is useful
35+
# when cross compiling like on conda-forge
36+
py_mod = import('python')
37+
py3_command = py_mod.find_installation()
38+
if get_option('python_target') != ''
39+
py3_target = py_mod.find_installation(get_option('python_target'))
40+
else
41+
py3_target = py3_command
42+
endif
43+
py3_dep = py3_target.dependency()
44+
45+
subdir('pyoptsparse')

‎meson_options.txt

+12
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
option('ipopt_dir', type: 'string', value: '',
2+
description: 'Top-level dir for ipopt')
3+
4+
option('incdir_numpy', type: 'string', value: '',
5+
description: 'Include directory for numpy. If left empty Meson will try to find it on its own.')
6+
7+
option('python_target', type: 'string', value: '',
8+
description: '''Target python path. This is used in the case that the Python installation that PyOptSparse is intended
9+
to be built for is different than the Python installation that is used to run Meson. For example, Meson may be installed
10+
on the user's system which is run using the system Python installation, but the user may want build PyOptSparse for
11+
a Python installation in a virtual environment. Leave as an empty string to build for Python installation running
12+
Meson.''')

‎pyoptsparse/__init__.py

+1-1
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
__version__ = "2.8.3"
1+
__version__ = "2.9.0"
22

33
from .pyOpt_history import History
44
from .pyOpt_variable import Variable

0 commit comments

Comments
 (0)