Switch -isystem with -I in build flags

[suvayu]

Changes

• -isystem marks an include directory as a system include, which breaks conda-free builds, because it breaks the order of header file discovery for dependencies like Eigen.
• Update conda environment to include compilers, and use that for building so that conda builds do not pick system includes.

Testing

In a fresh conda environment created with the updated environment file, I did two separate builds, one with build.py script, the other with the cmake flags from CI.

Build with build.py

python ./build.py
python ./install.py -e
pytest

• The build has errors as shown below, but AFAIU, that is acceptable

  pybind11_stubgen - [  ERROR] In tudatpy.kernel.trajectory_design.transfer_trajectory.create_transfer_trajectory : Invalid expression 'tudat::simulation_setup::SystemOfBodies'
  pybind11_stubgen - [  ERROR] In tudatpy.kernel.trajectory_design.transfer_trajectory.set_low_thrust_acceleration : Invalid expression 'tudat::simulation_setup::SystemOfBodies'
  pybind11_stubgen - [WARNING] Enum-like str representations were found with no matching mapping to the enum class location.
  

• Running pytest gives me this failure, but it seems like an unrelated failure.

  __________________________________________________ test_compare_mpc_horizons_eph ____________________________________
  
      def test_compare_mpc_horizons_eph():
          """Compares true observations from BatchMPC to interpolated simulated RA/DEC from JPL Horizons"""
          batch = BatchMPC()
  >       batch.get_observations([433])
  
  tests/test_tudatpy/test_data_mpc.py:184:
  _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
  /opt/miniforge3/envs/tudatpy-dev/lib/python3.11/site-packages/tudatpy/data/mpc/mpc.py:874: in get_observations
      self._refresh_metadata()
  /opt/miniforge3/envs/tudatpy-dev/lib/python3.11/site-packages/tudatpy/data/mpc/mpc.py:730: in _refresh_metadata
      self._observatories = list(self._table.observatory.unique())
                                 ^^^^^^^^^^^^^^^^^^^^^^^
  _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
  
  self = Empty DataFrame
  Columns: []
  Index: [], name = 'observatory'
  
      @final
      def __getattr__(self, name: str):
          """
          After regular attribute access, try looking up the name
          This allows simpler access to columns for interactive use.
          """
          # Note: obj.x will always call obj.__getattribute__('x') prior to
          # calling obj.__getattr__('x').
          if (
              name not in self._internal_names_set
              and name not in self._metadata
              and name not in self._accessors
              and self._info_axis._can_hold_identifiers_and_holds_name(name)
          ):
              return self[name]
  >       return object.__getattribute__(self, name)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  E       AttributeError: 'DataFrame' object has no attribute 'observatory'
  
  /opt/miniforge3/envs/tudatpy-dev/lib/python3.11/site-packages/pandas/core/generic.py:6318: AttributeError

CMake build from CI flags

cmake -B _build -DCMAKE_BUILD_TYPE=Release \
    -DCMAKE_CXX_COMPILER=g++ -DCMAKE_CXX_COMPILER_LAUNCHER=ccache \
    -DCMAKE_C_COMPILER=gcc -DCMAKE_C_COMPILER_LAUNCHER=ccache \
    -S .
cmake --build _build/ --config Release -j6
cd _build && ctest --build-config Release --output-on-failure -j6

All tests pass!

Notes on my environment

I'm on a Fedora Linux machine, with many Boost libraries, including the development headers are installed.

$ rpm -qa boost\* | wc -l
31
$ rpm -qa boost\* | grep devel
boost-devel-1.83.0-12.fc42.x86_64

But both builds still pick the Boost libraries from conda:

• build.py build

$ ldd src/tudatpy/kernel.so | grep boost
	libboost_filesystem.so.1.85.0 => /opt/miniforge3/envs/tudatpy-dev/lib/libboost_filesystem.so.1.85.0 (0x00007f23f9bff000)
	libboost_system.so.1.85.0 => /opt/miniforge3/envs/tudatpy-dev/lib/libboost_system.so.1.85.0 (0x00007f23f9bfa000)
	libboost_regex.so.1.85.0 => /opt/miniforge3/envs/tudatpy-dev/lib/libboost_regex.so.1.85.0 (0x00007f23f9bb5000)
	libboost_date_time.so.1.85.0 => /opt/miniforge3/envs/tudatpy-dev/lib/libboost_date_time.so.1.85.0 (0x00007f23f9bb0000)
	libboost_thread.so.1.85.0 => /opt/miniforge3/envs/tudatpy-dev/lib/libboost_thread.so.1.85.0 (0x00007f23f9b90000)
	libboost_chrono.so.1.85.0 => /opt/miniforge3/envs/tudatpy-dev/lib/libboost_chrono.so.1.85.0 (0x00007f23f9b85000)
	libboost_atomic.so.1.85.0 => /opt/miniforge3/envs/tudatpy-dev/lib/libboost_atomic.so.1.85.0 (0x00007f23f9b7c000)
	libboost_unit_test_framework.so.1.85.0 => /opt/miniforge3/envs/tudatpy-dev/lib/libboost_unit_test_framework.so.1.85.0 (0x00007f23f7d57000)

• CI build

$ ldd tests/test_system_models_SelfShadowing | grep boost 
	libboost_filesystem.so.1.85.0 => /opt/miniforge3/envs/tudatpy-dev-ci/lib/libboost_filesystem.so.1.85.0 (0x00007fd3e01d5000)
	libboost_system.so.1.85.0 => /opt/miniforge3/envs/tudatpy-dev-ci/lib/libboost_system.so.1.85.0 (0x00007fd3e098f000)
	libboost_regex.so.1.85.0 => /opt/miniforge3/envs/tudatpy-dev-ci/lib/libboost_regex.so.1.85.0 (0x00007fd3e0190000)
	libboost_date_time.so.1.85.0 => /opt/miniforge3/envs/tudatpy-dev-ci/lib/libboost_date_time.so.1.85.0 (0x00007fd3e0988000)
	libboost_thread.so.1.85.0 => /opt/miniforge3/envs/tudatpy-dev-ci/lib/libboost_thread.so.1.85.0 (0x00007fd3e0172000)
	libboost_chrono.so.1.85.0 => /opt/miniforge3/envs/tudatpy-dev-ci/lib/libboost_chrono.so.1.85.0 (0x00007fd3e097d000)
	libboost_atomic.so.1.85.0 => /opt/miniforge3/envs/tudatpy-dev-ci/lib/libboost_atomic.so.1.85.0 (0x00007fd3e0169000)
	libboost_unit_test_framework.so.1.85.0 => /opt/miniforge3/envs/tudatpy-dev-ci/lib/libboost_unit_test_framework.so.1.85.0 (0x00007fd3e00c0000)

Caveats

I don't know how this behaves on Mac or Windows. I have a hunch for a Mac the conda compiler packages might have different names (clang and clangxx). For Windows, I don't know yet.

[DominicDirkx]

@suvayu I see you cancelled the latest build, was that intentional?

[DominicDirkx]

@suvayu It may be relevant for you: I've seen the Mac and Windows CI fail consistently everywhere. It looks to be a package incompatibility that occurs since Github updated they're runners. Some of our own-built dependencies (cspice, sofa, nrlmsise) are 1-2 years old. I'll rebuild their conda packages on Azure the coming week

[suvayu]

@DominicDirkx , thanks! That's very helpful! I was going a bit crazy 😛 The solution I found for our original problem got the builds past the configure step. In that case, I'll do the following:

• Make sure the Linux builds succeed
• Make sure the Windows & Mac builds get past the configure step
• Try to document where the Windows & Mac builds fail but won't try to fix it

[alfonsoSR]

Sorry, I deleted develop_10 by mistake 😅

[larshinueber]

@suvayu Since develop_10 has been merged into the regular develop branch, I've changed the base branch to develop so we can go ahead and delete develop_10 :)

[suvayu]

Hi, can someone on a Mac check if the build finishes successfully? I don't really know what to do for the build time. I guess some of the planned changes based on the comments from Louis will address this.