ament_vendor: Add IS_VENDORED_OUTPUT_VARIABLE_NAME argument and AMENT_VENDOR_POLICY CMake option

[traversaro]

Evolution of #552 .

This PR aim is based on our experience at RoboStack with the ament_vendor, and aims to improve the experience of downstream packagers of ROS on non-officially supported platform (being them conda/robostack, nix or similar), reducing the need of patching the vendor package to obtain the required behaviour.

In particular, a common pattern that we are experiencing is that compare to stable linux distributions, rolling releases as conda-forge have much more recent packages, so in many case we want to disable completely the vendoring in _vendor packages.

A few examples:

• zenoh_cpp_vendor: In conda-forge, we have a build of zenoh_cpp that includes all the features required by rmw_zenoh (see https://github.com/ros2/rmw_zenoh/blob/56f2688994600a176b9b26f7c2f23c00487e220c/zenoh_cpp_vendor/CMakeLists.txt#L18) and also all the default features enabled, so we want to force the use of the conda-forge version, instead of having zenoh_cpp_vendor builds its own limited variant of zenoh, that could conflict with the conda-forge' zenoh.
• gz-*-vendor packages: In conda-forge, we have a build of the gz libraries that support Python bindings (differently from vendor packages, see Enable Python Bindings gazebo-tooling/gz_vendor#2) so we want to use them. However, the satisfied check in gz-math-vendor and other packages is EXACT by default (see https://github.com/gazebo-release/gz_math_vendor/blob/51d2e68e872d2f071bdf40fe7017b9a00ddd8a19/CMakeLists.txt#L31-L37), and the check is not easily disabled by CMake arguments (just from environment variables, but that is tricky for example to set via colcon.pkg files).

At the moment, we handle that with a lot of patches (see for example https://github.com/RoboStack/ros-kilted/blob/main/patch/ros-kilted-zenoh-cpp-vendor.patch, https://github.com/RoboStack/ros-kilted/blob/main/patch/ros-kilted-gz-cmake-vendor.patch, https://github.com/RoboStack/ros-kilted/blob/main/patch/ros-kilted-gz-rendering-vendor.patch and all similar patch in that directory). However, patches are typically cumbersome to maintain, as you need to frequently update them if patched project changes.

To deal with this, this PR adds the AMENT_VENDOR_POLICY CMake option, whose documentation is included in the code, and is reported in the following:

#   AMENT_VENDOR_POLICY: String option that specifies how ament_vendor behaves,
#                        the allowed values are listed in the following.
#     DEFAULT: Vendor if ``SATISFIED`` argument is not supplied or false,
#              do not vendor otherwise.
#     FORCE_BUILD_VENDOR: Always vendor, independently of the value of the
#                   ``SATISFIED`` argument.
#     NEVER_VENDOR: Never vendor, and raise an error if ``SATISFIED`` argument
#                   is not supplied or false.
#     NEVER_VENDOR_IGNORE_SATISFIED_CHECK: Never vendor, and do not raise
#                   an error even if ``SATISFIED`` argument is not supplied
#                   or false. This option is in unsupported by most packages,
#                   so use at your own risk, as it could break the buid.

One downside of having this option, is that now it is not anymore trivial to know if ament_vendor actually vendored something or not, while before it was sufficient to check if the variable passed to SATISFIED was TRUE or not (see for example https://github.com/gazebo-release/gz_sim_vendor/blob/423f5eaf34d1c69735c99704c610e434e902886e/CMakeLists.txt#L90). For this reason, the PR also adds a IS_VENDORED_OUTPUT_VARIABLE_NAME argument that pass the variable name that is populated by ament_vendor reporting if the package is actually vendored or not.

For example, the gz-vendor packages could be modified as in the following:

ament_vendor(${LIB_NAME_UNDERSCORE}_vendor
  SATISFIED ${${LIB_NAME_FULL}_FOUND}
  VCS_URL https://github.com/gazebosim/${GITHUB_NAME}.git
  VCS_VERSION ${GITHUB_NAME}${LIB_VER_MAJOR}_${LIB_VER}${LIB_VER_SUFFIX}
  CMAKE_ARGS
    -DSKIP_PYBIND11:BOOL=ON
  GLOBAL_HOOK
  IS_VENDORED_OUTPUT_VARIABLE_NAME ${LIB_NAME_UNDERSCORE}_is_vendored
)

# [...]

# Code that only should done if the package is actually vendored
if(${LIB_NAME_UNDERSCORE}_is_vendored)
  ament_environment_hooks("${CMAKE_CURRENT_SOURCE_DIR}/${PROJECT_NAME}.dsv.in")
  # Create a dummy .sh file needed for ament_package to source the .dsv file.
  # See https://github.com/ament/ament_package/issues/145
  file(WRITE ${CMAKE_CURRENT_BINARY_DIR}/${PROJECT_NAME}.sh "# Dummy .sh file needed for .dsv file to be sourced.")
  ament_environment_hooks("${CMAKE_CURRENT_BINARY_DIR}/${PROJECT_NAME}.sh")
endif()

[YuanYuYuan]

As one of the ros-nix-overlay users, I'm excited about this solution 😄

[traversaro]

As one of the ros-nix-overlay users, I'm excited about this solution 😄

Great, feel free to comment if there is anything that could be improved from the nix point of view, as I a not so familiar with nix.

[sloretz]

Thank you for the PR!

I have a couple nits and a question, but this code seems to do what it says.

[sloretz]

@cottsay this seems to differ from your suggestion of using the presence of the SATISFIED argument to enable the check in #552 (comment) , but I think it accomplishes the same thing as FORCE_BUILD_VENDOR by building the package regardless of SATISFIED being set.

It seems like we'd need CMake 3.31 to handle the case where SATISFIED is given but empty string, so I think if we used if(DEFINED... then the warning would be skipped if SATISFIED was provided with a variable containing the empty string.

[cottsay]

It really depends on how @traversaro thinks that an unconditionally vendored package should behave in the presence of NEVER_VENDOR_IGNORE_SATISFIED_CHECK. The naming makes me think that a vendored package which doesn't even check for satisfaction would NOT raise an error, in which case, I agree that using DEFINED to check if _ARG_SATISFIED was even supplied would be a better behavior than raising an error 100% of the time.

[traversaro]

I am not sure I follow here. The value of _ARG_SATISFIED is normalized earlier:

  if(NOT _ARG_SATISFIED)
    set(_ARG_SATISFIED FALSE)
  endif()

after that check, it is not possible for _ARG_SATISFIED to be undefined or to be defined as an empty string. The only possible values after that check are FALSE or all values that are considered true by CMake, i.e. 1, ON, YES, TRUE, Y, or a non-zero number (including floating point numbers), see https://cmake.org/cmake/help/latest/command/if.html#basic-expressions .

[cottsay]

Elsewhere in ROS 2 we're already using if(TARGET target_name) for this purpose. Is there some reason that approach isn't sufficient?

For example: https://github.com/ros2/libyaml_vendor/blob/c69b99721471d47f88042b44e510cfefafb385b9/CMakeLists.txt#L27

[traversaro]

Good point! I did not think about this approach, and apparently I am not the only one (all gz vendor packages duplicate the satifisfied condition to check it the package was vendored or not, see https://github.com/gazebo-release/gz_math_vendor/blob/51d2e68e872d2f071bdf40fe7017b9a00ddd8a19/CMakeLists.txt#L67). But using the target defined by the ExternalProject make total sense, I will revise the PR to suggest doing that instead.

[cottsay]

Suggested change

	# or false. This option is in unsupported by most packages,
	# or false. This option is unsupported by most packages,

[cottsay]

Is there a reason to hide this?

[traversaro]

Probably not, I typically hide options defined in functions to avoid littering the downstream projects option space (for example, last time I checked calling find_package(rclcpp) basically hides all project specific options if they start with a letter later in the alphabet), but in this case if ament_vendor is used in a project, then the main purpouse of a project is to be a vendor package, and then it make sense for the option to be visible.

[cottsay]

It really depends on how @traversaro thinks that an unconditionally vendored package should behave in the presence of NEVER_VENDOR_IGNORE_SATISFIED_CHECK. The naming makes me think that a vendored package which doesn't even check for satisfaction would NOT raise an error, in which case, I agree that using DEFINED to check if _ARG_SATISFIED was even supplied would be a better behavior than raising an error 100% of the time.