[feature] Support for running tools built with *:shared=True on Windows from CMake

[timpatt]

What is your suggestion?

I'm not sure if this is a bug (supported feature, but not working) or a feature request (not currently supported feature)...

I'm upgrading our project to use Conan 2 and have hit an issue on Windows running Qt's moc.exe that was built as part of a conan install with the -o *:shared=True option. moc.exe depends on pcre2-16.dll and a bunch of other libs which are successfully built with conan, but are in their own directories and not on the path. I know there's the conan deploy stuff now, but that appears to be meant for runtime, not build-time. On Linux/mac, I suspect rpath sidesteps this issue with shared libraries, and when using static libraries there's no issues.

Here's a minimal repro: QtConanFail.zip

Note that in my actual use-case, CMake is the primary tool - we use CMakePresets.json to inject a customised version of conan_provider.cmake making Conan effectively a hidden implementation detail. One solution that does work (but not in my preferred usecase) is to use the "VirtualRunEnv" generator, run conan install, run the conanrun script and then run CMake.

I see a couple of ways forward with my preferred usecase:

• Find some way to update the PATH environment in the CMake process at configure-time (or pre-PROJECT()-time?). I'm not sure this is possible; here's my attempts so far:
  ** I extract the bin_paths from each dependency installed with conan install and set the PATH environment variable in CMake configure step set(ENV{PATH} "<qt tool dependency paths>") , but it doesn't get used in the generate step...
  ** I can set CMAKE_AUTOMOC_EXECUTABLE to cmake -e env PATH=<qt tool dependency paths> -- moc.exe. This alternative makes it past the CMake generation step, but the actual automoc process fails indicating that the CMAKE_AUTOMOC_EXECTUABLE doesn't exist...
• Somehow specify shared libraries only for immediate dependencies; transitive dependencies would default to static libraries and moc, uic, etc would work again
• Manually go through each of the tool dependencies and set (for example) pcre2:shared=False for each.
• Have a version of the "conan deploy" step that copies all "programs/tools" and required dependencies into a folder.

See also this slack discussion.

Have you read the CONTRIBUTING guide?

• I've read the CONTRIBUTING guide

[timpatt]

BTW - this use-case somehow worked with Conan 1.x... Maybe "*:shared=True" did not apply to transitive dependencies previously?

[memsharded]

Hi @timpatt

Thanks for your report.

It seems that this hasn't been discussed as a possibility, because you are intending to run purely from cmake, but running purely from CMake as you have realized can have some challenges.

For reference, the reported case can work with:

conan install . -of=build/conan -s compiler.cppstd=17

call build/conan/build/generators/conanrun.bat

:: Generate
cmake -B build -S . -DCMAKE_BUILD_TYPE=Release

:: Build
cmake --build build --config Release

As there are DLLs, after conan install activating the environment will put the DLLs in the path.
(Note, the VirtualRunEnv always run implicitly, no need to declare it explicitly in the conanfile.txt)

Then, an important point to take into account is that executable tools are expected to be used from the "build" context, that is, tool-requires. If you were cross-building, the above will never work, as the qt package would be one for the "host" context, and its executables wont run in the current "build" machine. For these cases, the approach is to use the same dependency both as tool_requires and requires, see the example with protobuf: https://docs.conan.io/2/examples/graph/tool_requires/using_protobuf.html. Where you aware of this use case?

With this use case, controlling the options for the "build" context is trivial, you can use *:shared=True for the "host" profile, but the qmoc executables will be by default statically linked, because they use the default "build" profile, which would be equivalent to *:shared=False for most ConanCenter packages.

[timpatt]

Thanks, @memsharded... I hadn't consider cross-compilation as a solution. I've never used cross compilation before, but spent a fair bit of today looking into it.

Looking at the conan-generated files, it looks like find_package(Qt6 ...) will always find the "host" version of moc (in conan_qt_executable_variables.cmake) unless the Qt6::moc target has already been created. If we use the conan-generated toolchain file, find_program(moc) will find the correct version because CMAKE_PROGRAM_PATH points to the build-context binary dir - we can create the target manually and then call find_package(Qt6) and it should all work. Is it standard practice to have to manually add support for build-context-tools or should the find_package(..) "just work"? The protobuf example you gave above would appear to "just work"...

I have worked out a couple of solutions to my problem that I've confirmed to work. Both parse the output of conan install ... --format=json and extract the bindirs from each dependency from it:

• Without separate build context: Generate a script for each build tool that sets the PATH appropriately and executes the build tool itself. Setting CMAKE_AUTOMOC_EXECUTABLE to point to the script works.
• With separate build context: Extract the bindirs paths from the packages with a "build" context, set CMAKE_PROGRAM_PATH and then find_program(CMAKE_AUTOMOC_EXECUTABLE moc).

[memsharded]

Looking at the conan-generated files, it looks like find_package(Qt6 ...) will always find the "host" version of moc (in conan_qt_executable_variables.cmake) unless the Qt6::moc target has already been created. If we use the conan-generated toolchain file, find_program(moc) will find the correct version because CMAKE_PROGRAM_PATH points to the build-context binary dir - we can create the target manually and then call find_package(Qt6) and it should all work. Is it standard practice to have to manually add support for build-context-tools or should the find_package(..) "just work"? The protobuf example you gave above would appear to "just work"...

This is mostly a recipe thing. Recipes might need to provide extra custom modules to be able to do the right thing.

This is one of the things we are trying to improve with the new CMakeDeps generator described in the "Incubating" section of the docs, check https://docs.conan.io/2/incubating.html, how executables are modeled (with .exe definition in recipes), how executables create targets, either in the host context or in the build context and used from there.
It is still dev-testing only, but if you want to give it a try, that would be useful feedback.

With separate build context: Extract the bindirs paths from the packages with a "build" context, set CMAKE_PROGRAM_PATH and then find_program(CMAKE_AUTOMOC_EXECUTABLE moc)

Interesting. This is already done automatically by the CMakeToolchain generating the conan_toolchain.cmake file. It defines the CMAKE_PROGRAM_PATH. This is one of the reasons why using the toolchain is recommended, it avoids hardcoding list(APPEND CMAKE_PREFIX_PATH ${CMAKE_BINARY_DIR}/conan/build/generators) in your CMakeLists.txt (which by the way, is a bit limited, as this can change based on CMake generators or platforms), and it will do many other things besides defining CMAKE_PROGRAM_PATH.

Related to the new incubating CMakeDeps generator, it will also create a new conan_cmakedeps_paths.cmake file that can be included that defines paths like CMAKE_PROGRAM_PATH.