[question] Define option in python_requires conanfile (conan2)

[Aki1608]

What is your question?

Hi, I am working with migration to conan2.

I have a recipe "Libs" which consumes another recipe "Base_recipe" through python_requires. Now the "Base_recipe" contains a file which generates a new conanfile by defining everything. Options, requires(), build_requires(), etc. The thing is this has been created to add some specific build_requirements() and options to different recipes which consumes this recipe through python_requires.

Now, In conan1, we used to directly define or modify options by:

        options = self.conan_data.get("options", {})

        for (name, values) in options.items():
            name = str(name)
            self.options[name] = values
            self.default_options[name] = values[0]

But directly assigning value to options is not allowed in conan2. So I tried

        opts = self.conan_data.get("options", {})
        self.options.update(options_values=OrderedDict(opts))
        for name, values in opts.items():
            self.default_options[name] = values[0]

But at later stage, I am getting this error:

======== Computing necessary packages ========

ERROR: Traceback (most recent call last):
  File "/home/user/.local/lib/python3.8/site-packages/conans/model/options.py", line 39, in copy_conaninfo_option
    assert self._possible_values is not None  # this should always come from recipe, with []
AssertionError

ERROR: 

I tried

        self.options = self.conan_data.get("options", {})
        self.options.update({f"{name}": values})
        self.default_options.update({f"{name}": values[0]})

But then I am getting this error:

  File "/home/user/.local/lib/python3.8/site-packages/conan/internal/methods.py", line 97, in run_configure_method
    conanfile.options.apply_downstream(down_options, profile_options, ref, is_consumer)
AttributeError: 'dict' object has no attribute 'apply_downstream'

ERROR: 'dict' object has no attribute 'apply_downstream'

I am defining this in "Base_recipe" in config_options(). I tried defining this in init() as well.

The type of options has been changed in conan2? What am I doing wrong here? I hope I was able to explain my problem properly.

Have you read the CONTRIBUTING guide?

• I've read the CONTRIBUTING guide

[memsharded]

Hi @Aki1608

Thanks for your question.

Now, In conan1, we used to directly define or modify options by:

Could you please give a bit more of context for this? What is the method or part of the file this is running?

I have tried a simple case of inheriting options with:

    def test_basic_option_inherit(self):
        client = TestClient()
        base = textwrap.dedent("""
            from conan import ConanFile
            class MyBase(object):
                settings = "os", "build_type", "arch"
                options = {"base_option": [1, 2, 3]}
                default_options = {"base_option": 2}

            class BaseConanFile(ConanFile):
                pass
            """)
        client.save({"conanfile.py": base})
        client.run("export . --name=base --version=1.0")
        derived = textwrap.dedent("""
            from conan import ConanFile
            class DerivedConan(ConanFile):
                settings = "os", "build_type", "arch"

                python_requires = "base/1.0"
                python_requires_extend = 'base.MyBase'

                def generate(self):
                    self.output.info(f"MYOPTION: {self.options.base_option}!!!!")
                """)
        client.save({"conanfile.py": derived})
        client.run("install .")
        print(client.out)
        assert "conanfile.py: MYOPTION: 2!!!!" in client.out

And it seems to work fine.

It would be ideal to have a full and minimal base_conanfile.py and another full and minimal conanfile.py that does the python_requires to the base one, to illustrate the issue, so we can reproduce here and fully understand it. Thanks!

[Aki1608]

I can access/read the value of options. What I want to do is that I want to read the options from conandata.yml and then I want to assign that to self.options.

I have created a minimal code.

conandata.yml

requires:
  #jansson/2.12:
  jsoncpp/1.9.1@conan-center-index/pr-1:

requires.OpenSSL:
  openssl/1.1.1s@conan-center-index/pr-1:

requires.BoringSSL:
  boringssl/cci.20200724@conan-center-index/pr-1:

options:
  native: ["auto", False, True]
  with_ssl: [OpenSSL, BoringSSL]

py_req.py

   from conan import ConanFile
   from urllib.parse import urlparse
   from collections import OrderedDict

   class PyReq(ConanFile):
    
    name = "py_req"
    version = "1.0"
    settings = "os", "arch", "compiler", "build_type"

    def configure(self):
        self.settings.rm_safe("compiler.cppstd")
        self.settings.rm_safe("compiler.libcxx")

    def config_options(self):
        if self.settings.os== "Linux":
            del self.settings.compiler.threads
            del self.settings.compiler.exception
            self.settings.compiler.sdkversion = [None]

        if self.conan_data is None:
            self.conan_data = {}
        opts = self.conan_data.get("options", {})

        if self.options is None:
            self.options = {}
        if self.default_options is None:
            self.default_options = {}

        self.options.update(options_values=OrderedDict(opts))
        for name, values in opts.items():
            self.default_options[name] = values[0]

        if self.options:
            if "fPIC" not in self.options:
                self.options.update(options_values=OrderedDict({"fPIC": [False, True]}))
                self.default_options.update({"fPIC": True})
            if "shared" not in self.options:
                self.options.update(options_values=OrderedDict({"shared": [False, True]}))
                self.default_options.update({"shared": False})


        print(f"final Options and their values: {self.options, self.default_options}")


    def requirements(self):
        pass

    def build(self):
        pass

    def package(self):
        pass

So running conan create py_req.py is giving me error:

======== Computing necessary packages ========
ERROR: Traceback (most recent call last):
  File "/home/einfochips/.local/lib/python3.8/site-packages/conan/cli/cli.py", line 307, in main
    cli.run(args)
  File "/home/einfochips/.local/lib/python3.8/site-packages/conan/cli/cli.py", line 192, in run
    command.run(self._conan_api, args[0][1:])
  File "/home/einfochips/.local/lib/python3.8/site-packages/conan/cli/command.py", line 180, in run
    info = self._method(conan_api, parser, *args)
  File "/home/einfochips/.local/lib/python3.8/site-packages/conan/cli/commands/create.py", line 114, in create
    conan_api.graph.analyze_binaries(deps_graph, build_modes, remotes=remotes,
  File "/home/einfochips/.local/lib/python3.8/site-packages/conan/api/subapi/graph.py", line 201, in analyze_binaries
    binaries_analyzer.evaluate_graph(graph, build_mode, lockfile, remotes, update,
  File "/home/einfochips/.local/lib/python3.8/site-packages/conans/client/graph/graph_binaries.py", line 440, in evaluate_graph
    self._evaluate_package_id(node, config_version)
  File "/home/einfochips/.local/lib/python3.8/site-packages/conans/client/graph/graph_binaries.py", line 400, in _evaluate_package_id
    compute_package_id(node, self._modes, config_version=config_version)
  File "/home/einfochips/.local/lib/python3.8/site-packages/conans/client/graph/compute_pid.py", line 40, in compute_package_id
    copied_options = conanfile.options.copy_conaninfo_options()
  File "/home/einfochips/.local/lib/python3.8/site-packages/conans/model/options.py", line 340, in copy_conaninfo_options
    result._package_options = self._package_options.copy_conaninfo_options()
  File "/home/einfochips/.local/lib/python3.8/site-packages/conans/model/options.py", line 155, in copy_conaninfo_options
    result._data[k] = v.copy_conaninfo_option()
  File "/home/einfochips/.local/lib/python3.8/site-packages/conans/model/options.py", line 39, in copy_conaninfo_option
    assert self._possible_values is not None  # this should always come from recipe, with []
AssertionError

ERROR: 

I know in the original question I said that I have two components: "base_recipe" and "libs" recipe. I was trying to recreate the error with minimal code but I got the same error while trying to build py_req.py file. So I have included only that file here.

In my case, py_req.py is a module of "base_recipe" component. py_req.py creates a new conanfile by adding requirements and options. And then in "Libs" I am doing python_requires for "base_recipe".

I hope I was able to explain properly this time.

[memsharded]

The code that you shared seems unrelated to python-requires at this moment, it is just regular config_options management.

        if self.conan_data is None:
            self.conan_data = {}
        opts = self.conan_data.get("options", {})

        if self.options is None:
            self.options = {}
        if self.default_options is None:
            self.default_options = {}

Please don't do that. You cannot overwrite the self.options object, because at this point, it is already an internal Options object, if you change it by a dictionary things will break.
Same for default_options and conan_data. These objects are loaded and initialized by Conan, do not re-create those objects.

if self.settings.os== "Linux":
            del self.settings.compiler.threads
            del self.settings.compiler.exception
            self.settings.compiler.sdkversion = [None]

Don't do this in the config_options() method. Do it at most in the configure() method, as the method name shows, this method is intended only for options.
Furthermore, you cannot change the value of a setting in a recipe, see https://docs.conan.io/2/knowledge/guidelines.html

Settings and configuration (conf) are read-only in recipes: The settings and configuration cannot be defined or assigned values in recipes. Something like self.settings.compiler = "gcc" in recipes shouldn’t be done. That is undefined behavior and can crash at any time, or just be ignored. Settings and configuration can only be defined in profiles, in command line arguments or in the profile.py plugin.

So you cannot assign a value for a setting in a recipe. Not to None. And in this case, it was asigned to a list [] type, which is not supported and breaking.

self.options.update(options_values=OrderedDict(opts))
        for name, values in opts.items():
            self.default_options[name] = values[0]

        if self.options:
            if "fPIC" not in self.options:
                self.options.update(options_values=OrderedDict({"fPIC": [False, True]}))
                self.default_options.update({"fPIC": True})
            if "shared" not in self.options:
                self.options.update(options_values=OrderedDict({"shared": [False, True]}))
                self.default_options.update({"shared": False})

You can't create new options or new default values in the config_options() method either.

If you want to initialize options based on conandata.yml, that can be done in the init() method. This works:

from conan import ConanFile

class PyReq(ConanFile):

    name = "py_req"
    version = "1.0"
    settings = "os", "arch", "compiler", "build_type"

    def init(self):
        opts = self.conan_data.get("options", {})
        self.options.update(opts)

    def generate(self):
        self.output.info(f"NATIVE: {self.options.native}")
        self.output.info(f"WITH SSL: {self.options.with_ssl}")

[Aki1608]

You cannot overwrite the self.options object, because at this point, it is already an internal Options object, if you change it by a dictionary things will break.

If I don't change it by dictionary, and directly try assigning value by self.options.update(opts), I am getting this error:

TypeError: 'NoneType' object does not support item assignment

For the py_req.py, I changed it and defined the option intialization in init().

If I use:

self.options.update(options_values=OrderedDict(opts))
        for name, values in opts.items():
            self.default_options[name] = values[0]

I am getting the same error.

If I use:

self.options.update(opts)
        for name, values in opts.items():
            self.default_options[name] = values[0]

I get this error:

======== Computing necessary packages ========
Requirements
    py_req/1.0#686380df8b17d53a4886d9cb056ffb5b:e38f6b37ea54e2b058a51ab6225c858a59c5c77e - Invalid
ERROR: There are invalid packages:
py_req/1.0: Invalid: 'options.native' value not defined

This is new puy_req.py

from conan import ConanFile
from urllib.parse import urlparse
from collections import OrderedDict

class PyReq(ConanFile):
    
    name = "py_req"
    version = "1.0"
    settings = "os", "arch", "compiler", "build_type"


    def init(self):
        if self.conan_data is None:
            self.conan_data = {}
        opts = self.conan_data.get("options", {})

        if self.options is None:
            self.options = {}
        if self.default_options is None:
            self.default_options = {}

        self.options.update(opts)
        for name, values in opts.items():
            self.default_options[name] = values[0]

        self.output.info(f"final Options and their values: {self.options, self.default_options}")

    def generate(self):
        self.output.info(f"NATIVE: {self.options.native}")
        self.output.info(f"WITH SSL: {self.options.with_ssl}")

[memsharded]

def init(self):
        if self.conan_data is None:
            self.conan_data = {}
        opts = self.conan_data.get("options", {})

        if self.options is None:
            self.options = {}
        if self.default_options is None:
            self.default_options = {}

You can't do this either.
Both the self.conan_data and the self.options are constructed and initialized when the conanfile is loaded from disk.
So when calling init, the options is already an Options object, and self.conan_data will be initialized. So please don't do that.
Trying to change default_options dictionary here is also useless.

If you want to assign the first value of the list in conan_data.yml it would be:

from conan import ConanFile

class PyReq(ConanFile):

    name = "py_req"
    version = "1.0"
    settings = "os", "arch", "compiler", "build_type"

    def init(self):
        opts = self.conan_data.get("options", {})
        self.options.update(opts)
        for k, v in opts.items():
            setattr(self.options, k, v[0])

    def generate(self):
        self.output.info(f"NATIVE: {self.options.native}")
        self.output.info(f"WITH SSL: {self.options.with_ssl}")