[Python] Regression in accessing protected data members in ROOT 6.32 after cppyy upgrade

[guitargeek]

Check duplicate issues.

• Checked for duplicates

Description

Getting access to protected data members when inheriting from a C++ stopped working (for protected functions it still works).

Reproducer

import cppyy

cppyy.cppdef("""

class MyBaseClass {
public:
   virtual ~MyBaseClass() = default;
protected:
   int protectedFunc() { return 5; }
   int _protectedData = 4;
};

""")

class MyDerivedClass(cppyy.gbl.MyBaseClass):
    pass

my_obj = MyDerivedClass()

print(my_obj.protectedFunc()) # works!
print(my_obj._protectedData) # doesn't work with ROOT 6.32!

The reproducer fails on lxplus9 with ROOT 6.32.06 and Python 3.9.18

5
Traceback (most recent call last):
  File "/afs/cern.ch/user/r/rembserj/repro.py", line 21, in <module>
    print(my_obj._protectedData) # doesn't work!
AttributeError: 'MyDerivedClass' object has no attribute '_protectedData'

However, on lxplus8 with ROOT 6.30/08 and Python 3.6.8, it works.

ROOT version

ROOT >=6.32.00

Installation method

Any

Operating system

Any

Additional context

The priority of this issue is set to low, since I'm not aware of any people making use of that feature. The 6.32 release cycle came and went without any complaints at least.

[dpiparo]

Maybe a more fundamental question: what is the expected behaviour?
Are private and protected data members and functions meant to be accessible in Python? They are not in C++, they could change name, type, disappear and re-appear version after version - I may even claim that 6.30 was faulty and the correct behaviour was restored afterwards...

[guitargeek]

They are not in C++, they could change name, type, disappear and re-appear version after version

That's not really possible, because protected is public to subclasses. So we can't change protected data members as we like, because we have to assume that they are used in derived classes. That's also the reason why protected members used to be made accessible in cppyy: so that they can be used when deriving a Python class from a C++ class.

That being said, I agree with you that one can claim that 6.30 was faulty there, because it enabled you to access protected data members anywhere, and not only in the context of derived classes. So going back to that state is indeed not great. And it's impossible to have a mechanism in Python that makes it only possible to access these members from within the derived class implementation.

But then, at least it should be consistent between data members and member functions 🙂 So we can maybe fix this issue by also removing access to protected member functions?

[guitargeek]

Ok, thinking more about this I think giving users no way at all to access protected data would maybe be too restrictive.

So I was thinking of how to fix this regression.

The underlying reason is that in the C++ trampoline class that is implicitly created for inheriting from C++ in Python, the protected data members are turned into private data members by public: using BaseClass::_protectedData. That's completely valid C++.

However, the cppyy-backend based on ROOT meta doesn't recognize these using members correctly as public, which is why it doesn't work:

import cppyy

cppyy.cppdef("""

class BaseClass {
protected:
   int _protectedData = 4;
};

class Trampoline : public BaseClass {
public:
   using BaseClass::_protectedData;
};

// The data member is public in the trampoline
std::cout << Trampoline{}._protectedData << std::endl;

""")

my_obj = cppyy.gbl.Trampoline()

# doesn't work with ROOT but with compiler-research cppyy!
print(my_obj._protectedData)

Fortunately, this works with the new cppyy based on CppInterOp, at least when using clang-repl (hopefully also with Cling)! So this issue will go away itself with the upcoming cppyy transition.

FYI @Vipul-Cariappa, @aaronj0.