Skip to content

fix: Enhance settings processing to support nested dictionaries. #433

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 8 commits into
base: master
Choose a base branch
from

Conversation

Akanshu-2u
Copy link
Contributor

@Akanshu-2u Akanshu-2u commented Jul 8, 2025

Problem Description

Boolean toggles nested deeply in Django settings dictionaries (3+ levels) were not being detected by the Toggle State API endpoint /api/toggles/v0/state/ .

Root Cause

The _add_settings() function in report.py only processed dictionaries one level deep, missing boolean values nested deeper in the configuration hierarchy.

Fix

Enhanced the _add_settings() function with recursion to traverse arbitrarily deep nested dictionaries:

  • Added recursive _add_setting_recursively() helper function.
  • Maintains proper bracket notation naming using existing setting_dict_name() function.
  • Preserves all existing functionality while extending deep dictionary support.
  • Future-proof for any depth of nested configurations.

Conclusion

      - Number of toggle dictionaries in previous implementation : 321
      - Number of toggle dictionaries in current implementation: 524 

Example:

      -     {
                "name": "EVENT_BUS_PRODUCER_CONFIG['org.openedx.content_authoring.xblock.deleted.v1']['course-authoring-xblock-lifecycle']['enabled']",
                 "is_active": false
             },
      -       {
                "name": "EVENT_BUS_PRODUCER_CONFIG['org.openedx.content_authoring.xblock.duplicated.v1']['course-authoring-xblock-lifecycle']['enabled']",
                 "is_active": false
             },
      -       {
                "name": "EVENT_BUS_PRODUCER_CONFIG['org.openedx.content_authoring.xblock.published.v1']['course-authoring-xblock-lifecycle']['enabled']",
                 "is_active": false
             },
      -       {
                 "name": "EVENT_BUS_PRODUCER_CONFIG['org.openedx.enterprise.learner_credit_course_enrollment.revoked.v1']['learner-credit-course-enrollment-lifecycle']['enabled']",
                 "is_active": true
             },

Problem encountered after the fix: Duplicate Detection
The recursive fix is detecting boolean values from multiple copies of the same configuration that exist in different settings dictionaries.

For example:
Same boolean detected in 3 different places:

      - "CACHES['default']['OPTIONS']['ignore_exc']": true
      - "AUTH_TOKENS['CACHES']['default']['OPTIONS']['ignore_exc']": true  
      - "ENV_TOKENS['CACHES']['default']['OPTIONS']['ignore_exc']": true


      - "DATABASES['default']['ATOMIC_REQUESTS']": true
      - "AUTH_TOKENS['DATABASES']['default']['ATOMIC_REQUESTS']": true
      - "ENV_TOKENS['DATABASES']['default']['ATOMIC_REQUESTS']": true

Further approach
Working to fix this till then it is suggested not to merge.

Open to suggestions regarding the fixes to the current problem.

TIcket
https://2u-internal.atlassian.net/browse/BOM2-24?atlOrigin=eyJpIjoiYTFhNzZkN2Q2ZWFiNDE5YTg4YTMxMTllYTQ2ZDE5OWEiLCJwIjoiaiJ9

@Akanshu-2u Akanshu-2u changed the title fix: Enhance settings processing to support nested dictionaries.(Do not merge) [Do not merge] fix: Enhance settings processing to support nested dictionaries. Jul 11, 2025
@robrap
Copy link
Contributor

robrap commented Jul 16, 2025

High-level feedback:

  1. Please add a test.
  2. We could consider ticketing the duplicate issue, but you did not cause this issue, so it is independent of this PR.

@Akanshu-2u Akanshu-2u changed the title [Do not merge] fix: Enhance settings processing to support nested dictionaries. fix: Enhance settings processing to support nested dictionaries. Jul 17, 2025
@Akanshu-2u Akanshu-2u requested a review from regisb July 17, 2025 11:42
"""
settings_dict_copy = {}
default_attribute_value = object() # default is not a bool, so won't be included
for attr in dir(settings):
if not attr.startswith('__'):
value = getattr(settings, attr, default_attribute_value)
settings_dict_copy[attr] = value
print("hello:settings_dict_copy", settings_dict_copy)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need to remove this.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, got it. Was just for testing I will remove this.

Test that deeply nested dictionary-based toggles are properly extracted.
"""

def test_deeply_nested_event_bus_producer_config_extraction(self):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This test is really bloated, we should really simplify it to make sure that it's a unit test, not a full integration test. I fail to see the benefit of testing a dozen settings at once. Just a single nested dictionary with a single entry should suffice.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Following the suggestion, created a single class with two functions checking level 1 nesting and level 4 nesting.

self.assertNotIn(non_boolean_key, settings_dict)


if __name__ == '__main__':
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this needed?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It was not needed so removed it.

@@ -0,0 +1,416 @@
"""
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we really need to create a new test module? Wouldn't it make more sense to add unit tests to test_setting_toggles.py?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Created the class within test_setting_toggles.py.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems like other tests for state/internal/report.py live in test_state.py. I agree we don't want a new file, but maybe this is the right test file? You can see how to best incorporate the tests. But maybe I'm missing something.

@Akanshu-2u Akanshu-2u force-pushed the akanshu/BOM2-24-bug-in-toggle-reporting-for-event-bus-toggles branch from d220a6f to 9ef6749 Compare July 17, 2025 13:46
@Akanshu-2u Akanshu-2u requested a review from regisb July 18, 2025 09:46
Copy link
Contributor

@robrap robrap left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the other updates. It looks much closer. Some minor improvements recommended.

Comment on lines 90 to 95
"""
Test that Level 1 nested boolean values are correctly extracted.

Level 1: CONFIG['simple_toggle']
"""
# Level 1 nesting: CONFIG['simple_toggle']
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removing unnecessary comments. This is clear from the test itself.

Suggested change
"""
Test that Level 1 nested boolean values are correctly extracted.
Level 1: CONFIG['simple_toggle']
"""
# Level 1 nesting: CONFIG['simple_toggle']
"""
Test that Level 1 nested boolean values are correctly extracted.
"""

_add_setting(settings_dict, nested_config, 'CONFIG')

# Test Level 1 nesting
level1_toggle = "CONFIG['simple_toggle']"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: I like using expected sometimes to make it more clear that this isn't being used for setup, but just as an expected value. This is just an opinion, so no change is necessary if you don't like the suggestion.

Suggested change
level1_toggle = "CONFIG['simple_toggle']"
expected_level1_toggle = "CONFIG['simple_toggle']"

'advanced': {
'settings': {
'notifications': {
'enabled': False
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because of how Python treats Falsey stuff, I think True will be a slightly better test.

Suggested change
'enabled': False
'enabled': True

Comment on lines 232 to 233
Fill the `settings_dict`: will only include values that are set to true or false.
Now supports arbitrarily deep nested dictionaries with recursion.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. The first line of a docstring should be a 1-liner summary.
  2. I removed the word "Now". No one needs to know that it didn't support this in the past. I try to avoid temporal comments.
Suggested change
Fill the `settings_dict`: will only include values that are set to true or false.
Now supports arbitrarily deep nested dictionaries with recursion.
Fill the `settings_dict`: will only include values that are set to true or false.
Supports arbitrarily deep nested dictionaries with recursion.

@Akanshu-2u Akanshu-2u requested a review from robrap July 24, 2025 14:38
@robrap
Copy link
Contributor

robrap commented Jul 24, 2025

[request] In addition to the possible test move:

  1. Please add a CHANGELOG entry for 5.4.1, and
  2. Update the version to 5.4.1:
    __version__ = '5.4.0'

    I know Régis tends to do the version bump separately, but I like to just merge and release when ready.
    Thank you.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants