Add an OEP defining the use of OpenFeature for feature toggles

[blarghmatey]

No description provided.

[openedx-webhooks]

Thanks for the pull request, @blarghmatey!

This repository is currently maintained by @sarina.

Once you've gone through the following steps feel free to tag them in a comment and let them know that your changes are ready for engineering review.

🔘 Get product approval

If you haven't already, check this list to see if your contribution needs to go through the product review process.

• If it does, you'll need to submit a product proposal for your contribution, and have it reviewed by the Product Working Group.
  • This process (including the steps you'll need to take) is documented here.
• If it doesn't, simply proceed with the next step.

🔘 Provide context

To help your reviewers and other members of the community understand the purpose and larger context of your changes, feel free to add as much of the following information to the PR description as you can:

• Dependencies

  This PR must be merged before / after / at the same time as ...

• Blockers

  This PR is waiting for OEP-1234 to be accepted.

• Timeline information

  This PR must be merged by XX date because ...

• Partner information

  This is for a course on edx.org.

• Supporting documentation
• Relevant Open edX discussion forum threads

🔘 Get a green build

If one or more checks are failing, continue working on your changes until this is no longer the case and your build turns green.

---

Where can I find more information?

If you'd like to get more details on all aspects of the review process for open source pull requests (OSPRs), check out the following resources:

• Overview of Review Process for Community Contributions
• Pull Request Status Guide
• Making changes to your pull request

When can I expect my changes to be merged?

Our goal is to get community contributions seen and reviewed as efficiently as possible.

However, the amount of time that it takes to review and merge a PR can vary significantly based on factors such as:

• The size and impact of the changes that it introduces
• The need for product review
• Maintenance status of the parent repository

💡 As a result it may take up to several weeks or months to complete a review and merge your PR.

[sarina]

Hi @blarghmatey, thank you for your submission!

I added a few style comments to the doc, hopefully they'll help fix the build. I only skimmed for style concerns in this pass.

Any thoughts as to who might make a good Arbiter for this OEP? They'll help drum up reviewers, post to the forums, etc as per OEP-1. Let me know how I can help!

[robrap]

Thanks. Some questions:

1. Is it expected that someone needs to read and become familiar with all of https://openfeature.dev/ as part of accepting this proposal, or is it possible to summarize what we are taking from the spec?
2. If the original OEP still stands, and this is just an extension of it, it still might make sense to add a note and link in the original toggle OEP.
3. OpenFeature has features like Hooks and Events. Are those compatible with chained providers? Would we need to explain in which ways these could and could not be used?
4. Does OpenFeature have details around how toggles should be annotated that would override any of the ADRs or How-Tos in edx-toggles? See https://edx.readthedocs.io/projects/edx-toggles/en/latest/readme.html.

[blarghmatey]

Thanks for those clarifying questions, that's helpful. I'll review them and update the OEP accordingly.

[kdmccormick]

Thanks for opening this @blarghmatey .

Separate but related: Feanil and I are currently trying to radically simplify our edx-platform settings. Any input or help you can provide there is appreciated.

[kdmccormick]

Agreed on the motivation. It is currently too hard to determine whether any given Open edX feature is enabled and how to toggle it on/off. The naming is inconsistent, there are many outdated toggles, and the interplay between Django settings and the various kinds of Waffle flags gets confusing. The edx-toggles abstraction layer was intended to smooth things out, but it seems to have only been partially rolled out, so I currently see it as only adding to the complexity.

Tutor has both made this better (by automating the toggling for most deployers) and worse (by adding another layer of abstraction for us to worry about).

[kdmccormick]

I'm seeking more detail on how we would use OpenFeature. The spec is quite involved and it'll take me some time to absorb their model of providers, values, and evaluation. Any code snippets of what you're thinking, even pseudo/informal code snippets, would be really helpful.

Does this replace or wrap Django settings? Same question site/org/course/experiment waffle flags. My understanding is that this would be a wrapper layer, but correct me if I'm wrong there. I am also presuming that this would replace edx-toggles so that we don't have two separate abstraction layers wrapping our toggles, but again, tell me if you're thinking otherwise.

Lastly, does this imply a separate microservice, or is it a library we install into edx-platform and/or the frontend apps?

[blarghmatey]

This would be a library that gets included in all of the IDAs and MFEs so that there is a single interface for setting and evaluating flag/toggle values. The default provider implementation will fall back to the current setup of waffle flags and Django settings for IDAs and build-time environment variables for MFEs. The benefit that it provides is that site operators can have more flexibility in how they manage those values if they already have an investment in something like LaunchDarkly, Optimizely, PostHog, etc.

[robrap]

Regarding customized toggle capabilities, like CourseWaffleFlag, is the idea that an operator could either use a custom tool and lose all the custom capabilities OR use the native implementation and keep all the additional features? Is this a decision that an operator can make on a toggle-by-toggle basis, or is it all or nothing when moving to some other tool?

[blarghmatey]

That's a good question. The CourseWaffleFlag is set up to return a value based on a course_key that is passed in. The OpenFeature spec allows for "evaluation options" which could be used to pass the course key. The provider implementation would then be able to evaluate the flag state with that context included, so there would be no loss of functionality. Again, the default provider implementation for a vanilla open edX deployment would be to use the django-waffle implementation so that it is backwards compatible with existing installations.

[blarghmatey]

Also, on the note of edx-toggles, I imagine that it could be evolved to be the implementation of the OpenFeature interface for Python-side toggles. That way it would reduce the overall effort of implementation since it is already a point of integration for the edX ecosystem.

[kdmccormick]

That's a good question. The CourseWaffleFlag is set up to return a value based on a course_key that is passed in. The OpenFeature spec allows for "evaluation options" which could be used to pass the course key. The provider implementation would then be able to evaluate the flag state with that context included, so there would be no loss of functionality. Again, the default provider implementation for a vanilla open edX deployment would be to use the django-waffle implementation so that it is backwards compatible with existing installations.

Also, on the note of edx-toggles, I imagine that it could be evolved to be the implementation of the OpenFeature interface for Python-side toggles. That way it would reduce the overall effort of implementation since it is already a point of integration for the edX ecosystem.

Perfect. My concern when I first read the OEP was that we'd be adding a new a wrapper (OpenFeature) without removing an existing wrapper (edx-toggles) and without taking into account database-dependent toggles (coursewaffle, site-configuration, and django-config-models). But, if this proposal is essentially:

• Back/replace edx-toggles with OpenFeature
• Finish rolling out edx-toggles/OpenFeature to the various types of existing toggles: django, waffle, configmodels
• Use OpenFeature contexts to implement context-aware models: coursewaffle, orgwaffle, site config)

then I could be convinced to back this proposal.

[itsjeyd]

Hey @blarghmatey, what are the next steps for this PR? Are you waiting for another round of review?

[feanil]

@itsjeyd, I've been asked to Arbiter this and I have failed to do so yet, I'm gonna post on the forums now to get this generally reviewed and figure out next steps for it.

[feanil]

Suggested change

.. _pep_based_template:

[feanil]

I've posted this on the forums for comment over the next month: https://discuss.openedx.org/t/oep-68-openfeature-toggles/15498

[bradenmacdonald]

I think that a lot of the useful context from the discussion above is missing in the text of the OEP itself. From reading the draft OEP, it's unclear to me how this would work. It sounds like it just provides a new API to be used when consuming a config value:

    # In your Django settings.py
    FEATURE_FLAG_EXAMPLE = True

Old API:

    from django.conf import settings
    is_enabled = settings.FEATURE_FLAG_EXAMPLE

New API:

    import openfeature.api
    # In your application code
    client = openfeature.api.get_client("edx")
    openfeature.api.set_provider(DjangoSettingsProvider())
    # Evaluate a flag
    is_enabled = client.get_boolean_value("FEATURE_FLAG_EXAMPLE", False)

This looks like a step backwards, and it implies that it only changes the "consuming config" side of the equation, not the "setting config" side. So I think the text of the API needs to reflect that:

• The use of the new API is simpler than it looks in this example (client/provider setup is done once per IDA ?)
• We can remove edx-toggles and perhaps other code, replacing it with this
• While operators could continue to use waffle/django/messy config as today (obviously needed for back compat), this OEP would allow them to replace all that with a nice new consolidated management tool (LaunchDarkly, Optimizely, PostHog ??) - an example showing the benefits / consolidation would be great.

[bradenmacdonald]

Suggested change

from django.conf import settings

This line doesn't seem to be doing anything?

[bradenmacdonald]

This seems to be against best practice - the default value of FEATURE_FLAG_EXAMPLE as well as the literal string value of the flag name should be defined in one place, and not potentially defined separately in every place where the value is consumed. I know this example isn't necessarily saying that, but it's ambiguous and we want to discourage that.

The current approach of annotating toggles with explicit metadata will be maintained.

If that's the case (which is great), I think the example code should reflect that and include the toggle metadata.

[bradenmacdonald]

A big problem with toggles in our codebase today is that they are planned to be temporary but never removed.

Actual examples from the codebase today:

# .. toggle_target_removal_date: 2020-01-31
# .. toggle_target_removal_date: 2020-06-01
# .. toggle_target_removal_date: 2020-06-01
# .. toggle_target_removal_date: 2021-05-01
# .. toggle_target_removal_date: 2021-05-15
# .. toggle_target_removal_date: 2021-12-31
# .. toggle_target_removal_date: 2022-1-30
# .. toggle_target_removal_date: 2022-1-30
# .. toggle_target_removal_date: 2022-05-30
... many more

I'm just curious if this OEP would provide any way to help this problem, or no?

[robrap]

• My understanding is that there are no reporting features, and we'd continue our support of the annotations separately. @blarghmatey can correct me if I'm wrong.
• That said - I have an agenda item to discuss this issue with the DEPR WG. There is also this old ticket: Improve DEPR process for toggles edx-toggles#283. @bradenmacdonald - if you have any thoughts or wish to help move this forward to help stop this problem from growing and/or help it shrink - that would be great.

[itsjeyd]

Hey @blarghmatey, just checking in to see if you're still planning to continue working on this PR?