Prototype requirement for Provisional acceptance

[rhc54]

Overview

It has come to my attention that we don't currently require a prototype implementation or example code before accepting a provisional RFC. I believe this is an oversight on our part and that such a requirement is important to ensure that even a provisional proposal is at least feasible.

Motivation

There currently is no requirement in section 1.3.3 that an RFC for provisional acceptance be accompanied by a prototype implementation (e.g., in OpenPMIx) or at least example code that shows how the proposed change would be used. The concern is that this could lead to changes in the Standard that later prove to be impossible to implement. Even though provisional status means it can be reverted rather quickly, it still causes issues for both users and implementers:

• implementers are faced with pressure to support a newly accepted feature, only to spend multiple days/weeks on it to discover that it isn't feasible
• users get frustrated and lose confidence in the Standard itself

Discussion Items

We did have some discussion about this before. Thinking was that provisional items could be quickly removed if they proved infeasible, so let's keep the barrier to entry low. However, I'm concerned that we now have a provisional item that looks reasonable on the surface, but has not been able to produce a prototype implementation despite nearly a year of off/on effort. This raises the concern that perhaps we have been too lenient - being left with the burden of implementing something that may prove impossible doesn't feel like a good position to be in. I'm proposing that we reconsider our position and put a slightly higher burden on those proposing a new feature.

[jjhursey]

Thanks for opening this issue. I've added this for discussion in our montly meeting this week.

[jjhursey]

Notes from March 11 teleconf:

• The community needs some assurance that an implementation of the proposed change is possible before acceptance. Otherwise, this will likely undermine the usefulness of the document. This is the overarching desire for requiring an implementation with any change to the document.
• The degree of completeness and openness of the implementation can be decided based on the community discussion on the PR. Flexibility is important in some of the following circumstances:
  • Does there need to be a resource manager implementation in addition to a PMIx implementation for attributes whose backing implementation happens in the RM?
  • What if the source code for that RM (or component to OpenPMix) is closed or otherwise inaccessible?
  • What if a full implementation is not possible yet, but will be in the near future? Can pseudo-code suffice for the remainder?
• A PR should clearly define the minimal set of requirements for an implementation that are expected from the authors of the PR.
• Do we need to add clarifications to the implementation requirements to the governance document or wiki or PR templates?
• We should add some guidance to the wiki regarding the responsibilities of the working groups. Some items mentioned:
  • Role of participating in the working group may require helping with implementation requirements for a PR. The implementation requirements should not fall on any one member (e.g., WG champion), but be taken by the WG as a part or whole.
  • Clarification on implementation requirements for PRs that add/remove/modify any functional aspects of the standard.
  • Discussion on how to start a new working group (e.g., logistics)
• Does the governance document language need to change? Section 1.3.3 "Proposed modification" in the Governance document v1.4.

Action Items:

• Complete a discussion on this topic on this GH Issue.
• Identify specific things we need to do to clearly communicate implementation requirements and WG expectations.

[abouteiller]

We wanted to revive this for the may 2023 meeting.

[abouteiller]

For reference, this is the current text governing implementation requirements (gov doc 1.7, section 1.3.3)

An open-source implementation of the proposal is available and a reference to the implementation included in the PR. A complete implementation is not necessarily required to meet this condition - e.g., pseudocode detailing the execution path supporting the proposed modification can be provided. More complete prototype implementations may be requested by the PMIx community when proposals involving significant changes in behavior are made.