Skip to content

Commit

Permalink
Add files for ZEP(Zarr Enhancement Proposal)
Browse files Browse the repository at this point in the history
  • Loading branch information
sanketverma1704 committed Mar 14, 2022
1 parent 9c933f5 commit f374ced
Show file tree
Hide file tree
Showing 5 changed files with 280 additions and 0 deletions.
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
.DS_Store
4 changes: 4 additions & 0 deletions ZEP/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
# ZEP
Zarr Enhancement Proposal

This folder contains instructions, template and ZEP 0 files.
Binary file added ZEP/assets/flowchart.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
201 changes: 201 additions & 0 deletions ZEP/instructions/zep0000.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,201 @@
# Zarr Community Feeback Process - Zarr Enhancement Proposal (ZEP)

---

## ZEP 0 — Purpose and process

Author: Sanket Verma <[email protected]>

Status: Active

Type: Process

Created: 2022-14-03

## What is ZEP?

ZEP stands for Zarr Enhancement Proposal. A ZEP is a design document providing information to the Zarr community, or describing a new feature for Zarr or its
processes or environment. The ZEP should provide a concise technical specification of the feature and a rationale for the feature.

We intend ZEPs to be the primary mechanisms for proposing major new features, for collecting community input on an issue, and for documenting the design
decision that has gone into Zarr. The ZEP author is responsible for building consensus within the community and documenting dissenting opinions.

Because the ZEPs are maintained as text files in a versioned repository, their revision history is the historical record of the feature proposal.

## ZEP audience

The typical primary audience for ZEPs is the developers working on Zarr and its various implementations, the Zarr steering council as well as the Zarr community.

The broader Zarr community may also choose to use the process to document expected feature additions and to manage complex design coordination problems that
require collaboration across multiple projects.

## ZEP types

> Standard Track ZEP
Describes a new feature or implementation for Zarr

> Informational ZEP
Describes a ZEP design issue or provides general guidelines or information to the Zarr community but does not propose a new feature. Informational ZEPs do not
necessarily represent Zarr community consensus or recommendation, so users and implementers are free to ignore informational ZEPS or follow their advice.

> Process ZEP
Describes a new process around Zarr and its implementations, or proposes a change to(or an event in) a process. Process ZEPs are like Standard Track ZEPs but apply to areas other than the Zarr project itself.
They may propose an implementation, but not to Zarr’s codebase; they require community consensus; unlike informational ZEPs, they are more than recommendations, and users are typically not free to ignore them.
Examples include procedures, guidelines, changes to the decision-making process, and changes to the tools or environment used in Zarr’s development. Any meta-ZEP is also considered a Process ZEP.

## ZEP Workflow

The ZEP process begins with a new idea for Zarr. It is highly recommended that a single ZEP contain a single key proposal or new idea. Small enhancements or
patches often don’t need a ZEP and can be injected into the Zarr development workflow with a pull request to the Zarr ZEP repo. The more focused the ZEP,
the more successful it tends to be. If in doubt, split your ZEP into several well-focused ones.

Each ZEP must have a champion—someone who writes the ZEP using the style and format described below, shepherds the discussions in the appropriate forums,
and attempts to build community consensus around the idea. The ZEP champion (a.k.a. Author) should first attempt to ascertain whether the idea is suitable for a ZEP.
Posting to the [Gitter](https://gitter.im/zarr-developers/community) is the best way to go about doing this.

Vetting an idea publicly before going as far as writing a ZEP is meant to save the potential author time. Asking the Zarr community first if an idea is original
helps prevent too much time being spent on something that is guaranteed to be rejected based on prior discussion. It also helps to make sure the idea applies to
the entire community and not just the author. Just because an idea sounds good to the author does not mean it will work for most people.

Once the champion has asked the Zarr community whether an idea has any chance of acceptance, a draft ZEP should be presented to the appropriate venue mentioned
above. This gives the author a chance to flesh out the draft ZEP to make properly formatted, of high quality, and to address initial concerns about the proposal.

After the PR for the ZEP is in place, a post should be made on the Gitter channel containing the sections up to “Backward compatibility”, to limit discussion
there to usage and impact. Discussion on the pull request will have a broader scope, also including details of implementation.

Standards Track ZEPs consist of two parts, a design document and a reference implementation. It is generally recommended that at least a prototype implementation
be co-developed with the ZEP, as ideas that sound good in principle sometimes turn out to be impractical when subjected to the test of implementation.
Often it makes sense for the prototype implementation to be made available as PR to the Zarr repo (making sure to appropriately mark the PR as a WIP).

### Submitting a ZEP

The proposal should be submitted as a draft ZEP via a GitHub pull request to the zarr-developers/zep directory with the name `zep-<n>.rst` where `<n>` is an appropriately assigned four-digit number (e.g., zep-0000.rst). The draft ZEP must use the ZEP X - Template and Instructions file.

A few points to consider while submitting your ZEP:

- It should sound complete. The ideas must make technical sense.
- The title should accurately describe the content.
- The ZEP’s language (spelling, grammar, sentence structure etc.) and code style should be correct and conformant.

The [Zarr Steering Council](https://github.com/zarr-developers/governance/blob/master/GOVERNANCE.md#steering-council) will not unreasonably deny publication of a ZEP. Reasons for denying ZEP include duplication of effort, being technically unsound, not providing proper
motivation or addressing backwards compatibility, or not taking care of Zarr [CODE OF CONDUCT](https://github.com/zarr-developers/.github/blob/main/CODE_OF_CONDUCT.md).

### Discussing a ZEP

As soon as the draft ZEP is committed to the ZEP repository, a discussion thread for the ZEP should be created to provide a central place to discuss and review
its contents. The ZEP author(s) may select the [GitHub discussion feature](https://github.com/zarr-developers/zarr-python/discussions) in the zarr-python repository,
open a new issue in the Zarr community repository or start the discussion in the Zarr Gitter channel. The discussion regarding the ZEP should follow Zarr’s
[CODE OF CONDUCT](https://github.com/zarr-developers/.github/blob/main/CODE_OF_CONDUCT.md) at all times.

ZEP authors are responsible for collecting community feedback on a ZEP. However, to avoid long-winded and open-ended discussions, strategies such as soliciting
private or more narrowly-tailored feedback in the early design phase, collaborating with other community members with expertise in the ZEP’s subject matter,
and picking appropriately-specialised discussion for the ZEP’s topic should be considered. ZEP authors should use their discretion here.

Once the ZEP is committed to the ZEP repository, substantive issues should generally be discussed on the canonical public thread, as opposed to private
channels or unrelated venues. This ensures everyone can follow and contribute, avoids fragmenting the discussion, and makes sure it is fully considered as part
of the ZEP review process. Comments, support, concerns and other feedback on this designated thread are a critical part when reviewing the ZEP.

### Review and Resolution

The possible paths of the status of ZEPs are as follows:

![Flowchart](/ZEP/assets/flowchart.png)

All ZEPs should be created with the `Draft` status.

Eventually, after the discussion, there may be a consensus that the ZEP should be accepted - see the next section for details. At this point, the status becomes
`Accepted`.

Once a ZEP has been `Accepted`, the reference implementation must be completed. When the reference implementation is complete and incorporated into the main source
code repository, the status will be changed to `Final`.

To allow gathering of additional design and interface feedback before committing to long term stability for a feature or standard library API, ZEP may also be
marked as `“Provisional”`. This is short for “Provisionally Accepted”, and indicates that the proposal has been accepted for inclusion in the reference
implementation, but additional user feedback is needed before the full design can be considered “Final”. Unlike regular accepted ZEPs, provisionally accepted
ZEPs may still be Rejected or Withdrawn even after the related changes have been included in a Zarr release.

Wherever possible, it is considered preferable to reduce the scope of a proposal to avoid the need to rely on the `“Provisional”` status (e.g. by deferring some features to later ZEPs), as this status can lead to version compatibility challenges in the wider Zarr ecosystem.

A ZEP can also be assigned status `Deferred`. The ZEP author or [Zarr Steering Council](https://github.com/zarr-developers/governance/blob/master/GOVERNANCE.md#steering-council) can assign the ZEP this status when no progress is being made on the ZEP.

A ZEP can also be `Rejected`. Perhaps, after all, is said and done it was not a good idea. It is still important to have a record of this fact. The `Withdrawn`
status is similar—it means that the ZEP author themselves has decided that the ZEP is a bad idea, or has accepted that a competing proposal is a better alternative.

When a ZEP is `Accepted`, `Rejected`, or `Withdrawn`, the ZEP should be updated accordingly. In addition to updating the status field, at the very least the
`Resolution` header should be added with a link to the relevant link of the discussion.

ZEPs can also be `Superseded` by a different ZEP, rendering the original obsolete. The `Replaced-By` and `Replaces` headers should be added to the original and
new ZEPs respectively.
Process ZEPs may also have a status of `Active` if they are never meant to be completed, e.g. ZEP 0 (this ZEP).

### How does a ZEP becomes accepted?

A ZEP is `Accepted` by the consensus of all interested contributors. We need a concrete way to tell whether consensus has been reached. When you think ZEP is ready to accept, create a new discussion in [zarr-python](https://github.com/zarr-developers/zarr-python/discussions) using ‘General’ as the category with a subject like:

> `Proposal to accept ZEP <number>: <title>`
In the body of your discussion, you should:

- Link to the latest version of ZEP,
- Briefly describe any major points of contention and how they were resolved,
- Include a sentence like: “If there are no substantive objections within 7 days from this post, then the ZEP will be accepted;

After you create the discussion, you should make sure to link the newly created thread in the `Discussion` section of the ZEP, so that people can find it later.

Generally, the ZEP author will be the one to create this post, but anyone can do it – the important thing is to make sure that everyone knows when a ZEP is on
the verge of acceptance, and give them a final chance to respond. If there’s some special reason to extend this final comment period beyond `7 days`, then that’s fine, just say so in the post. You shouldn’t do less than `7 days`, because sometimes people are travelling or similar and need some time to respond.

In general, the goal is to make sure that the community has consensus, not provide a rigid policy for people to try to game. When in doubt, err on the side of
asking for more feedback and looking for opportunities to compromise.

If the final comment period passes without any substantive objections, then the ZEP can officially be marked `Accepted`. You should create a follow-up discussion thread notifying everyone (celebratory emoji optional but encouraged 🎉✨), and then update the ZEP by setting its `:Status:` to `Accepted`, and
it's `:Resolution:` header to a link to your follow-up discussion thread.

If there are substantive objections, then the ZEP remains in `Draft` state, discussion continues as normal, and it can be proposed for acceptance again later
once the objections are resolved.

In unusual cases, the [Zarr Steering Council](https://github.com/zarr-developers/governance/blob/master/GOVERNANCE.md#steering-council) may be asked to decide
whether a controversial ZEP is `Accepted`.

### Maintenance

In general, Standard Track ZEPs are no longer modified after they have reached the Final state as the code and project documentation are considered the ultimate
reference for the implemented feature. However, finalised Standard track ZEPs may be updated as needed.

Process ZEPs may be updated over time to reflect changes to development practices and other details. The precise process followed in these cases will
depend on the nature and purpose of the ZEP being updated.

## ZEP Format

ZEPs are UTF-8 encoded text files using the [reStructureText](http://docutils.sourceforge.net/rst.html) format. Please see the ZEP X - Template and Instructions file and the [reStructuredTextPrimer](https://www.sphinx-doc.org/en/stable/rest.html) for more information.

### Header Preamble

```
:Author: <list of authors’ real names and email addresses>
:Status: < Draft | Active | Accepted | Deferred | Rejected | Withdrawn | Final | Superseded >
:Type: <Standards Track | Process>
:Created: <date created on, in dd-mmm-yyyy format>
:Require: <Previous ZEP number>
:Zarr-Version: <version number>
:Replaces: <ZEP number>
:Replaced-By: <ZEP number>
:Resolution: <Link to discussion thread>
```

The Author header lists the names and the email addresses of all the authors of the ZEP. The format of the Author header value must be:

```
Random J. User <[email protected]>
```

## Discussion

https://github.com/zarr-developers/zarr-python/discussions

## Copyright

This document has been placed in the public domain.
74 changes: 74 additions & 0 deletions ZEP/template/template.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,74 @@
# ZEP — Template and Instructions

---

```
Author: <list of authors’ real name and email addresses>
Status: < Draft | Active | Accepted | Deferred | Rejected | Withdrawn | Final | Superseded >
Type: <Standards Track | Informational | Process>
Created: <date created on, in dd-mmm-yyyy format>
Resolution: <url> (required for Accepted | Rejected | Withdrawn)
```

## Abstract

The abstract should be a short description of what the ZEP will achieve.

## Motivation and Scope

This section describes the need for the proposed change. It should describe the existing problem, who it affects, what it is trying to solve, and why.
This section should explicitly address the scope of and key requirements for the proposed change.

## Usage and Impact

This section describes how users of Zarr will use the features described in this ZEP. It should be comprised mainly of code examples that wouldn’t be possible
without acceptance and implementation of this ZEP, as well as the impact the proposed changes would have on the ecosystem. This section should be written from
the perspective of the users of Zarr, and the benefit it will provide them; as such, it should include implementation details only if necessary to explain the
functionality.

## Backward Compatibility

This section describes how the ZEP breaks backward compatibility.

Its purpose is to provide a high-level summary to users who are not interested in detailed technical discussion, but may have opinions around, e.g., usage and
impact.

## Detailed description

This section should provide a detailed description of the proposed change. It should include examples of how the new functionality would be used, intended
use-cases and pseudo-code illustrating its use.

## Related Work

This section should list relevant and/or similar technologies, possibly in other libraries. It does not need to be comprehensive, just list the major examples
of prior and relevant art.

## Implementation

This section lists the major steps required to implement the ZEP. Where possible, it should be noted where one step is dependent on another, and which steps may
be optionally omitted. Where it makes sense, each step should include a link to related pull requests as the implementation progresses.

Any pull requests or development branches containing work on this ZEP be linked to from here. (A ZEP does not need to be implemented in a single pull request if
it makes sense to implement it in discrete phases).

## Alternatives

If there were any alternative solutions to solving the same problem, they should be discussed here, along with a justification for the chosen approach.

## Discussion

This section should have links related to any discussion regarding the ZEP. It could be GitHub issues and/or discussions. (The links to discussions in past
if any, goes in this section.)

## References and Footnotes

Each ZEP must either be explicitly labelled as placed in the public domain (see this ZEP as an example) or licensed under the
[Open Publication License](https://www.opencontent.org/openpub/).

## Copyright

This document has been placed in the public domain.

0 comments on commit f374ced

Please sign in to comment.