Do we need to fork?

[jlopezpena]

Hi all!

Let me start apologising for the incendiary title, it was just a cheap way to get your attention -- but keep reading!

About two years ago I started running into issues with omegaconf/hydra in my repos due to conflicting antlr4 runtime versions. After a few discussions with the community, I went ahead and spearheaded the vendoring of the antlr4 runtime to avoid the conflict, and also pushed the relevant changes to hydra.

Both PRs were merged a very long time ago, but there hasn't been a proper release including those (and many other!) changes. Last hydra release was in February 2023, and last omegaconf official release was in December 2022 (albeit we got a "dev" release in February 2024 including my changes which I used to test the changes to hydra). Both @Jasha10, and @odelalleau have hinted that these projects are no longer actively maintained, and AFAIK @omry hasn't given any indication on the contrary.

This has been very frustrating for me, and by the number of posts and issues asking about new releases I am guessing it has been frustrating for many other people as well. In my case, frustration led me to stop using hydra/omegaconf altogether in my personal projects, in favor of other solutions like dynaconf.

However, I started a new gig last year, and there is plenty of code in our codebase that still relies on hydra, or dependencies that do (like dvc), which brings me here again, and thus to the title of my post.

Do we need a community fork of this project? omegaconf is still under omry's ownership, which I guess it's ok, but hydra seems to be under the umbrella of Facebook Research, with no development or maintenance effort coming from Meta. This is more worrying because if the company or the main devs lose interest in it (as they seem to have) there is no simple way for other people to step in and keep the ball rolling.

Perhaps it would be better to create a purpose-made githhub org to take ownership of both projects (or fork them if Meta wants to keep naming rights or whatever), and assemble a varied set of maintainers willing to take over the release burden?

I have two people in mind which would IMO be a good fit to take over:

• @astrojuanlu - one of the main devs at kedro, which relies pretty heavily on omegaconf
• @rsokl - creator of hydra-zen, which keeps using hydra and I can see is actively maintained

Sprry for throwing you under the bus, chaps, I just believe you have a vested interest in the omegaconf/hydra ecosystem that could be leveraged for the good of the community ;-)
I am sure there are many other community members that might want to step in and have a saying on these projects, as well as side projects that rely on either hydra or omegaconf and would thus also benefit from regular maintenance.

So, what do you all think? Do we give this a go, or should I just make my own stripped-down fork with the minimal functionality I need to keep my dependencies running?

[astrojuanlu]

Thanks for putting this together @jlopezpena !

I am equally worried about the fate of Omegaconf. I have very little faith that the project will see significant activity again (although I'd be happy to be proven wrong - we've seen worse in open source!)

I would love to see a coalition of actors step up for the maintenance of an Omegaconf fork (or even Omegaconf itself, if the maintainers consider the possibility of handing the PyPI keys to somebody else). However, I think the list of stakeholders is incomplete without mentioning huge corporate open source projects that also depend on Omegaconf:

• sagemaker-python-sdk https://github.com/aws/sagemaker-python-sdk/blob/31f34ddfcca3d98d044bf3ebd46ef3ed0faf7591/pyproject.toml#L43
• pytorch-lightning https://github.com/Lightning-AI/pytorch-lightning/blob/afa7d56eb7d6566af1bacc644435b7bde2e50487/requirements/pytorch/extra.txt#L6

So I'd love to see people from AWS and Meta comment on this.

Also DVC, which is well known among data scientists, depends on Omegaconf https://github.com/iterative/dvc/blob/75083cc472e0812a558819b2a7f4b220c67b691f/pyproject.toml#L59

---

diffusers is also listed as dependant in some sites, but they actually moved away from Omegaconf 1.5 years ago huggingface/diffusers#6488

Which takes me to the second part of my answer: is it time to move away from "YAML on steroids" for configuration? Is the pendulum swinging back to config as code, with things like Pydantic and such?

For example, fairseq depends on Omegaconf https://github.com/facebookresearch/fairseq/blob/d13e14a800bb588e5a77fb4e551f554ff9b24a72/setup.py#L183

...but fairseq2 doesn't! If Meta is no longer maintaining Omegaconf nor Hydra, what are they using instead?

(And yes, this is an open question we in Kedro need an answer for too.)

[odelalleau]

If there are dedicated people who are willing to step up to maintain OmegaConf / Hydra OSS, I may be able to help track down people at Meta to try and give them proper access rights so they can do this job (no promise though -- some key people have left and I doubt it'll be a priority for others, so it might be tricky to get some traction).

But the key question is -- would anyone actually be willing (and able) to do it?

[odelalleau]

Oh and for OmegaConf I might be able to reach out to @omry as well (fingers crossed)

[rsokl]

Hi all, I probably have the most up-to-date information about these repos. I am the creator of hydra-zen, and I reached out to jasha and omry a few months ago about the maintenance status of these projects. I am strictly interested in keeping the lights on for these so that hydra-zen continues to function smoothly.

Jasha and omry graciously discussed this with me. These projects are not actively maintained, but Hydra is still used internally at Meta some. They provided me with commit and pypi-publish privileges for omegconf, and commit rights for Hydra. I do not have pypi publish rights for Hydra, as Meta does not want to share tokens externally. That being said, I have a contact with someone at Meta who is happy to facilitate publishing Hydra releases. Omry's preference for Hydra is to not fork unless necessary.
.
That all being said, I have yet to find time to do work on either project even just to keep the lights on due to work and home-life demands.

My top priorities for both projects would be to:

• Get CI passing
  • For hydra, reduce the test surface area -- dropping forma support for 3rd party plugins
• Release new versions of omegaconf and hydra, with the main aim being shipping vendored antlr3 and adding support for python 3.12+

I wonder how this aligns with your priorities, @jlopezpena and @astrojuanlu ? Currently hydra-zen's CI passes for python 3.13, so none of the above is essential for me, which is one reason this hasn't risen to the level of actionable for me yet.

[astrojuanlu]

• Get CI passing
  • For hydra, reduce the test surface area -- dropping forma support for 3rd party plugins
• Release new versions of omegaconf and hydra, with the main aim being shipping vendored antlr3 and adding support for python 3.12+

Fantastic news!! Sounds perfect for us 👏🏼

[jlopezpena]

All that sounds great, thanks for the update!