Render deliveries are timing out

[asmeurer]

See https://github.com/sympy/sympy/settings/hooks/400119645?tab=deliveries. This is causing the release notes bot to not deliver a status a lot of the time.

Oddly enough, several "closed" deliveries are marked as failed, but the bot still seems to have updated the release notes for them.

In some cases, though, they haven't been added, like sympy/sympy#25133.

[asmeurer]

I have no idea how to fix this. This wasn't ever a problem with Heroku. Maybe I should re-investigate the choice to use Render (#102 #101)

[asmeurer]

Maybe we can try adding a health check https://render.com/docs/deploys#zero-downtime-deploys https://render.com/docs/deploys#health-checks

[certik]

I started using the built-in github release notes: https://github.com/lcompilers/lpython/releases/tag/v0.15.0. It just lists the merged PRs. It's not perfect, but I don't have to maintain any bots.

[asmeurer]

Yeah, that's pretty crappy though. Compare the quality of that to https://github.com/sympy/sympy/wiki/Release-Notes-for-1.12.

If we want to move away from the bot we should use something on GitHub actions that does the same thing, or something similar (e.g., making it so you write the notes in a file in the repo instead of in the PR description).

It may be a good idea to do this as it's less maintenance than the bot, but it's also work to set that up. The bot should work as it is, except Render isn't actually doing the job correctly like Heroku used to.

I sent an email to Render support about this. Unfortunately, the alter alternative I tried, replit, also seemed to have this same problem #101 (comment), and it doesn't implement its own logging. So Render is definitely nicer, if it would actually work.

Of course the other alternative is to just go back to Heroku and pay for the paid plan. That's assuming Heroku still works as well as it used to. I believe this is what the conda-forge people are doing with their bot, for instance.

[certik]

Yes, the quality of the SymPy release notes is much higher, no doubt.

[asmeurer]

BTW, for a small project like LPython I would just write the release notes manually. I takes a few extra minutes, but it's worth it. For SymPy, we have so many changes per release that this is basically impossible.

[oscarbenjamin]

If we want to move away from the bot we should use something on GitHub actions that does the same thing, or something similar (e.g., making it so you write the notes in a file in the repo instead of in the PR description).

It may be a good idea to do this as it's less maintenance than the bot, but it's also work to set that up.

The current approach involves a lot of manual work during the release process because you need to create new release notes pages in the wiki add links to them, copy in the authors and sha etc. Some of that can be automated but it would be better if everything was just managed from within the sympy repo so that both the code for the bot and also the release notes themselves would be in the codebase. The release notes could then be part of the docs rather than on a separate web page.

I also think that release notes for a PR would usually get better review if they showed up in the diff.

It is as you say work to set that up though and potentially another barrier to new contributors (currently anyone merging the PR can easily just write/edit the release note before merging).

[asmeurer]

I'm +1 to moving to another system if someone wants to do the work to move over. I've been sticking with the bot because it's already set up. But while it's more annoying, even for existing contributors, to add notes to the PR itself, the system that manages it would be much more easy to maintain.

[certik]

At least half of the notes are one liners, and many of the two liners can be made one liner. Why not using the title of the PR with something like https://github.com/marketplace/actions/pull-request-title-rules to enforce which category in the release note it should show up?

Then for breaking changes (or some other categories) you can enforce having a release notes file as a diff. That way for most contributions just the title will be enough.

[skieffer]

Any interest in trying towncrier?

It's used, for example, by pip, and their rendered news file looks like this. Their contributing guide has nice notes on how to add a NEWS entry to a PR. Might also look at their CI for hints, such as checking a PR for presence of a news entry. Finally generating the news file seems pretty easy.

I don't know how hard it would be to reproduce everything that SymPy's bot currently does. There is a section on customization. If that's not enough, maybe we would need a custom fork.

[oscarbenjamin]

I think towncrier could work. It could produce output like:

Polys
-----

- Provide method to reduce an algebraic number
  to a small representative mod an ideal. #23409

The bot produces:

* polys

  * Provide method to reduce an algebraic number
    to a small representative mod an ideal.
    ([#23409](https://github.com/sympy/sympy/pull/23409)
    by [@skieffer](https://github.com/skieffer))

The difference is just linking directly to the PR and also giving the GitHub handle for the author. Discovering the GitHub handle is nontrivial for anything that is not running on GitHub itself but linking directly to the PR from the number should be easy.

Currently a significant stumbling block for new contributors is the .mailmap file. Maybe there could be a way to combine the towncrier inputs with whatever is needed for .mailmap which might make that easier.

[asmeurer]

NumPy also has a system that they use. We should look into that. The actual formatting isn't that important. Although I do think it's a good idea to sort the notes by submodule as we do now. Any other way of ordering them makes them useless for users since there are so many changes per release.

Anyway, let's move this discussion to the SymPy issue tracker.