Snakemake report failing due to deletion of next branch

[skchronicles]

Hello there,

Thank you for creating such an awesome tool. Vega is an amazing data visualization library.

While using another tool (snakemake/7.32.4), I noticed an issue that seems to be an issue related to the existence of the next branch within your repository. Older versions of snakemake appear to be using this branch to download your license. The license is then embedded in their reports. The issue is described here in more detail.

Would it be possible to re-create the next branch again? I am not sure if this branch gets created or destroyed as part of your ci. I am sorry if this causes any troubles on your side. Snakemake is an amazing tool if you haven't used it, and it would be awesome if the reporting feature worked in older versions.

Please let me know what you think.

Best regards,
@skchronicles

[domoritz]

We don't use the next branch anymore since it was used for an outdated release process. We don'e ever use it again. Does that help with fixing snakemake? I don't really want to restore an old branch to fix a bug in another tool. Does the fact that this branch is not used help fix the issue?

[skchronicles]

Hey @domoritz,

I hope you are having a great day and all is going well on your side. Thank you for the quick reply, and thank you for taking some time out of your day to look into this issue!

We don't use the next branch anymore since it was used for an outdated release process. We don'e ever use it again.

Okay, I see. That makes sense.

Does that help with fixing snakemake? I don't really want to restore an old branch to fix a bug in another tool. Does the fact that this branch is not used help fix the issue?

Older versions of snakemake were pulling your LICENSE information to embed within the report using this URI on the next branch:

Because the next branch no longer exists, the built-in reporting feature of this tool no longer works. It results in the error below to occur because it can no longer find the LICENSE file (resulting in a non-zero exit code):

Creating report...
WorkflowError:
Failed to download resource needed for report: https://raw.githubusercontent.com/vega/vega-embed/next/LICENSE

I totally get and understand this is not really your problem, but I just wanted to reach out to see if it would be possible to restore this branch as so many people within the greater scientific community use snakemake and are stuck using older versions of the tool. Hindsight is 20/20, and I am sure if the maintainers of snakemake knew that branch would no longer exist at some point in the future, they wouldn't have used that method to get your license information.

With that being said, I whole-heartedly respect your final decision as to whether you decide to restore this branch. Vega and altair are amazing visualization libraries that I will continue to use. If this is something you do not want to restore, I can ask the sysadmins of our effected HPC systems to patch their snakemake installations. I just wanted to reach out to you because this issue will likely impact a significant proportion of snakemake users that are still using older versions of the tool.

Thank you again for all you hard work, and countless hours you spend to help maintain this awesome tool.

Best regards,
@skchronicles

[domoritz]

Thanks for the in-depth explanation. I can temporarily reinstate the next branch, but it would be good for snakemake to update all references from next to be to main instead. Is that reasonable to have next reinstated for sometime but eventually deleted it?

[skchronicles]

Hey @domoritz,

Yes, thank you, that would be perfect!

I think this issue may be fixed in latest version of snakemake, but I have not tested it out yet. That is something I can test out on my side and look into more. A lot of my snakemake workflows are using the previous major version of the tool (7.X) due to breaking changes that were introduced with the latest major version (8.X) related to job submission in HPC environments. Either way, this grace period will give me some time to port the code base for all of our pipelines to work with the latest version of the tool.

Thank you again for your help. Please let me know if you have any questions.

Best regards,
@skchronicles

[domoritz]

I restored next for now in https://github.com/vega/vega-embed/tree/next but plan to delete it again in a few months. I understand the difficulties of updating dependencies in complex environments.

[skchronicles]

Hey @domoritz,

Thank you for temporarily restoring the next branch in vega-embed.

I just had some time to look into this more. It looks like this specific issue was fixed in snakemake/8.20.5 so any users using snakemake>=8.20.5 will should not be effected by this problem.

Thank you again for temporarily restoring the next branch though. This will give myself and other users some time to port and test our workflows over from snakemake 7.X to the latest release of 8.X.

Best regards,
@skchronicles