Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Upgrade path for ml-verse (moving to R v 4.4)? #806

Closed
mskyttner opened this issue May 7, 2024 · 14 comments
Closed

Upgrade path for ml-verse (moving to R v 4.4)? #806

mskyttner opened this issue May 7, 2024 · 14 comments
Labels

Comments

@mskyttner
Copy link
Contributor

Container image name

rocker/ml-verse:4.3.2

Container image digest

No response

What operating system related to this question?

Linux

System information

  • Docker version 26.1.1, build 4cf5afa
  • Linux Mint 21.1 Vera

Question

I noticed an R vulnerability which seems to be mitigated if using R v 4.4.

In conjunction with making some updates of an image which is based on and extends ml-verse:4.3.2, I was therefore planning initially just to switch it to start off with rocker/ml-verse:4.4 but noticed that the ml-verse images are no longer updated...

I wonder if you have any advice or suggestions on the upgrade path I should take for moving to "rocker/ml-verse:4.4"?

I also wanted to ask about the images provided from ghcr.io (rather than the docker hub) - figuring using those from GitHub runners might provide some slight latency advantage when spun up in GHAs. Are the most recent variants of the rocker images served from docker hub or are the ones in the GitHub Container Registry equally "up-to-date"?

@eitsupi
Copy link
Member

eitsupi commented May 7, 2024

Sorry for bothering you.
I recently did a major rewrite of the repository and have not yet succeeded in building rocker/ml-verse.
Hopefully we will have a successful build by the end of the day.

I noticed an R vulnerability which seems to be mitigated if using R v 4.4.

I suggest you rethink whether that is really a reason to start using R4.4.0.
https://github.com/hrbrmstr/rdaradar

figuring using those from GitHub runners might provide some slight latency advantage when spun up in GHAs. Are the most recent variants of the rocker images served from docker hub or are the ones in the GitHub Container Registry equally "up-to-date"?

Yes.

@eitsupi
Copy link
Member

eitsupi commented May 7, 2024

I fixed the CI and triggered a build, but unfortunately it seems to be unable to build due to a bad connection to CTAN.
See the log https://github.com/rocker-org/rocker-versioned2/actions/runs/8985639266/job/24681391070

@cboettig I have seen too many build failures caused by latex.
Do you have any suggestions for a solution? I think it would be better to copy the contents of rocker/verse etc. in a multi-stage build to reduce the number of times such a high probability of failure step is executed.

@mskyttner
Copy link
Contributor Author

@eitsupi thanks for the update and the advice! Thanks for de-hyping that vuln, it doesn't appear to be too scary. There are other better reasons I guess for using R v 4.4, perhaps including the initially fuzzy feeling of being up-to-date and being able to support use of the fancy new %||% operator etc :). I see now ghcr.io/rocker-org/ml-verse:4.4.0, thanks so much!

@benz0li
Copy link
Contributor

benz0li commented May 7, 2024

I have seen too many build failures caused by latex.

@eitsupi Is it a bad connection to a CTAN mirror? My builds sometimes fail because of

tlmgr: Remote database (revision 71082 of the texlive-scripts package) seems to be older than the local installation (rev 71089 of texlive-scripts); please use a different mirror and/or wait a day or two.

Do you have any suggestions for a solution?

I simply set retry: 2 in my .gitlab-ci.yml

@eitsupi
Copy link
Member

eitsupi commented May 7, 2024

I see now ghcr.io/rocker-org/ml-verse:4.4.0, thanks so much!

Sorry, That tag is wrong. See #810.
(And thanks for making me aware that it had been pushed.)

@eitsupi
Copy link
Member

eitsupi commented May 7, 2024

I simply set retry: 2 in my .gitlab-ci.yml

Thanks, but I failed twice today, so retries don't seem to make sense to me.

@eitsupi
Copy link
Member

eitsupi commented May 7, 2024

This is the third time today that I have failed. Will not go any further, there seems to be a problem with the CTAN mirror.

https://github.com/rocker-org/rocker-versioned2/actions/runs/8987434399/job/24685907698#step:7:3077

#55 68.77 --2024-05-07 14:47:17--  https://mirror.ctan.org/systems/texlive/tlnet/install-tl-unx.tar.gz
#55 68.78 Resolving mirror.ctan.org (mirror.ctan.org)... 89.58.7.101
#55 83.79 Connecting to mirror.ctan.org (mirror.ctan.org)|89.58.7.101|:443... connected.
#55 84.01 HTTP request sent, awaiting response... 307 Temporary Redirect
#55 84.23 Location: https://ctan.math.washington.edu/tex-archive/systems/texlive/tlnet/install-tl-unx.tar.gz [following]
#55 84.23 --2024-05-07 14:47:33--  https://ctan.math.washington.edu/tex-archive/systems/texlive/tlnet/install-tl-unx.tar.gz
#55 84.23 Resolving ctan.math.washington.edu (ctan.math.washington.edu)... 128.95.224.254
#55 84.44 Connecting to ctan.math.washington.edu (ctan.math.washington.edu)|128.95.224.254|:443... connected.
#55 84.55 ERROR: cannot verify ctan.math.washington.edu's certificate, issued by ‘CN=InCommon RSA Server CA 2,O=Internet2,C=US’:
#55 84.55   Unable to locally verify the issuer's authority.
#55 84.55 To connect to ctan.math.washington.edu insecurely, use `--no-check-certificate'.
#55 ERROR: process "/bin/sh -c /rocker_scripts/install_verse.sh" did not complete successfully: exit code: 5

@benz0li
Copy link
Contributor

benz0li commented May 7, 2024

This is the third time today that I have failed. Will not go any further, there seems to be a problem with the CTAN mirror.

No.

Open https://ctan.math.washington.edu/tex-archive/systems/texlive/tlnet/install-tl-unx.tar.gz in the browser.

@eitsupi
Copy link
Member

eitsupi commented May 7, 2024

This problem seems to reproduce on Ubuntu.

$ wget https://ctan.math.washington.edu/tex-archive/systems/texlive/tlnet/install-tl-unx.tar.gz
--2024-05-07 15:10:10--  https://ctan.math.washington.edu/tex-archive/systems/texlive/tlnet/install-tl-unx.tar.gz
Resolving ctan.math.washington.edu (ctan.math.washington.edu)... 128.95.224.254, 128.95.224.254
Connecting to ctan.math.washington.edu (ctan.math.washington.edu)|128.95.224.254|:443... connected.
ERROR: cannot verify ctan.math.washington.edu's certificate, issued by ‘CN=InCommon RSA Server CA 2,O=Internet2,C=US’:
  Unable to locally verify the issuer's authority.
To connect to ctan.math.washington.edu insecurely, use `--no-check-certificate'.

@eddelbuettel
Copy link
Member

Maybe try wget --no-check-certificate ... ? Or install the ca-certificates package? (Both just guesses from here...)

@benz0li
Copy link
Contributor

benz0li commented May 7, 2024

This problem seems to reproduce on Ubuntu.

Debian 12 (bookworm) with ca-certificates installed:

$ wget https://ctan.math.washington.edu/tex-archive/systems/texlive/tlnet/install-tl-unx.tar.gz
--2024-05-07 17:13:40--  https://ctan.math.washington.edu/tex-archive/systems/texlive/tlnet/install-tl-unx.tar.gz
Resolving ctan.math.washington.edu (ctan.math.washington.edu)... 128.95.224.254
Connecting to ctan.math.washington.edu (ctan.math.washington.edu)|128.95.224.254|:443... connected.
ERROR: The certificate of ‘ctan.math.washington.edu’ is not trusted.
ERROR: The certificate of ‘ctan.math.washington.edu’ doesn't have a known issuer.

@eitsupi Report to [email protected], then.

(Luckily, my server is far away from Washington 😉)


Note:
Please take care not to send any HTML mails to these addresses, because HTML mails are held in CTAN's SPAM filter, and it may take some time until a postmaster comes along to set them free.

https://ctan.org/contact

@cboettig
Copy link
Member

cboettig commented May 7, 2024 via email

@eitsupi
Copy link
Member

eitsupi commented May 10, 2024

I sent an email to CTAN and the problem seems to be resolved.
I triggered the build again.

@eitsupi
Copy link
Member

eitsupi commented May 11, 2024

A new build has been pushed, thanks all.
https://github.com/rocker-org/rocker-versioned2/wiki/ml-verse_acca11003d86

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

5 participants