updating R packages with r2u

[rvUser777]

Hi,
I am grateful for the help that I received in setting up r2u on my ubuntu system a while back (on R-Sig-Debian). This is working fine. But I have some strange problem with r2u on another PC that, as far as I know, I installed in the the same way.

Let me first show the results in Rstudio from the successfully working Ubuntu system:

install.packages(c("broom", "curl", "data.table", "dbplyr", "dtplyr", "evaluate", "forcats", "gargle", "ggplot2", "googledrive", "googlesheets4", "haven", "hms", "igraph", "magrittr", "openssl", "pillar", "pracma", "purrr", "Radviz", "ragg", "Rcpp", "RcppArmadillo", "reformulas", "renv", "rmarkdown", "rvest", "stringr", "systemfonts", "textshaping", "tibble", "utf8", "vroom", "xfun", "xml2"))
Install system packages...
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
Hit https://cloud.r-project.org/bin/linux/ubuntu noble-cran40/ InRelease
Hit http://archive.ubuntu.com/ubuntu noble InRelease
Hit https://r2u.stat.illinois.edu/ubuntu noble InRelease
Fetched 0 B in 0s (0 B/s)
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
Get:1 https://r2u.stat.illinois.edu/ubuntu noble/main amd64 r-cran-rbibutils amd64 2.3-1.ca2404.2 [940 kB]
Get:2 https://r2u.stat.illinois.edu/ubuntu noble/main all r-cran-rdpack all 2.6.4-1.ca2404.1 [623 kB]
Get:3 https://r2u.stat.illinois.edu/ubuntu noble/main all r-cran-reformulas all 0.4.2-1.ca2404.1 [132 kB]
Fetched 1696 kB in 0s (0 B/s)

Now, I will show the results from the Ubuntu system that appears to work OK, but does not download anything:

install.packages(c("broom", "forcats", "hms", "magrittr", "openssl", "pillar", "rmarkdown", "vroom"))
Install system packages...
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
Hit https://cloud.r-project.org/bin/linux/ubuntu noble-cran40/ InRelease
Hit http://security.ubuntu.com/ubuntu noble-security InRelease
Hit http://archive.ubuntu.com/ubuntu noble InRelease
Hit http://archive.ubuntu.com/ubuntu noble-updates InRelease
Hit http://archive.ubuntu.com/ubuntu noble-backports InRelease
Hit https://r2u.stat.illinois.edu/ubuntu noble InRelease
Fetched 0 B in 0s (0 B/s)
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
Fetched 0 B in 0s (0 B/s)

"Fetched 0 B" ? What can be the reason? The only differences that I can see from the two versions are the following two lines in the latter:

Hit http://security.ubuntu.com/ubuntu noble-security InRelease
Hit http://archive.ubuntu.com/ubuntu noble-backports InRelease

The .libPaths for the latter are :

> .libPaths()
[1] "/home/ravi/R/x86_64-pc-linux-gnu-library/4.5" "/usr/local/lib/R/site-library" "/usr/lib/R/site-library"
[4] "/usr/lib/R/library"

I don't have a problem with updating and installing ubuntu for the latter. The problem appears to only with r2u.

Is this a known problem? Please let me know if any additional information is needed for troubleshooting.
Thanks.
Ravi

[eddelbuettel]

Thanks for asking here, and in a more legible fashion. I added a few sets of three backticks (ie ` without the space) for better formatting,

From the output presented, it seems that you may not have all required setup steps. They are documented on the r2u website both in the main README as well as via scripts. When it works, then it should look like this (where I drop into Docker just to show this 'from an r2u-enabled system that is otherwise empty') (and I keep it at two packages)

edd@paul:~$ docker run --rm -ti rocker/r2u
root@e62ae105547b:/# R -q
> install.packages(c("curl", "data.table"))
Get:1 http://archive.ubuntu.com/ubuntu noble InRelease [256 kB]                                                                                                                                                   
Get:2 https://cloud.r-project.org/bin/linux/ubuntu noble-cran40/ InRelease [3631 B]                                                                                                                               
Get:3 https://r2u.stat.illinois.edu/ubuntu noble InRelease [9161 B]                                                                                                                                               
Get:4 https://r2u.stat.illinois.edu/ubuntu noble/main arm64 Packages [2519 kB]                                                                                                                                    
Get:5 https://cloud.r-project.org/bin/linux/ubuntu noble-cran40/ Packages [42.6 kB]                                                                                                                               
Get:6 http://archive.ubuntu.com/ubuntu noble-updates InRelease [126 kB]                                                                                                                                           
Get:7 https://r2u.stat.illinois.edu/ubuntu noble/main all Packages [8697 kB]                                                                                                                                      
Get:8 http://archive.ubuntu.com/ubuntu noble-backports InRelease [126 kB]                                                                                                                                         
Get:9 http://archive.ubuntu.com/ubuntu noble/universe amd64 Packages [19.3 MB]                                                                                                                                    
Get:10 http://security.ubuntu.com/ubuntu noble-security InRelease [126 kB]                                                                                                                                        
Get:11 https://r2u.stat.illinois.edu/ubuntu noble/main amd64 Packages [2594 kB]                                                                                                                                   
Get:12 http://archive.ubuntu.com/ubuntu noble/restricted amd64 Packages [117 kB]                                                                                                                                  
Get:13 http://archive.ubuntu.com/ubuntu noble/multiverse amd64 Packages [331 kB]                                                                                                                                  
Get:14 http://archive.ubuntu.com/ubuntu noble/main amd64 Packages [1808 kB]                                                                                                                                       
Get:15 http://archive.ubuntu.com/ubuntu noble-updates/main amd64 Packages [2007 kB]                                                                                                                               
Get:16 http://archive.ubuntu.com/ubuntu noble-updates/universe amd64 Packages [1940 kB]                                                                                                                           
Get:17 http://archive.ubuntu.com/ubuntu noble-updates/multiverse amd64 Packages [35.9 kB]                                                                                                                         
Get:18 http://archive.ubuntu.com/ubuntu noble-updates/restricted amd64 Packages [2847 kB]                                                                                                                         
Get:19 http://archive.ubuntu.com/ubuntu noble-backports/main amd64 Packages [49.4 kB]                                                                                                                             
Get:20 http://archive.ubuntu.com/ubuntu noble-backports/universe amd64 Packages [33.9 kB]                                                                                                                         
Get:21 http://security.ubuntu.com/ubuntu noble-security/multiverse amd64 Packages [33.1 kB]                                                                                                                       
Get:22 http://security.ubuntu.com/ubuntu noble-security/main amd64 Packages [1648 kB]                                                                                                                             
Get:23 http://security.ubuntu.com/ubuntu noble-security/universe amd64 Packages [1172 kB]                                                                                                                         
Get:24 http://security.ubuntu.com/ubuntu noble-security/restricted amd64 Packages [2714 kB]                                                                                                                       
Fetched 48.6 MB in 0s (0 B/s)                                                                                                                                                                                     
Install system packages as root...
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
Reading package lists... Done    
Building dependency tree... Done 
Reading state information... Done
Get:1 https://r2u.stat.illinois.edu/ubuntu noble/main amd64 r-cran-curl amd64 7.0.0-1.ca2404.1 [457 kB]                                                                                                           
Get:2 https://r2u.stat.illinois.edu/ubuntu noble/main amd64 r-cran-data.table amd64 1.17.8-1.ca2404.1 [2257 kB]                                                                                                   
Fetched 2714 kB in 0s (0 B/s)                                                                                                                                                                                     
Selecting previously unselected package r-cran-curl.
(Reading database ... 17702 files and directories currently installed.)
Preparing to unpack .../r-cran-curl_7.0.0-1.ca2404.1_amd64.deb ...
Unpacking r-cran-curl (7.0.0-1.ca2404.1) ...
Selecting previously unselected package r-cran-data.table.
Preparing to unpack .../r-cran-data.table_1.17.8-1.ca2404.1_amd64.deb ...
Unpacking r-cran-data.table (1.17.8-1.ca2404.1) ...
Setting up r-cran-curl (7.0.0-1.ca2404.1) ...
Setting up r-cran-data.table (1.17.8-1.ca2404.1) ...
> 

So this hits the apt repo, downloads and installs. A check you can do on your end involves asking apt about what it knows about packages. If I do that for data.table (which for it is r-cran-data.table) we get two answers for the one from the Ubuntu distribution itself, and from r2u:

[ continuing from above, so first exiting R ]
> q()
Save workspace image? [y/n/c]: n
root@e62ae105547b:/# apt-cache policy r-cran-data.table
r-cran-data.table:
  Installed: 1.17.8-1.ca2404.1
  Candidate: 1.17.8-1.ca2404.1
  Version table:
 *** 1.17.8-1.ca2404.1 700
        700 https://r2u.stat.illinois.edu/ubuntu noble/main amd64 Packages
        100 /var/lib/dpkg/status
     1.14.10+dfsg-1 500
        500 http://archive.ubuntu.com/ubuntu noble/universe amd64 Packages
root@e62ae105547b:/# 

which it now shows as installed (via the three ***).

Did around your system and remember that for r2u one of the keys is the handoff from R via the bspm package to apt.

(Also: 'Fetched 0 B' is not a problem per se. It can mean your cached information is current to what the remote site has.)

[eddelbuettel]

Also, maybe we need to backtrack and ask what your actual question. Facundo raised a good point; maybe the system already has the packages you want? You can check that from within R via

IP <- installed.packages()

and then examine IP. For those R-installed packages that via r2u you can also do this on the command-line:

dpkg -l | grep r-cran- 

and maybe paginate this by piping into less or alike.

[rvUser777]

As Facundo and you have suspected, I think that everything is working fine. I have just not understood the r2u updating scheme. Let me explain.
Broom version 1.0.9 is already installed in my machine. When I update with Rstudio, I see that version 1.0.10 is available. I am guessing that the r2u update has stopped at 1.0.9 (for now). That is why nothing is fetched.
I have now tried to install a package that I have not installed yet.

install.packages("parallelPlot")
Install system packages...
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
Hit https://cloud.r-project.org/bin/linux/ubuntu noble-cran40/ InRelease
Hit http://security.ubuntu.com/ubuntu noble-security InRelease
Hit http://archive.ubuntu.com/ubuntu noble InRelease
Hit http://archive.ubuntu.com/ubuntu noble-updates InRelease
Hit http://archive.ubuntu.com/ubuntu noble-backports InRelease
Hit https://r2u.stat.illinois.edu/ubuntu noble InRelease
Fetched 0 B in 0s (0 B/s)
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
Get:1 https://r2u.stat.illinois.edu/ubuntu noble/main all r-cran-htmlwidgets all 1.6.4-1.ca2404.1 [417 kB]
Get:2 https://r2u.stat.illinois.edu/ubuntu noble/main all r-cran-parallelplot all 0.4.0-1.ca2404.1 [166 kB]
Fetched 583 kB in 0s (0 B/s)

Now, the fetching has been done. That's fine.
I now understand what you mean by:
(Also: 'Fetched 0 B' is not a problem per se. It can mean your cached information is current to what the remote site has.)

[eddelbuettel]

You have some other problem you need to disentangle. The broom package and r-cran-broom are at 1.0.10 in r2u for jammy (22.04) and noble (24.04).

[eddelbuettel]

There is some finely calibrated interplay between R and apt thanks to bspm so I suggest you look into IP <- installed.packages() and AP <- available.packages() for your default repos (that is outside of apt, just R's view) and we try to disentangle from there.

[eddelbuettel]

Nothing like milling around the kitchen doing other things -- I think I now know what your issue is.

With R, first match wins and beats highest version wins. You may have the package in question in your personal R package path below your home directory (i.e. the .libPaths()[1] directory). If a version is there it will not matter that r2u has a newer one.

But looking carefully at installed.packages() and available.packages() you can sort this out. Inaki and I also both wrote helper functions to find such 'shadowed' packages. His is in bspm so you have, mine is in my utilities packages dang.

[eddelbuettel]

I am closing this now as @rvUser777 appears to have solved his issue, chiefly by clearing an apparent misunderstanding about what was or was not to happen on his machines. Feel free to re-open as needed with additional detail.

[rvUser777]

Look at the lock symbol above parallelPlot folder. This is what I mean.

[eddelbuettel]

I have no idea. Ask the ubuntu documentation about symbols in their graphical display? I am sure it is documented. I would look at the difference between ls -ld parallely parallelPlot. They will have different modes and/or owner:group I guess.

[rvUser777]

Thanks. I will figure it out. I think that you can now close the issue. Thanks for your help.

[famuvie]

This is how directories where you do not have write permission are displayed in Ubuntu.

This could have been the result of installing those packages with administrator privileges, like opening RStudio or R alone with sudo and then installing packages. However, this would typically get the packages installed in a system directory, not in the user's library along with the other packages, unless told otherwise. But this is unlikely.
Another possibility is having changed the directory permissions accidentally, or have the directory locked (not writable) as a result of a corrupted or interrupted installation process. This last scenario is my best guess.

I suggest you remove those directories (you will have to use sudo) and install those packages again.

[eddelbuettel]

Note that the paths for installed packages (i.e. the elements 2 to 4 of the vector returned from .libPaths() as shown in my answer below) are fixed for all users. But it could have a one-off installation with sudo.

[eddelbuettel]

As a recap, on Debian and Ubuntu we can have

$ Rscript -e 'options(width=30); .libPaths()'
[1] "/home/USERNAME/R/x86_64-pc-linux-gnu-library/4.5"
[2] "/usr/local/lib/R/site-library"              
[3] "/usr/lib/R/site-library"                    
[4] "/usr/lib/R/library"                         
$ 

Here we have in detail:

1. Is optional. I actually turn it off on my systems because I believe in system-wide installation ... even though I am often the only user. So I typically set R_LIBS_USER="" in my ~/.Renviron to suppress this. But if you have it, it is the default target for install.packages() when you do not use r2u ie under normal use.
2. If you do not have the first, this becomes the first and default on installation. As the first match matters when loading a package it means that either 1. or 2. can dominate a later package. So if you have an old package here, it will be loaded. That is how R works.
3. By convention on Debian / Ubuntu all binary packages from r-cran-* or r-bioc-* go here and are managed by apt. This may include r2u if you use, otherwise it is the package from the distribution. This can be older than one from source so having 2. ahead of it can make sense.
4. By convention on Debian / Ubuntu all binary packages labelled by R Core as 'recommended' go here. They are all included in Debian and Ubuntu itself as well.

Hope this helps.

[eddelbuettel]

Closing as solved.