R packages depending on multiple python environments

[jrosell]

I'm trying to do some experiment using two R packages depending on different python environments. They are farach/huggingfaceR and OscarKjell/text. It seems booth require different python packages versions. I started ok with OscarKjell/text and begin to have problems with farach/huggingfaceR. I want both in the same project. How do you think is the best practice to solve this with the "rixverse"?

I think it's a common problem and could be better found in the documentation.

[b-rodrigues]

How does that work usually, so without nix? Does each package call its own python interpreter to run its functions? Which Python versions and packages are required for each package?

[b-rodrigues]

My first try would simply to use a single python interpreter and python packages and see if these two R packages simply use what is available

[jrosell]

Not working. I was thinking of using two or even three environments. Subfolders in the same project? {rixpress}?

[b-rodrigues]

Really depends what you want to achieve, rixpress could be a solution or rix::with_nix()

[b-rodrigues]

Can you share the default.nix you tried?

[jrosell]

Don't worrry for now. I solved my problem using it only OscarKjell/text package. But next time I will give rix::with_nix() a try :)

[b-rodrigues]

could you share the default.nix or the genv-env.R? I’d like to give it a shot.

[jrosell]

# This file was generated by the {rix} R package v0.17.2 on 2025-09-12
# with following call:
# >rix::rix(r_ver = "577a4fbf618c3cc4ad8fe85e36211aa23ab8ea13",
#  > date = r_date,
#  > r_pkgs = r_pkgs,
#  > system_pkgs = system_pkgs,
#  > git_pkgs = git_pkgs,
#  > ide = "none",
#  > project_path = path_default_nix,
#  > overwrite = TRUE,
#  > print = TRUE,
#  > shell_hook = shell_hook)
# It uses upstream nixpkgs' revision 577a4fbf618c3cc4ad8fe85e36211aa23ab8ea13 for reproducibility purposes
# which will install R version latest-upstream.
# Report any issues to https://github.com/ropensci/rix
let
 pkgs = import (fetchTarball "https://github.com/NixOS/nixpkgs/archive/577a4fbf618c3cc4ad8fe85e36211aa23ab8ea13.tar.gz") {};
 
  rpkgs = builtins.attrValues {
    inherit (pkgs.rPackages) 
      reticulate
      text;
  };
 
    huggingfaceR = (pkgs.rPackages.buildRPackage {
      name = "huggingfaceR";
      src = pkgs.fetchgit {
        url = "https://github.com/farach/huggingfaceR/";
        rev = "a9dc96af798e30ef58c3d89d80ac05f5a1f044e5";
        sha256 = "sha256-qWT3WlUlMUCrgGJwPtXXI8W5UISIxmoYcmzKavIr7i8=";
      };
      propagatedBuildInputs = builtins.attrValues {
        inherit (pkgs.rPackages) 
          dplyr
          glue
          httr2
          jsonlite
          lsa
          magrittr
          purrr
          reticulate
          stringr
          tibble;
      };
    });
      
  system_packages = builtins.attrValues {
    inherit (pkgs) 
      glibcLocales
      nix
      R;
  };
  
  shell = pkgs.mkShell {
    LOCALE_ARCHIVE = if pkgs.system == "x86_64-linux" then "${pkgs.glibcLocales}/lib/locale/locale-archive" else "";
    LANG = "en_US.UTF-8";
    LC_ALL = "en_US.UTF-8";
    LC_TIME = "en_US.UTF-8";
    LC_MONETARY = "en_US.UTF-8";
    LC_PAPER = "en_US.UTF-8";
    LC_MEASUREMENT = "en_US.UTF-8";
    
    buildInputs = [ huggingfaceR rpkgs system_packages ];
    
  }; 
in
  {
    inherit pkgs shell;
  }

The error I get:

building '/nix/store/jnlxrhh0xqvb8q71szkvga1yi84j21d1-r-huggingfaceR.drv'...
Running phase: unpackPhase
unpacking source archive /nix/store/48as2l646l2w5mqp24iv1lrjkhghfvba-huggingfaceR-a9dc96a
source root is huggingfaceR-a9dc96a
Running phase: patchPhase
Running phase: updateAutotoolsGnuConfigScriptsPhase
Running phase: configurePhase
Running phase: buildPhase
Running phase: installPhase
* installing *source* package 'huggingfaceR' ...
** this is package 'huggingfaceR' version '0.1.0'
** using staged installation
** R
** data
*** moving datasets to lazyload DB
** byte-compile and prepare package for lazy loading
** help
*** installing help indices
*** copying figures
** building package indices
** installing vignettes
** testing if installed package can be loaded from temporary location
Error: package or namespace load failed for 'huggingfaceR':
 .onAttach failed in attachNamespace() for 'huggingfaceR', details:
  call: fun(libname, pkgname)
  error: Miniconda is not installed.
Use reticulate::install_miniconda() to install Miniconda.
Error: loading failed
Execution halted
ERROR: loading failed
* removing '/nix/store/d3d69p6pc228xlgij5pl214kv5b7qxhi-r-huggingfaceR/library/huggingfaceR'
building '/nix/store/734jgvz4brc1srrzxlhr77yzzsdvjyp9-r-markdown-2.0.drv'...
�[31;1merror:�[0m Cannot build '�[35;1m/nix/store/jnlxrhh0xqvb8q71szkvga1yi84j21d1-r-huggingfaceR.drv�[0m'.
       Reason: �[31;1mbuilder failed with exit code 1�[0m.
       Output paths:
         �[35;1m/nix/store/d3d69p6pc228xlgij5pl214kv5b7qxhi-r-huggingfaceR�[0m
�[31;1merror:�[0m build of �[35;1m'/nix/store/jnlxrhh0xqvb8q71szkvga1yi84j21d1-r-huggingfaceR.drv', '/nix/store/spw2b7k8dfjnz8szf9gn11d1npl4nw50-r-text-1.6.drv'�[0m failed

This is my script: https://github.com/jrosell/rix-run/blob/python-huggingface/examples/python-huggingface.R
I was able to make it work with the same miniconda venv interactively locally without nix but I'm not sure how i can make it work using nix.

[b-rodrigues]

it would be best to use https://cran.r-project.org/web/packages/hfhub/index.html instead

[jrosell]

As far as I know, the {hfhub} is not doing all what I need to do.

The {text} package is using huggingface and sentence-transformers python packages with {reticulate} miniconda and it's fine for me.

I don't try to use huggingfaceR package anymore but I send my .nix file to you with both packages as you asked previously.

[b-rodrigues]

What do you use now?

the isseu with hugginfaceR is that it’s a GitHub only package. I tend to avoid spending too much time to make these work, because as it stands, I’m not even sure it could land on CRAN. The way it checks for an available Python env is not best practice. Plus, it seems to be unmaintained now.

[jrosell]

To use {text} R package I need to install specific python package versions in a miniconda or vritualenv environment. I don't know if {text} can work in nix using uv from shellHook.

[b-rodrigues]

I meant that you could certainly use the python packages installed and managed by nix, without any other layer like uv

[jrosell]

I don't see in the rix documentation how to specify the exact version of the python package versions.

"transformers==4.38.0",
"huggingface_hub==0.20.0",
"numpy==1.26.0",
"pandas==2.0.3",
"nltk==3.8.1",
"scikit-learn==1.3.0", # higher versions 1.4 and 1.5 yield errors in textTopics() and warnings in textTrain
"datasets==2.16.1",
"evaluate==0.4.0",
"accelerate==0.26.0",
"bertopic==0.16.3",
"jsonschema==4.19.2",
"sentence-transformers==2.2.2",
"flair==0.13.0",
"umap-learn==0.5.6",
"hdbscan==0.8.33",
"scipy==1.10.1",
"aiohappyeyeballs==2.4.4"

[b-rodrigues]

try without specifying version it could work 😉

[jrosell]

I will try but I'm worried about this:

"scikit-learn==1.3.0", # higher versions 1.4 and 1.5 yield errors in textTopics() and warnings in textTrain