Errors when running pya.pred.predict_age()

[hayleywelsh]

I have been trying to use the pyaging package through spyder, as it is one of the only packages that includes the Horvath centenarian clocks. I have been following the tutorial available, but keep running into an issue when using pya.pred.predict_age(). I am very new to python, and have only used it one other time. I have included the code I ran and the output it gave, hoping for some troubleshooting advice.
Thank you!

import pandas as pd
import pyaging as pya
betas = pd.read_csv("/Users/hayleywelsh/Desktop/Methyl paper 3/Paper_3_QC/Raw-beta-values-N87-n866895-no-annotation.csv", index_col=0)
betas = betas.T
betas = pya.pp.epicv2_probe_aggregation(betas)
|-----> 🏗️ Starting epicv2_probe_aggregation function
|-----> ⚙️ Looking for duplicated probes started
|-----------> in progress: 100.0000%
|-----------> There are no duplicated probes. Returning original data
|-----> 🎉 Done! [17.0187s]

adata = pya.pp.df_to_adata(betas, imputer_strategy='knn')
|-----> 🏗️ Starting df_to_adata function
|-----> ⚙️ Create anndata object started
|-----------? Dropping 669 columns with only NAs: ['cg25399512', 'cg15734032', 'cg16292782'], etc.
|-----> ⚠️ Create anndata object finished [1.5476s]
|-----> ⚙️ Add metadata to anndata started
|-----------? No metadata provided. Leaving adata.obs empty
|-----> ⚠️ Add metadata to anndata finished [0.0013s]
|-----> ⚙️ Log data statistics started
|-----------> There are 87 observations
|-----------> There are 866226 features
|-----------> Total missing values: 41628
|-----------> Percentage of missing values: 0.06%
|-----> ✅ Log data statistics finished [0.0886s]
|-----> ⚙️ Impute missing values started
|-----------> Imputing missing values using knn strategy
|-----> ✅ Impute missing values finished [21.9341s]
|-----> ⚙️ Add imputer strategy to adata.uns started
|-----> ✅ Add imputer strategy to adata.uns finished [0.0008s]
|-----> 🎉 Done! [23.5831s]

encen100_results = pya.pred.predict_age(adata, ['encen100'])
|-----> 🏗️ Starting predict_age function
|-----> ⚙️ Set PyTorch device started
|-----------> Using device: cpu
|-----> ✅ Set PyTorch device finished [0.0015s]
|-----> 🕒 Processing clock: encen100
|-----------> ⚙️ Load clock started
|-----------------> Data found in pyaging_data/encen100.pt
|-----------> ✅ Load clock finished [0.0091s]
|-----------> ⚙️ Check features in adata started
|-----------------> All features are present in adata.var_names.
|-----------> ✅ Check features in adata finished [0.2808s]
|-----------> ⚙️ Predict ages with model started
|-----------------> There is no preprocessing necessary
|-----------------> There is no postprocessing necessary
|-----------------> in progress: 100.0000%
---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
Cell In[8], line 1
----> 1 encen100_results = pya.pred.predict_age(adata, ['encen100'])

File ~/Library/spyder-6/envs/spyder-runtime/lib/python3.11/site-packages/pyaging/predict/_pred.py:107, in predict_age(adata, clock_names, dir, batch_size, clean, verbose)
     99 check_features_in_adata(
    100     adata,
    101     model,
    102     logger,
    103     indent_level=2,
    104 )
    106 # Perform age prediction using the model applying preprocessing and postprocessing steps
--> 107 predicted_ages_tensor = predict_ages_with_model(adata, model, device, batch_size, logger, indent_level=2)
    109 # Add predicted ages and clock metadata to adata
    110 add_pred_ages_and_clock_metadata_adata(adata, model, predicted_ages_tensor, dir, logger, indent_level=2)

File ~/Library/spyder-6/envs/spyder-runtime/lib/python3.11/site-packages/pyaging/utils/_utils.py:66, in progress.<locals>.decorator.<locals>.wrapper(*args, **kwargs)
     64 logger = args[-1]  # Assumes logger is the last positional argument
     65 logger.start_progress(f"{message} started", indent_level=indent_level)
---> 66 result = func(*args, **kwargs)
     67 logger.finish_progress(f"{message} finished", indent_level=indent_level)
     68 return result

File ~/Library/spyder-6/envs/spyder-runtime/lib/python3.11/site-packages/pyaging/predict/_pred_utils.py:315, in predict_ages_with_model(adata, model, device, batch_size, logger, indent_level)
    313 with torch.inference_mode():
    314     for batch in main_tqdm(dataloader, indent_level=indent_level + 1, logger=logger):
--> 315         batch_pred = model(batch.obsm[f"X_{model.metadata['clock_name']}"])
    316         predictions.append(batch_pred)
    317 # Concatenate all batch predictions

File ~/Library/spyder-6/envs/spyder-runtime/lib/python3.11/site-packages/torch/nn/modules/module.py:1511, in Module._wrapped_call_impl(self, *args, **kwargs)
   1509     return self._compiled_call_impl(*args, **kwargs)  # type: ignore[misc]
   1510 else:
-> 1511     return self._call_impl(*args, **kwargs)

File ~/Library/spyder-6/envs/spyder-runtime/lib/python3.11/site-packages/torch/nn/modules/module.py:1520, in Module._call_impl(self, *args, **kwargs)
   1515 # If we don't have any hooks, we want to skip the rest of the logic in
   1516 # this function, and just call forward.
   1517 if not (self._backward_hooks or self._backward_pre_hooks or self._forward_hooks or self._forward_pre_hooks
   1518         or _global_backward_pre_hooks or _global_backward_hooks
   1519         or _global_forward_hooks or _global_forward_pre_hooks):
-> 1520     return forward_call(*args, **kwargs)
   1522 try:
   1523     result = None

File ~/Library/spyder-6/envs/spyder-runtime/lib/python3.11/site-packages/pyaging/models/_base_models.py:40, in pyagingModel.forward(self, x)
     38 def forward(self, x):
     39     x = self.preprocess(x)
---> 40     x = self.base_model(x)
     41     x = self.postprocess(x)
     42     return x

File ~/Library/spyder-6/envs/spyder-runtime/lib/python3.11/site-packages/torch/nn/modules/module.py:1511, in Module._wrapped_call_impl(self, *args, **kwargs)
   1509     return self._compiled_call_impl(*args, **kwargs)  # type: ignore[misc]
   1510 else:
-> 1511     return self._call_impl(*args, **kwargs)

File ~/Library/spyder-6/envs/spyder-runtime/lib/python3.11/site-packages/torch/nn/modules/module.py:1520, in Module._call_impl(self, *args, **kwargs)
   1515 # If we don't have any hooks, we want to skip the rest of the logic in
   1516 # this function, and just call forward.
   1517 if not (self._backward_hooks or self._backward_pre_hooks or self._forward_hooks or self._forward_pre_hooks
   1518         or _global_backward_pre_hooks or _global_backward_hooks
   1519         or _global_forward_hooks or _global_forward_pre_hooks):
-> 1520     return forward_call(*args, **kwargs)
   1522 try:
   1523     result = None

File ~/Library/spyder-6/envs/spyder-runtime/lib/python3.11/site-packages/pyaging/models/_base_models.py:68, in LinearModel.forward(self, x)
     67 def forward(self, x):
---> 68     x = self.linear(x)
     69     return x

File ~/Library/spyder-6/envs/spyder-runtime/lib/python3.11/site-packages/torch/nn/modules/module.py:1511, in Module._wrapped_call_impl(self, *args, **kwargs)
   1509     return self._compiled_call_impl(*args, **kwargs)  # type: ignore[misc]
   1510 else:
-> 1511     return self._call_impl(*args, **kwargs)

File ~/Library/spyder-6/envs/spyder-runtime/lib/python3.11/site-packages/torch/nn/modules/module.py:1520, in Module._call_impl(self, *args, **kwargs)
   1515 # If we don't have any hooks, we want to skip the rest of the logic in
   1516 # this function, and just call forward.
   1517 if not (self._backward_hooks or self._backward_pre_hooks or self._forward_hooks or self._forward_pre_hooks
   1518         or _global_backward_pre_hooks or _global_backward_hooks
   1519         or _global_forward_hooks or _global_forward_pre_hooks):
-> 1520     return forward_call(*args, **kwargs)
   1522 try:
   1523     result = None

File ~/Library/spyder-6/envs/spyder-runtime/lib/python3.11/site-packages/torch/nn/modules/linear.py:116, in Linear.forward(self, input)
    115 def forward(self, input: Tensor) -> Tensor:
--> 116     return F.linear(input, self.weight, self.bias)

RuntimeError: mat1 and mat2 must have the same dtype, but got Float and Double

[lcamillo]

Hi! Do you mind sharing your version of pyaging and pytorch and which environment you are running it from?

[hayleywelsh]

The version of pyaging is 0.1.21 and the pytorch version is 2.2.2. Python version 3.11.11 and IPython version 8.34.0 in spyder 6.

[lcamillo]

Are you running Linux, MacOS, or Windows? Does it have a GPU?

Do you mind trying betas = betas.astype('float32') and betas = betas.astype('float64') prior to creating the adata object?

I would also try to upgrade pytorch to the latest version.

Let me know if it helps!

[hayleywelsh]

I am running MacOS with the Intel Iris Plus Graphics GPU. I have tried the betas = betas.astype('float32') and betas = betas.astype('float64') as suggested. When trying to update pytorch, it is telling me that I have the lasted version:

!pip install --upgrade torch torchvision torchaudio
Requirement already satisfied: torch in ./Library/spyder-6/envs/spyder-runtime/lib/python3.11/site-packages (2.2.2)
Collecting torchvision
  Downloading torchvision-0.17.2-cp311-cp311-macosx_10_13_x86_64.whl.metadata (6.6 kB)
Collecting torchaudio
  Downloading torchaudio-2.2.2-cp311-cp311-macosx_10_13_x86_64.whl.metadata (6.4 kB)
Requirement already satisfied: filelock in ./Library/spyder-6/envs/spyder-runtime/lib/python3.11/site-packages (from torch) (3.18.0)
Requirement already satisfied: typing-extensions>=4.8.0 in ./Library/spyder-6/envs/spyder-runtime/lib/python3.11/site-packages (from torch) (4.12.2)
Requirement already satisfied: sympy in ./Library/spyder-6/envs/spyder-runtime/lib/python3.11/site-packages (from torch) (1.13.3)
Requirement already satisfied: networkx in ./Library/spyder-6/envs/spyder-runtime/lib/python3.11/site-packages (from torch) (3.4.2)
Requirement already satisfied: jinja2 in ./Library/spyder-6/envs/spyder-runtime/lib/python3.11/site-packages (from torch) (3.1.6)
Requirement already satisfied: fsspec in ./Library/spyder-6/envs/spyder-runtime/lib/python3.11/site-packages (from torch) (2025.3.2)
Requirement already satisfied: numpy in ./Library/spyder-6/envs/spyder-runtime/lib/python3.11/site-packages (from torchvision) (2.2.4)
Requirement already satisfied: pillow!=8.3.*,>=5.3.0 in ./Library/spyder-6/envs/spyder-runtime/lib/python3.11/site-packages (from torchvision) (11.1.0)
Requirement already satisfied: MarkupSafe>=2.0 in ./Library/spyder-6/envs/spyder-runtime/lib/python3.11/site-packages (from jinja2->torch) (3.0.2)
Requirement already satisfied: mpmath<1.4,>=1.1.0 in ./Library/spyder-6/envs/spyder-runtime/lib/python3.11/site-packages (from sympy->torch) (1.3.0)
Downloading torchvision-0.17.2-cp311-cp311-macosx_10_13_x86_64.whl (1.7 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.7/1.7 MB 11.8 MB/s eta 0:00:00
ownloading torchaudio-2.2.2-cp311-cp311-macosx_10_13_x86_64.whl (3.4 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3.4/3.4 MB 16.6 MB/s eta 0:00:00
�[?25hInstalling collected packages: torchvision, torchaudio
Successfully installed torchaudio-2.2.2 torchvision-0.17.2 

When running pya.pred.predict_age() again, I am still getting an error message:

encen100_results = pya.pred.predict_age(adata, ['encen100'])
|-----> 🏗️ Starting predict_age function
|-----> ⚙️ Set PyTorch device started
|-----------> Using device: cpu
|-----> ✅ Set PyTorch device finished [0.0017s]
|-----> 🕒 Processing clock: encen100
|-----------> ⚙️ Load clock started
|-----------------> Data found in pyaging_data/encen100.pt
|-----------> ✅ Load clock finished [0.0440s]
|-----------> ⚙️ Check features in adata started
|-----------------> All features are present in adata.var_names.
|-----------> ✅ Check features in adata finished [0.2943s]
|-----------> ⚙️ Predict ages with model started
|-----------------> There is no preprocessing necessary
|-----------------> There is no postprocessing necessary
|-----------------> in progress: 100.0000%
---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
Cell In[13], line 1
----> 1 encen100_results = pya.pred.predict_age(adata, ['encen100'])

File ~/Library/spyder-6/envs/spyder-runtime/lib/python3.11/site-packages/pyaging/predict/_pred.py:107, in predict_age(adata, clock_names, dir, batch_size, clean, verbose)
     99 check_features_in_adata(
    100     adata,
    101     model,
    102     logger,
    103     indent_level=2,
    104 )
    106 # Perform age prediction using the model applying preprocessing and postprocessing steps
--> 107 predicted_ages_tensor = predict_ages_with_model(adata, model, device, batch_size, logger, indent_level=2)
    109 # Add predicted ages and clock metadata to adata
    110 add_pred_ages_and_clock_metadata_adata(adata, model, predicted_ages_tensor, dir, logger, indent_level=2)

File ~/Library/spyder-6/envs/spyder-runtime/lib/python3.11/site-packages/pyaging/utils/_utils.py:66, in progress.<locals>.decorator.<locals>.wrapper(*args, **kwargs)
     64 logger = args[-1]  # Assumes logger is the last positional argument
     65 logger.start_progress(f"{message} started", indent_level=indent_level)
---> 66 result = func(*args, **kwargs)
     67 logger.finish_progress(f"{message} finished", indent_level=indent_level)
     68 return result

File ~/Library/spyder-6/envs/spyder-runtime/lib/python3.11/site-packages/pyaging/predict/_pred_utils.py:315, in predict_ages_with_model(adata, model, device, batch_size, logger, indent_level)
    313 with torch.inference_mode():
    314     for batch in main_tqdm(dataloader, indent_level=indent_level + 1, logger=logger):
--> 315         batch_pred = model(batch.obsm[f"X_{model.metadata['clock_name']}"])
    316         predictions.append(batch_pred)
    317 # Concatenate all batch predictions

File ~/Library/spyder-6/envs/spyder-runtime/lib/python3.11/site-packages/torch/nn/modules/module.py:1511, in Module._wrapped_call_impl(self, *args, **kwargs)
   1509     return self._compiled_call_impl(*args, **kwargs)  # type: ignore[misc]
   1510 else:
-> 1511     return self._call_impl(*args, **kwargs)

File ~/Library/spyder-6/envs/spyder-runtime/lib/python3.11/site-packages/torch/nn/modules/module.py:1520, in Module._call_impl(self, *args, **kwargs)
   1515 # If we don't have any hooks, we want to skip the rest of the logic in
   1516 # this function, and just call forward.
   1517 if not (self._backward_hooks or self._backward_pre_hooks or self._forward_hooks or self._forward_pre_hooks
   1518         or _global_backward_pre_hooks or _global_backward_hooks
   1519         or _global_forward_hooks or _global_forward_pre_hooks):
-> 1520     return forward_call(*args, **kwargs)
   1522 try:
   1523     result = None

File ~/Library/spyder-6/envs/spyder-runtime/lib/python3.11/site-packages/pyaging/models/_base_models.py:40, in pyagingModel.forward(self, x)
     38 def forward(self, x):
     39     x = self.preprocess(x)
---> 40     x = self.base_model(x)
     41     x = self.postprocess(x)
     42     return x

File ~/Library/spyder-6/envs/spyder-runtime/lib/python3.11/site-packages/torch/nn/modules/module.py:1511, in Module._wrapped_call_impl(self, *args, **kwargs)
   1509     return self._compiled_call_impl(*args, **kwargs)  # type: ignore[misc]
   1510 else:
-> 1511     return self._call_impl(*args, **kwargs)

File ~/Library/spyder-6/envs/spyder-runtime/lib/python3.11/site-packages/torch/nn/modules/module.py:1520, in Module._call_impl(self, *args, **kwargs)
   1515 # If we don't have any hooks, we want to skip the rest of the logic in
   1516 # this function, and just call forward.
   1517 if not (self._backward_hooks or self._backward_pre_hooks or self._forward_hooks or self._forward_pre_hooks
   1518         or _global_backward_pre_hooks or _global_backward_hooks
   1519         or _global_forward_hooks or _global_forward_pre_hooks):
-> 1520     return forward_call(*args, **kwargs)
   1522 try:
   1523     result = None

File ~/Library/spyder-6/envs/spyder-runtime/lib/python3.11/site-packages/pyaging/models/_base_models.py:68, in LinearModel.forward(self, x)
     67 def forward(self, x):
---> 68     x = self.linear(x)
     69     return x

File ~/Library/spyder-6/envs/spyder-runtime/lib/python3.11/site-packages/torch/nn/modules/module.py:1511, in Module._wrapped_call_impl(self, *args, **kwargs)
   1509     return self._compiled_call_impl(*args, **kwargs)  # type: ignore[misc]
   1510 else:
-> 1511     return self._call_impl(*args, **kwargs)

File ~/Library/spyder-6/envs/spyder-runtime/lib/python3.11/site-packages/torch/nn/modules/module.py:1520, in Module._call_impl(self, *args, **kwargs)
   1515 # If we don't have any hooks, we want to skip the rest of the logic in
   1516 # this function, and just call forward.
   1517 if not (self._backward_hooks or self._backward_pre_hooks or self._forward_hooks or self._forward_pre_hooks
   1518         or _global_backward_pre_hooks or _global_backward_hooks
   1519         or _global_forward_hooks or _global_forward_pre_hooks):
-> 1520     return forward_call(*args, **kwargs)
   1522 try:
   1523     result = None

File ~/Library/spyder-6/envs/spyder-runtime/lib/python3.11/site-packages/torch/nn/modules/linear.py:116, in Linear.forward(self, input)
    115 def forward(self, input: Tensor) -> Tensor:
--> 116     return F.linear(input, self.weight, self.bias)

RuntimeError: mat1 and mat2 must have the same dtype, but got Float and Double

[lcamillo]

Thanks for the logs! I think I found the bug and updated the package to version 0.1.22. Hopefully now it works!

[hayleywelsh]

Great! How long does it typically take to run? I updated pyaging and it didn't immediately provide error messages like before. However, it has been running for >2 hours now. I am just wondering if this is normal or if it is "stuck". Thank you!

[lcamillo]

Ah! It should take less than a second. Could you please try going to the terminal, creating a new environment and then running the following script:

conda create -n pyaging_test python=3.11 -y
conda activate pyaging_test
pip install pyaging==0.1.22
python

Then:

import pandas as pd
import pyaging as pya
import torch

# Check if GPU is available for PyTorch operations
print(f"GPU available: {torch.cuda.is_available()}")

# Check if cupy is available for GPU-accelerated array operations
try:
    import cupy
    cupy_available = True
    print("Successfully imported CuPy")
except ImportError:
    cupy_available = False
    print("Failed to import CuPy - will use NumPy instead")
print(f"CuPy available: {cupy_available}")

# Download example methylation dataset
print("Downloading example dataset GSE139307...")
pya.data.download_example_data('GSE139307')

# Load the dataset into a pandas DataFrame
print("Loading dataset from pickle file...")
df = pd.read_pickle('pyaging_data/GSE139307.pkl')
print(f"Loaded DataFrame with shape: {df.shape}")

# Create binary gender feature (1 for female, 0 for male)
print("Creating binary 'female' feature from gender...")
df['female'] = (df['gender'] == 'F').astype(int)
print(f"Gender distribution: {df['female'].value_counts()}")

# Aggregate probes for EPIC v2 array
print("Performing EPIC v2 probe aggregation...")
df = pya.pp.epicv2_probe_aggregation(df)
print(f"DataFrame shape after probe aggregation: {df.shape}")

# Convert DataFrame to AnnData object with metadata and KNN imputation
print("Converting to AnnData with KNN imputation...")
adata = pya.pp.df_to_adata(df, metadata_cols=['gender', 'tissue_type', 'dataset'], imputer_strategy='knn')
print(f"Created AnnData object with shape: {adata.shape}")

# Predict age using multiple epigenetic clocks
print("Predicting age with first set of clocks...")
pya.pred.predict_age(adata, ['horvath2013'])
print("First age prediction complete. Observation data:")
print(adata.obs)

# Predict age using encen100 clock
print("Predicting age with encen100 clock...")
pya.pred.predict_age(adata, ['encen100'])
print("All age predictions complete. Final observation data:")
print(adata.obs)

Thanks a lot again for bringing this to my attention!

[hayleywelsh]

I downloaded anaconda and ran the first set of code in terminal without issue. However, when running the second set of code, it says my GPU is not available for PyTorch.

print(f"GPU available: {torch.cuda.is_available()}")
GPU available: False

Could this be because I am using Intel MacOS? Is it possible to run this on an Intel MacOS computer or do I need to try running this code on a different computer?

Thank you for your ongoing help!

[lcamillo]

It should still work without a GPU! When running the script, did you notice if cupy was installed? If so, I believe that might be the culprit and uninstalling should hopefully fix the problem.

[hayleywelsh]

I checked and cupy is not installed (numpy 2.2.4 is installed). The function seems to be running, it just isn't finishing and adding the results to my environment.

encen100_results = pya.pred.predict_age(adata, ['encen100'])
|-----> 🏗️ Starting predict_age function
|-----> ⚙️ Set PyTorch device started
|-----------> Using device: cpu
|-----> ✅ Set PyTorch device finished [0.0044s]
|-----> 🕒 Processing clock: encen100
|-----------> ⚙️ Load clock started
|-----------------> Data found in pyaging_data/encen100.pt
|-----------> ✅ Load clock finished [0.0167s]
|-----------> ⚙️ Check features in adata started
|-----------------> All features are present in adata.var_names.
|-----------> ✅ Check features in adata finished [0.2433s]
|-----------> ⚙️ Predict ages with model started
|-----------------> There is no preprocessing necessary
|-----------------> There is no postprocessing necessary
|-----------------> in progress: 100.0000%

[lcamillo]

I see. I've run out of ideas unfortunately. A suggestion would be that, at the very top of the tutorials, you can open a Google Colab notebook by clicking on the badge. If you pip install pyaging and upload your data, you should be able to use it. Let me know if that helps!