Checking the runmode of .prior files

[christophhillisch]

Dear Reinvent4 Community,

for a given .prior file I would like to check the run mode used for the training, to ensure that users only use Reinvent priors for Reinvent, Libinvent for Libinvent and so on.
I tried loading the prior with torch, but when doing this I get a ModuleNotFoundError.

torch.load(input_prior, map_location="cpu", weights_only=False)
----
ModuleNotFoundError: No module named 'reinvent.runmodes'

Does anyone have an idea on how this could be implemented?

[halx]

Hi,

the priors contain metadata to identify, basically, what code in REINVENT needs to be run. The user supplies the desired prior and the correct code is triggered.

Your error message suggests that your current Python environment does not have REINVENT installed. I would need to see the complete error message..

Many thanks,
Hannes.

[christophhillisch]

Hi Hannes,

Ah indeed, I had a folder called reinvent in my own code, causing a name overlap and the imports no longer worked properly.
After refactoring, loading the prior worked.

prior = torch.load(io.BytesIO(uploaded_prior.read()), map_location="cpu", weights_only=False)
run_mode = prior.get("model_type")

Gives the run mode the prior was trained on, allowing the verification that the user selects the correct prior for the correct mode.

Thanks!

[halx]

Many thanks. But can you stilll give me the original error message. Ideally the priors should not dependend on REINVENT code. The older ones still may do.

[christophhillisch]

Yes of course: here is the error summary.

File ".../torch/serialization.py", line 1964, in _load
  result = unpickler.load()
File ".../torch/serialization.py", line 1953, in find_class
  return super().find_class(mod_name, name)
ModuleNotFoundError: No module named 'reinvent'

Here the more detailed output:

C:\Users\UserName\project\...  in check_prior_model_type

    11 │   """
    12 │   try:
    13 │   │   uploaded_prior.seek(0)
  ❱ 14 │   │   prior = torch.load(
    15 │   │   │   io.BytesIO(uploaded_prior.read()), map_location="cpu", weights_only
    16 │   │   )
    17 │   │   run_mode = prior.get("model_type")

C:\Users\UserName\project\.venv\Lib\site-packages\torch\serialization.py:1471 in load

    1468 │   │   │   │   │   │   )
    1469 │   │   │   │   │   except pickle.UnpicklingError as e:
    1470 │   │   │   │   │   │   raise pickle.UnpicklingError(_get_wo_message(str(e)))
  ❱ 1471 │   │   │   │   return _load(
    1472 │   │   │   │   │   opened_zipfile,
    1473 │   │   │   │   │   map_location,
    1474 │   │   │   │   │   pickle_module,

C:\Users\UserName\project\.venv\Lib\site-packages\torch\serialization.py:1964 in _load

    1961 │   # not connected (wrapper subclasses and tensors rebuilt using numpy)
    1962 │   global _serialization_tls
    1963 │   _serialization_tls.map_location = map_location
  ❱ 1964 │   result = unpickler.load()
    1965 │   _serialization_tls.map_location = None
    1966 │   
    1967 │   torch._utils._validate_loaded_sparse_tensors()

C:\Users\UserName\project\.venv\Lib\site-packages\torch\serialization.py:1953 in find_class

    1950 │   │   │   │   except KeyError:
    1951 │   │   │   │   │   pass
    1952 │   │   │   mod_name = load_module_mapping.get(mod_name, mod_name)
  ❱ 1953 │   │   │   return super().find_class(mod_name, name)
    1954 │   
    1955 │   # Load the data (which may in turn use `persistent_load` to load tensors)
    1956 │   data_file = io.BytesIO(zip_file.get_record(pickle_file))

Steps to reproduce:

1. Save a prior using REINVENT4.
2. Attempt to load the prior in an environment where the reinvent package is not installed or not in the Python path.

I hope this helps!