error load_learner_all

[ramdhan1989]

Hi,
any idea on how to solve this?

---

NotImplementedError Traceback (most recent call last)
Cell In [4], line 1
----> 1 learn = load_learner_all(path='export', dls_fname='dls', model_fname='model', learner_fname='learner')
2 dls = learn.dls
3 valid_dl = dls.valid

File ~\AppData\Roaming\Python\Python310\site-packages\tsai\learner.py:106, in load_all(path, dls_fname, model_fname, learner_fname, device, pickle_module, verbose)
103 else: cpu = None
105 path = Path(path)
--> 106 learn = load_learner(path/f'{learner_fname}.pkl', cpu=cpu, pickle_module=pickle_module)
107 learn.path = path
108 learn.model_dir = Path() # '.'

File ~\AppData\Roaming\Python\Python310\site-packages\fastai\learner.py:448, in load_learner(fname, cpu, pickle_module)
446 distrib_barrier()
447 map_loc = 'cpu' if cpu else default_device()
--> 448 try: res = torch.load(fname, map_location=map_loc, pickle_module=pickle_module)
449 except AttributeError as e:
450 e.args = [f"Custom classes or functions exported with your Learner not available in namespace. Re-declare/import before loading:\n\t{e.args[0]}"]

File ~\AppData\Roaming\Python\Python310\site-packages\torch\serialization.py:789, in load(f, map_location, pickle_module, weights_only, **pickle_load_args)
787 except RuntimeError as e:
788 raise pickle.UnpicklingError(UNSAFE_MESSAGE + str(e)) from None
--> 789 return _load(opened_zipfile, map_location, pickle_module, **pickle_load_args)
790 if weights_only:
791 try:

File ~\AppData\Roaming\Python\Python310\site-packages\torch\serialization.py:1131, in _load(zip_file, map_location, pickle_module, pickle_file, **pickle_load_args)
1129 unpickler = UnpicklerWrapper(data_file, **pickle_load_args)
1130 unpickler.persistent_load = persistent_load
-> 1131 result = unpickler.load()
1133 torch._utils._validate_loaded_sparse_tensors()
1135 return result

File ~\AppData\Local\miniconda3\envs\poc\lib\pathlib.py:962, in Path.new(cls, *args, **kwargs)
960 self = cls._from_parts(args)
961 if not self._flavour.is_supported:
--> 962 raise NotImplementedError("cannot instantiate %r on your system"
963 % (cls.name,))
964 return self

NotImplementedError: cannot instantiate 'PosixPath' on your system

thank you

[vrodriguezf]

When you save a fastai Learner (which tsai's learners inherit from) using export or save_all, it serializes the model, optimizer, and potentially parts of the dataloaders. If your training process involved custom classes or functions (like custom models, loss functions, metrics, data transforms, or even custom data loading logic), these need to be defined and available in the exact same namespace when you try to load the Learner.

[ramdhan1989]

@vrodriguezf
Hi, would you mind elaborating more on how to implement it.
for detail background, I have trained in aws and save using following code:
tfms = [None, TSClassification()] batch_tfms = [TSStandardize(by_sample=True)] X, y, splits = combine_split_data([vectors_trn_, vectors_val_], [labels_trn, labels_val]) dsets = TSDatasets(X, y, tfms=tfms, splits=splits, inplace=True) dls = TSDataLoaders.from_dsets(dsets.train, dsets.valid, bs=[128], batch_tfms=batch_tfms, num_workers=0) model = XceptionTimePlus(dls.vars, dls.c) dsid = 'XceptionTimePlus' learn = ts_learner(dls, XceptionTimePlus, device='cuda', loss_func=LabelSmoothingCrossEntropyFlat(), metrics=[APScoreMulti(), accuracy, BalancedAccuracy()], cbs=ShowGraphCallback2(), pretrained=True, weights_path='./MVP_downhole/'+f'{dsid}_256'+'.pth' ) learn.fit_one_cycle(200, lr_max=1e-3) learn.save(dsid+'_256') learn.save_all(path='export', dls_fname='dls', model_fname='model', learner_fname='learner')

Then, I tried to load from window system using the following code:
learn = load_learner_all(path='export', dls_fname='dls', model_fname='model', learner_fname='learner')

loading the second code in the window system gives me error, but I think it is good when executing the second code in aws.

thanks