Downloads problem

[SherifAdel2]

There is a problem in the folder E:\Downloads\TTS_Arabic\tts-arabic-pytorch\venv\Lib\site-packages\tts_arabic\models. Every time the model runs, it downloads the same file repeatedly. The solution is to comment out line 32 as follows:

def get_model_path(package_path: Optional[Path] = None,
name: str = "fastpitch"
) -> str:
file_entry = files_dict[name]
if package_path is None:
package_path = Path(file).parent.parent
model_path = package_path.joinpath(file_entry['file'])
if not model_path.parent.exists():
model_path.parent.mkdir(parents=True)

#if not model_path.exists() or (model_path.lstat().st_mtime < file_entry.get('timestamp', 0)):

if not model_path.exists():
    gdown.download(file_entry['url'], output=model_path.as_posix(), fuzzy=True)

return model_path.as_posix()

[jaggzh]

It's quite fascinating that you post this at the same time as I try the project. My last edit was:
Mar 1 09:17 models/core.py
It's 6:55pm now, so I was literally working on this at the same time as you. :)

(I also did the same change: commented out, left commented version above.)

For now, I think removing the timestamp test is an okay option. @nipponjo
(We don't want to hit gdrive each run to test the file time -- hence the urls.py).
A future patch would be to have an --update option which does the tests.

urls.py has the timestamps hard-coded:

files_dict = {
    # TEXT->MEL MODELS
    # FASTPITCH
    'fastpitch': {
        'file': 'data/fp_ms.onnx',
        # 'url': 'https://drive.google.com/file/d/10zcWAgtINnIdmbfMdSm990-7l5znvARN/view?usp=sharing'
        'url': 'https://drive.google.com/file/d/1pD210QTN1IL3CTA1D65ldKB7ooZ2hANl/view?usp=sharing',
        'timestamp': 1716497335.55365,
    },
...

[nipponjo]

Hi, thanks for pointing that out. I have disabled that check for the moment.
It seems like the timestamp does not always update reliably, so I will think about a better way until the next time I update one of these files.
I wonder what timestamp you get with this: os.stat('.../tts_arabic/tts_arabic/data/fp_ms.onnx').st_mtime

[jaggzh]

The timestamp in urls.py:

$ datetotime 1716497335
2024-05-23 13:48:55

The file downloaded:

$ d fp_ms.onnx
-rw-r--r-- 1 187215347 May 23  2024 fp_ms.onnx

$ stat fp_ms.onnx
...
Modify: 2024-05-23 13:21:08.000000000 -0700
...
$ python -c 'import os; print(os.stat("fp_ms.onnx").st_mtime);'
1716495668.0
$ datetotime 1716495668
2024-05-23 13:21:08

[jaggzh]

It would repeatedly download, but it appears the downloaded timestamp is only 37 minutes younger (perhaps you uploaded to gdrive then updated urls.py a half hour later (after at least this model file), and that was the time used). Secrets! :)

I think one solution is just to ensure your urls.py updater doesn't put a date younger than the gdrive file's upload time. (I've been experimenting with just the http headers of drive and drive.usercontent.google.com hits, but it's not consistent. Maybe I'm missing something obvious, but this returns a last-modified time:

# mixer80.onnx
$ curl -I 'https://drive.usercontent.google.com/download?id=1C95RVIjhVttC8pdFAp1TOiEee9Cq50c8' | grep last.modified
last-modified: Sat, 21 Sep 2024 17:00:49 GMT

But

# fp_ms.onnx
$ curl -I 'https://drive.usercontent.google.com/download?id=1pD210QTN1IL3CTA1D65ldKB7ooZ2hANl' | grep last.modified

...does not. I'm not sure why one works and the other doesn't. Anyway, just experimenting with ways (avoiding actual google api calls (requiring auth?).