Open
Description
在windows系统下,通过mindnlp库加载模型的时候会发生报错:
import winfcntlock as fcntl # pylint: disable=import-error
ModuleNotFoundError: No module named 'winfcntlock'
经检查,修改utils/download.py里的import winfcntlock as fcntl为from . import winfcntlock as fcntl可以解决该问题。但是,继续运行之后仍有报错:
Traceback (most recent call last):
File "E:\git\biogpt\infer_mindspore.py", line 8, in <module>
tokenizer = BioGptTokenizer.from_pretrained(model_name)
File "D:\install\anaconda\envs\biogpt\lib\site-packages\mindnlp\transformers\tokenization_utils_base.py", line 2115, in from_pretrained
resolved_config_file = cached_file(
File "D:\install\anaconda\envs\biogpt\lib\site-packages\mindnlp\utils\download.py", line 527, in cached_file
resolved_file = download(
File "D:\install\anaconda\envs\biogpt\lib\site-packages\mindnlp\utils\download.py", line 655, in download
raise exp
File "D:\install\anaconda\envs\biogpt\lib\site-packages\mindnlp\utils\download.py", line 651, in download
pointer_path = threads_exclusive_http_get(url, storage_folder, download_file_name=relative_filename, proxies=proxies, headers=headers)
File "D:\install\anaconda\envs\biogpt\lib\site-packages\mindnlp\utils\download.py", line 162, in threads_exclusive_http_get
fcntl.flock(fd, fcntl.LOCK_UN)
AttributeError: module 'mindnlp.utils.winfcntlock' has no attribute 'flock'. Did you mean: 'lock'?
这是因为,download.py中引用winfcntlock.py中的函数名称和实际名称对不上,有两处不对:①download.py中引用的是fcntl.flock(),但是winfcntlock.py中实际名称为lock;②download.py中引用的是fcntl.LOCK_UN,但是winfcntlock.py中没有这个变量声明;
个人参考网上资料,找到了一个解决方案:修改winfcntlock.py中的lock函数名称为flock,并添加变量LOCK_UN = 0x08 。
修改后,为了解决报错:
Traceback (most recent call last):
File "D:\install\anaconda\envs\biogpt\lib\site-packages\mindnlp\utils\download.py", line 160, in threads_exclusive_http_get
raise exp
File "D:\install\anaconda\envs\biogpt\lib\site-packages\mindnlp\utils\download.py", line 156, in threads_exclusive_http_get
fcntl.flock(fd, fcntl.LOCK_EX)
File "D:\install\anaconda\envs\biogpt\lib\site-packages\winfcntl\winfcntl.py", line 180, in flock
handle = msvcrt.get_osfhandle(fd)
TypeError: '_io.TextIOWrapper' object cannot be interpreted as an integer
修改了winfcntlock.py中的hfile = win32file._get_osfhandle(file.fileno())为fd = hfile = win32file._get_osfhandle(file),不然对int变量取file.fileno()会报错。
修改后,下载模型能够正常进行,但是文件夹里会有很多.lock临时文件:

修改download.py文件,在下载完成后删除.lock临时文件:
lock_file.close()
try:
os.remove(lock_file_path)
except Exception as delete_exp:
logging.error(f"Failed to delete lock file: {delete_exp}")

现在在windows系统上可以正常运行了,对于download.py文件没有做很多修改,所以linux的运行应该不受影响。