From_pretrained of TableTransformerForObjectDetection could not use only local file #42214
Description
System Info
Python : 3.11.13
Transformers : 4.55.4
Linux
What I'm trying to do:
I'm trying to run a model locally using a directory in which I have my model downloaded but transformers keep trying to reach to the internet to download the model.
Since I'm working on a server not connected to the huggingface hub, it keeps giving me an error.
To download my model, I loaded them on another server which is connected to the hub and I went to the cache directory to take the repo downloaded.
Thank you in advance for your help.
Who can help?
No response
Information
- The official example scripts
- My own modified scripts
Tasks
- An officially supported task in the
examplesfolder (such as GLUE/SQuAD, ...) - My own task or dataset (give details below)
Reproduction
- Install transformers 4.55.4 on python 3.11.13
- Downloaded to the cache the repo "microsoft/table-transformer-detection"
- Copy the repo folder from the cache
- Use another server not connected to internet
- Try to load the model using the following command
Code:
`from transformers import TableTransformerForObjectDetection
import os
cache_dir = "/home/jovyan/workspaces/test/TEMP/OCR/"
if not os.path.exists(cache_dir):
os.makedirs(cache_dir, exist_ok=True)
os.chmod(cache_dir, 0o755)
model = TableTransformerForObjectDetection.from_pretrained(
pretrained_model_name_or_path="microsoft/table-transformer-detection",
local_files_only=True,
cache_dir=cache_dir
)
print("Modèle chargé avec succès !")`
Expected behavior
Error:
`---------------------------------------------------------------------------
OSError Traceback (most recent call last)
File /usr/conda/envs/custom/ocr-python-311/lib/python3.11/site-packages/urllib3/connection.py:198, in HTTPConnection._new_conn(self)
197 try:
--> 198 sock = connection.create_connection(
199 (self._dns_host, self.port),
200 self.timeout,
201 source_address=self.source_address,
202 socket_options=self.socket_options,
203 )
204 except socket.gaierror as e:
File /usr/conda/envs/custom/ocr-python-311/lib/python3.11/site-packages/urllib3/util/connection.py:85, in create_connection(address, timeout, source_address, socket_options)
84 try:
---> 85 raise err
86 finally:
87 # Break explicitly a reference cycle
File /usr/conda/envs/custom/ocr-python-311/lib/python3.11/site-packages/urllib3/util/connection.py:73, in create_connection(address, timeout, source_address, socket_options)
72 sock.bind(source_address)
---> 73 sock.connect(sa)
74 # Break explicitly a reference cycle
OSError: [Errno 101] Network is unreachable
The above exception was the direct cause of the following exception:
NewConnectionError Traceback (most recent call last)
File /usr/conda/envs/custom/ocr-python-311/lib/python3.11/site-packages/urllib3/connectionpool.py:793, in HTTPConnectionPool.urlopen(self, method, url, body, headers, retries, redirect, assert_same_host, timeout, pool_timeout, release_conn, chunked, body_pos, preload_content, decode_content, **response_kw)
792 # Make the request on the HTTPConnection object
--> 793 response = self._make_request(
794 conn,
795 method,
796 url,
797 timeout=timeout_obj,
798 body=body,
799 headers=headers,
800 chunked=chunked,
801 retries=retries,
802 response_conn=response_conn,
803 preload_content=preload_content,
804 decode_content=decode_content,
805 **response_kw,
806 )
808 # Everything went great!
File /usr/conda/envs/custom/ocr-python-311/lib/python3.11/site-packages/urllib3/connectionpool.py:491, in HTTPConnectionPool._make_request(self, conn, method, url, body, headers, retries, timeout, chunked, response_conn, preload_content, decode_content, enforce_content_length)
490 new_e = _wrap_proxy_error(new_e, conn.proxy.scheme)
--> 491 raise new_e
493 # conn.request() calls http.client.*.request, not the method in
494 # urllib3.request. It also calls makefile (recv) on the socket.
File /usr/conda/envs/custom/ocr-python-311/lib/python3.11/site-packages/urllib3/connectionpool.py:467, in HTTPConnectionPool._make_request(self, conn, method, url, body, headers, retries, timeout, chunked, response_conn, preload_content, decode_content, enforce_content_length)
466 try:
--> 467 self._validate_conn(conn)
468 except (SocketTimeout, BaseSSLError) as e:
File /usr/conda/envs/custom/ocr-python-311/lib/python3.11/site-packages/urllib3/connectionpool.py:1099, in HTTPSConnectionPool._validate_conn(self, conn)
1098 if conn.is_closed:
-> 1099 conn.connect()
1101 # TODO revise this, see urllib3/urllib3#2791
File /usr/conda/envs/custom/ocr-python-311/lib/python3.11/site-packages/urllib3/connection.py:616, in HTTPSConnection.connect(self)
615 sock: socket.socket | ssl.SSLSocket
--> 616 self.sock = sock = self._new_conn()
617 server_hostname: str = self.host
File /usr/conda/envs/custom/ocr-python-311/lib/python3.11/site-packages/urllib3/connection.py:213, in HTTPConnection._new_conn(self)
212 except OSError as e:
--> 213 raise NewConnectionError(
214 self, f"Failed to establish a new connection: {e}"
215 ) from e
217 # Audit hooks are only available in Python 3.8+
NewConnectionError: <urllib3.connection.HTTPSConnection object at 0x7f0edb3e2650>: Failed to establish a new connection: [Errno 101] Network is unreachable
The above exception was the direct cause of the following exception:
MaxRetryError Traceback (most recent call last)
File /usr/conda/envs/custom/ocr-python-311/lib/python3.11/site-packages/requests/adapters.py:644, in HTTPAdapter.send(self, request, stream, timeout, verify, cert, proxies)
643 try:
--> 644 resp = conn.urlopen(
645 method=request.method,
646 url=url,
647 body=request.body,
648 headers=request.headers,
649 redirect=False,
650 assert_same_host=False,
651 preload_content=False,
652 decode_content=False,
653 retries=self.max_retries,
654 timeout=timeout,
655 chunked=chunked,
656 )
658 except (ProtocolError, OSError) as err:
File /usr/conda/envs/custom/ocr-python-311/lib/python3.11/site-packages/urllib3/connectionpool.py:847, in HTTPConnectionPool.urlopen(self, method, url, body, headers, retries, redirect, assert_same_host, timeout, pool_timeout, release_conn, chunked, body_pos, preload_content, decode_content, **response_kw)
845 new_e = ProtocolError("Connection aborted.", new_e)
--> 847 retries = retries.increment(
848 method, url, error=new_e, _pool=self, _stacktrace=sys.exc_info()[2]
849 )
850 retries.sleep()
File /usr/conda/envs/custom/ocr-python-311/lib/python3.11/site-packages/urllib3/util/retry.py:515, in Retry.increment(self, method, url, response, error, _pool, _stacktrace)
514 reason = error or ResponseError(cause)
--> 515 raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]
517 log.debug("Incremented Retry for (url='%s'): %r", url, new_retry)
MaxRetryError: HTTPSConnectionPool(host='huggingface.co', port=443): Max retries exceeded with url: /timm/resnet18.a1_in1k/resolve/main/pytorch_model.bin (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f0edb3e2650>: Failed to establish a new connection: [Errno 101] Network is unreachable'))
During handling of the above exception, another exception occurred:
ConnectionError Traceback (most recent call last)
File /usr/conda/envs/custom/ocr-python-311/lib/python3.11/site-packages/huggingface_hub/file_download.py:1546, in _get_metadata_or_catch_error(repo_id, filename, repo_type, revision, endpoint, proxies, etag_timeout, headers, token, local_files_only, relative_filename, storage_folder)
1545 try:
-> 1546 metadata = get_hf_file_metadata(
1547 url=url, proxies=proxies, timeout=etag_timeout, headers=headers, token=token, endpoint=endpoint
1548 )
1549 except EntryNotFoundError as http_error:
File /usr/conda/envs/custom/ocr-python-311/lib/python3.11/site-packages/huggingface_hub/utils/_validators.py:114, in validate_hf_hub_args.._inner_fn(*args, **kwargs)
112 kwargs = smoothly_deprecate_use_auth_token(fn_name=fn.name, has_token=has_token, kwargs=kwargs)
--> 114 return fn(*args, **kwargs)
File /usr/conda/envs/custom/ocr-python-311/lib/python3.11/site-packages/huggingface_hub/file_download.py:1463, in get_hf_file_metadata(url, token, proxies, timeout, library_name, library_version, user_agent, headers, endpoint)
1462 # Retrieve metadata
-> 1463 r = _request_wrapper(
1464 method="HEAD",
1465 url=url,
1466 headers=hf_headers,
1467 allow_redirects=False,
1468 follow_relative_redirects=True,
1469 proxies=proxies,
1470 timeout=timeout,
1471 )
1472 hf_raise_for_status(r)
File /usr/conda/envs/custom/ocr-python-311/lib/python3.11/site-packages/huggingface_hub/file_download.py:286, in _request_wrapper(method, url, follow_relative_redirects, **params)
285 if follow_relative_redirects:
--> 286 response = _request_wrapper(
287 method=method,
288 url=url,
289 follow_relative_redirects=False,
290 **params,
291 )
293 # If redirection, we redirect only relative paths.
294 # This is useful in case of a renamed repository.
File /usr/conda/envs/custom/ocr-python-311/lib/python3.11/site-packages/huggingface_hub/file_download.py:309, in _request_wrapper(method, url, follow_relative_redirects, **params)
308 # Perform request and return if status_code is not in the retry list.
--> 309 response = http_backoff(method=method, url=url, **params, retry_on_exceptions=(), retry_on_status_codes=(429,))
310 hf_raise_for_status(response)
File /usr/conda/envs/custom/ocr-python-311/lib/python3.11/site-packages/huggingface_hub/utils/_http.py:310, in http_backoff(method, url, max_retries, base_wait_time, max_wait_time, retry_on_exceptions, retry_on_status_codes, **kwargs)
309 # Perform request and return if status_code is not in the retry list.
--> 310 response = session.request(method=method, url=url, **kwargs)
311 if response.status_code not in retry_on_status_codes:
File /usr/conda/envs/custom/ocr-python-311/lib/python3.11/site-packages/requests/sessions.py:589, in Session.request(self, method, url, params, data, headers, cookies, files, auth, timeout, allow_redirects, proxies, hooks, stream, verify, cert, json)
588 send_kwargs.update(settings)
--> 589 resp = self.send(prep, **send_kwargs)
591 return resp
File /usr/conda/envs/custom/ocr-python-311/lib/python3.11/site-packages/requests/sessions.py:703, in Session.send(self, request, **kwargs)
702 # Send the request
--> 703 r = adapter.send(request, **kwargs)
705 # Total elapsed time of the request (approximately)
File /usr/conda/envs/custom/ocr-python-311/lib/python3.11/site-packages/huggingface_hub/utils/_http.py:96, in UniqueRequestIdAdapter.send(self, request, *args, **kwargs)
95 try:
---> 96 return super().send(request, *args, **kwargs)
97 except requests.RequestException as e:
File /usr/conda/envs/custom/ocr-python-311/lib/python3.11/site-packages/requests/adapters.py:677, in HTTPAdapter.send(self, request, stream, timeout, verify, cert, proxies)
675 raise SSLError(e, request=request)
--> 677 raise ConnectionError(e, request=request)
679 except ClosedPoolError as e:
ConnectionError: (MaxRetryError("HTTPSConnectionPool(host='huggingface.co', port=443): Max retries exceeded with url: /timm/resnet18.a1_in1k/resolve/main/pytorch_model.bin (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f0edb3e2650>: Failed to establish a new connection: [Errno 101] Network is unreachable'))"), '(Request ID: 45522ad1-34f6-423f-b2bf-b6213133a716)')
The above exception was the direct cause of the following exception:
LocalEntryNotFoundError Traceback (most recent call last)
Cell In[16], line 13
10 os.chmod(cache_dir, 0o755)
12 # Charger le modèle à partir du cache local
---> 13 model = TableTransformerForObjectDetection.from_pretrained(
14 pretrained_model_name_or_path="microsoft/table-transformer-detection",
15 local_files_only=True,
16 cache_dir=cache_dir
17 )
19 print("Modèle chargé avec succès !")
File /usr/conda/envs/custom/ocr-python-311/lib/python3.11/site-packages/transformers/modeling_utils.py:317, in restore_default_torch_dtype.._wrapper(*args, **kwargs)
315 old_dtype = torch.get_default_dtype()
316 try:
--> 317 return func(*args, **kwargs)
318 finally:
319 torch.set_default_dtype(old_dtype)
File /usr/conda/envs/custom/ocr-python-311/lib/python3.11/site-packages/transformers/modeling_utils.py:4999, in PreTrainedModel.from_pretrained(cls, pretrained_model_name_or_path, config, cache_dir, ignore_mismatched_sizes, force_download, local_files_only, token, revision, use_safetensors, weights_only, *model_args, **kwargs)
4996 config = copy.deepcopy(config) # We do not want to modify the config inplace in from_pretrained.
4997 with ContextManagers(model_init_context):
4998 # Let's make sure we don't run the init function of buffer modules
-> 4999 model = cls(config, *model_args, **model_kwargs)
5001 # Make sure to tie the weights correctly
5002 model.tie_weights()
File /usr/conda/envs/custom/ocr-python-311/lib/python3.11/site-packages/transformers/models/table_transformer/modeling_table_transformer.py:1179, in TableTransformerForObjectDetection.init(self, config)
1176 super().init(config)
1178 # DETR encoder-decoder model
-> 1179 self.model = TableTransformerModel(config)
1181 # Object detection heads
1182 self.class_labels_classifier = nn.Linear(
1183 config.d_model, config.num_labels + 1
1184 ) # We add one for the "no object" class
File /usr/conda/envs/custom/ocr-python-311/lib/python3.11/site-packages/transformers/models/table_transformer/modeling_table_transformer.py:1005, in TableTransformerModel.init(self, config)
1002 super().init(config)
1004 # Create backbone + positional encoding
-> 1005 backbone = TableTransformerConvEncoder(config)
1006 object_queries = build_position_encoding(config)
1007 self.backbone = TableTransformerConvModel(backbone, object_queries)
File /usr/conda/envs/custom/ocr-python-311/lib/python3.11/site-packages/transformers/models/table_transformer/modeling_table_transformer.py:225, in TableTransformerConvEncoder.init(self, config)
223 if config.dilation:
224 kwargs["output_stride"] = kwargs.get("output_stride", 16)
--> 225 backbone = create_model(
226 config.backbone,
227 pretrained=config.use_pretrained_backbone,
228 features_only=True,
229 out_indices=out_indices,
230 in_chans=num_channels,
231 **kwargs,
232 )
233 else:
234 backbone = load_backbone(config)
File /usr/conda/envs/custom/ocr-python-311/lib/python3.11/site-packages/timm/models/_factory.py:126, in create_model(model_name, pretrained, pretrained_cfg, pretrained_cfg_overlay, checkpoint_path, cache_dir, scriptable, exportable, no_jit, **kwargs)
124 create_fn = model_entrypoint(model_name)
125 with set_layer_config(scriptable=scriptable, exportable=exportable, no_jit=no_jit):
--> 126 model = create_fn(
127 pretrained=pretrained,
128 pretrained_cfg=pretrained_cfg,
129 pretrained_cfg_overlay=pretrained_cfg_overlay,
130 cache_dir=cache_dir,
131 **kwargs,
132 )
134 if checkpoint_path:
135 load_checkpoint(model, checkpoint_path)
File /usr/conda/envs/custom/ocr-python-311/lib/python3.11/site-packages/timm/models/resnet.py:1340, in resnet18(pretrained, **kwargs)
1337 """Constructs a ResNet-18 model.
1338 """
1339 model_args = dict(block=BasicBlock, layers=(2, 2, 2, 2))
-> 1340 return _create_resnet('resnet18', pretrained, **dict(model_args, **kwargs))
File /usr/conda/envs/custom/ocr-python-311/lib/python3.11/site-packages/timm/models/resnet.py:642, in _create_resnet(variant, pretrained, **kwargs)
641 def _create_resnet(variant, pretrained: bool = False, **kwargs) -> ResNet:
--> 642 return build_model_with_cfg(ResNet, variant, pretrained, **kwargs)
File /usr/conda/envs/custom/ocr-python-311/lib/python3.11/site-packages/timm/models/_builder.py:436, in build_model_with_cfg(model_cls, variant, pretrained, pretrained_cfg, pretrained_cfg_overlay, model_cfg, feature_cfg, pretrained_strict, pretrained_filter_fn, cache_dir, kwargs_filter, **kwargs)
434 num_classes_pretrained = 0 if features else getattr(model, 'num_classes', kwargs.get('num_classes', 1000))
435 if pretrained:
--> 436 load_pretrained(
437 model,
438 pretrained_cfg=pretrained_cfg,
439 num_classes=num_classes_pretrained,
440 in_chans=kwargs.get('in_chans', 3),
441 filter_fn=pretrained_filter_fn,
442 strict=pretrained_strict,
443 cache_dir=cache_dir,
444 )
446 # Wrap the model in a feature extraction module if enabled
447 if features:
File /usr/conda/envs/custom/ocr-python-311/lib/python3.11/site-packages/timm/models/_builder.py:213, in load_pretrained(model, pretrained_cfg, num_classes, in_chans, filter_fn, strict, cache_dir)
211 state_dict = load_state_dict_from_hf(*pretrained_loc, cache_dir=cache_dir)
212 else:
--> 213 state_dict = load_state_dict_from_hf(pretrained_loc, weights_only=True, cache_dir=cache_dir)
214 else:
215 model_name = pretrained_cfg.get('architecture', 'this model')
File /usr/conda/envs/custom/ocr-python-311/lib/python3.11/site-packages/timm/models/_hub.py:225, in load_state_dict_from_hf(model_id, filename, weights_only, cache_dir)
222 pass
224 # Otherwise, load using pytorch.load
--> 225 cached_file = hf_hub_download(
226 hf_model_id,
227 filename=filename,
228 revision=hf_revision,
229 cache_dir=cache_dir,
230 )
231 _logger.debug(f"[{model_id}] Safe alternative not found for '{filename}'. Loading weights using default pytorch.")
232 try:
File /usr/conda/envs/custom/ocr-python-311/lib/python3.11/site-packages/huggingface_hub/utils/_validators.py:114, in validate_hf_hub_args.._inner_fn(*args, **kwargs)
111 if check_use_auth_token:
112 kwargs = smoothly_deprecate_use_auth_token(fn_name=fn.name, has_token=has_token, kwargs=kwargs)
--> 114 return fn(*args, **kwargs)
File /usr/conda/envs/custom/ocr-python-311/lib/python3.11/site-packages/huggingface_hub/file_download.py:1010, in hf_hub_download(repo_id, filename, subfolder, repo_type, revision, library_name, library_version, cache_dir, local_dir, user_agent, force_download, proxies, etag_timeout, token, local_files_only, headers, endpoint, resume_download, force_filename, local_dir_use_symlinks)
990 return _hf_hub_download_to_local_dir(
991 # Destination
992 local_dir=local_dir,
(...) 1007 local_files_only=local_files_only,
1008 )
1009 else:
-> 1010 return _hf_hub_download_to_cache_dir(
1011 # Destination
1012 cache_dir=cache_dir,
1013 # File info
1014 repo_id=repo_id,
1015 filename=filename,
1016 repo_type=repo_type,
1017 revision=revision,
1018 # HTTP info
1019 endpoint=endpoint,
1020 etag_timeout=etag_timeout,
1021 headers=hf_headers,
1022 proxies=proxies,
1023 token=token,
1024 # Additional options
1025 local_files_only=local_files_only,
1026 force_download=force_download,
1027 )
File /usr/conda/envs/custom/ocr-python-311/lib/python3.11/site-packages/huggingface_hub/file_download.py:1117, in _hf_hub_download_to_cache_dir(cache_dir, repo_id, filename, repo_type, revision, endpoint, etag_timeout, headers, proxies, token, local_files_only, force_download)
1114 return pointer_path
1116 # Otherwise, raise appropriate error
-> 1117 _raise_on_head_call_error(head_call_error, force_download, local_files_only)
1119 # From now on, etag, commit_hash, url and size are not None.
1120 assert etag is not None, "etag must have been retrieved from server"
File /usr/conda/envs/custom/ocr-python-311/lib/python3.11/site-packages/huggingface_hub/file_download.py:1661, in _raise_on_head_call_error(head_call_error, force_download, local_files_only)
1658 raise head_call_error
1659 else:
1660 # Otherwise: most likely a connection issue or Hub downtime => let's warn the user
-> 1661 raise LocalEntryNotFoundError(
1662 "An error happened while trying to locate the file on the Hub and we cannot find the requested files"
1663 " in the local cache. Please check your connection and try again or make sure your Internet connection"
1664 " is on."
1665 ) from head_call_error
LocalEntryNotFoundError: An error happened while trying to locate the file on the Hub and we cannot find the requested files in the local cache. Please check your connection and try again or make sure your Internet connection is on.`