-
Notifications
You must be signed in to change notification settings - Fork 1.6k
Description
Describe the bug
Sometimes when running suite.add_expectation() we get the following error:
[Errno 22] Invalid argument
The strange thing is that this happens sometimes. When re-running the code, the error usually disappears. Therefore, we are thinking it's a bug within the library.
To Reproduce
Here is the full track trace up to the relevant code:
`
File /local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.12/site-packages/expi/common/client_wrappers/gx_client.py:147, in GreatExpectationsClient.add_expectations(self, suite_name, expectations)
145 suite = self.context.suites.get(f"{suite_name}_suite")
146 for expectation in expectations:
--> 147 suite.add_expectation(expectation)
149 self.context.suites.add_or_update(suite)
File /local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.12/site-packages/great_expectations/core/expectation_suite.py:154, in ExpectationSuite.add_expectation(self, expectation)
151 if not already_added:
152 # suite is a set-like collection, so don't add if it not unique
153 if should_save_expectation:
--> 154 expectation = self._store.add_expectation(suite=self, expectation=expectation)
155 self.expectations.append(expectation)
157 expectation.register_save_callback(save_callback=self._save_expectation)
File /local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.12/site-packages/great_expectations/data_context/store/expectations_store.py:155, in ExpectationsStore.add_expectation(self, suite, expectation)
152 expectation.id = str(uuid.uuid4())
153 fetched_suite.expectations.append(expectation)
--> 155 self.update(key=suite_identifier, value=fetched_suite)
156 if self.cloud_mode:
157 # since update doesn't return the object we need (here), we refetch the suite
158 suite_identifier, fetched_suite = self._refresh_suite(suite)
File /local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.12/site-packages/great_expectations/data_context/store/store.py:312, in Store.update(self, key, value, **kwargs)
308 def update(self, key: DataContextKey, value: Any, **kwargs) -> None:
309 """
310 Essentially set
but validates that a given key-value pair does already exist.
311 """
--> 312 return self._update(key=key, value=value, **kwargs)
File /local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.12/site-packages/great_expectations/data_context/store/expectations_store.py:247, in ExpectationsStore._update(self, key, value, **kwargs)
245 value: ExpectationSuite = self._add_ids_on_update(value)
246 try:
--> 247 result = super()._update(key=key, value=value, **kwargs)
249 if self.cloud_mode:
250 # cloud backend has added IDs, so we update our local state to be in sync
251 assert isinstance(result, GXCloudResourceRef)
File /local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.12/site-packages/great_expectations/data_context/store/store.py:316, in Store._update(self, key, value, **kwargs)
314 def _update(self, key: DataContextKey, value: Any, **kwargs) -> None:
315 self._validate_key(key)
--> 316 return self._store_backend.update(self.key_to_tuple(key), self.serialize(value), **kwargs)
File /local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.12/site-packages/great_expectations/data_context/store/_store_backend.py:152, in StoreBackend.update(self, key, value, **kwargs)
148 def update(self, key, value, **kwargs):
149 """
150 Essentially set
but validates that a given key-value pair does already exist.
151 """
--> 152 return self._update(key=key, value=value, **kwargs)
File /local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.12/site-packages/great_expectations/data_context/store/_store_backend.py:159, in StoreBackend._update(self, key, value, **kwargs)
155 if not self.has_key(key):
156 raise StoreBackendError( # noqa: TRY003 # FIXME CoP
157 f"Store does not have a value associated the following key: {key}."
158 )
--> 159 return self.set(key=key, value=value, **kwargs)
File /local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.12/site-packages/great_expectations/data_context/store/_store_backend.py:132, in StoreBackend.set(self, key, value, **kwargs)
130 # Allow the implementing setter to return something (e.g. a path used for its key)
131 try:
--> 132 return self._set(key, value, **kwargs)
133 except ValueError as e:
134 logger.debug(str(e))
File /local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.12/site-packages/great_expectations/data_context/store/tuple_store_backend.py:325, in TupleFilesystemStoreBackend._set(self, key, value, **kwargs)
322 path, _filename = os.path.split(filepath)
324 os.makedirs(str(path), exist_ok=True) # noqa: PTH103 # FIXME CoP
--> 325 with open(filepath, "wb") as outfile:
326 if isinstance(value, str):
327 outfile.write(value.encode("utf-8"))
OSError: [Errno 22] Invalid argument
Workload failed, see run output for details
`
Expected behavior
We expect the expectation to be added without errors.
Environment (please complete the following information):
- Operating System: Linux
- Great Expectations Version: 1.5.0
- Data Source: Pandas
- Cloud environment: Azure Databricks
Additional context