Description
Is there an existing issue for this?
- I have searched the existing issues
Category of Bug / Issue
Application crashed
Current Behavior
Currently executing Remorph table reconn processes with "TABLE_THRESHOLDS" set fails for " 'dict' object has no attribute 'model'".
TABLE_THRESHOLDS = [
{
"lower_bound": "1%",
"upper_bound": "5%",
"model": "mismatch"
}
]
ERROR:
18:17:44 ERROR [RocketRemorphRecon] <built-in method with_traceback of AttributeError object at 0x7cf81ab70f90>
18:17:44 ERROR [RocketRemorphRecon] Exception : 'dict' object has no attribute 'model'
18:17:44 INFO [RocketRemorphRecon] ***************************
18:17:44 ERROR [RocketRemorphRecon] <built-in method with_traceback of AttributeError object at 0x7cf81ab70f90>
Recon Code:
table_setup = TableRecon(
source_schema=TEMP_SCHEMA,
target_catalog=REMORPH_CATALOG,
target_schema=TEMP_SCHEMA,
tables=[
Table(
source_name=hudi_temp_view,
target_name=delta_temp_view,
join_columns = primary_keys,
drop_columns = DROP_COLUMNS,
table_thresholds = TABLE_THRESHOLDS
)
],
)
try:
result = recon(ws, spark, table_setup, reconcile_data)
recon_id = result.recon_id
logger.info(f"Reconcilation Data successful : {recon_id}")
logger.info("***************************")
except Exception as e:
logger.error(f"Exception : {str(e)}")
Expected Behavior
No response
Steps To Reproduce
No response
Relevant log output or Exception details
Sample Query
Operating System
macOS
Version
latest via Databricks CLI