You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Hi, using deltalake 0.22.3 (and 0.21.0 before that), I'm running a daily job that merges some data to a table, after which I z-order (table.optimize.z_order(attributes)) the table and then vacuum (table.vacuum(dry_run=False) it.
This mostly works fine, but a few times a month, it throws the exception "Delta protocol violation: Generic action error: filed to get tombstones" (the typo not made by me, I assume they mean to throw with message 'failed to get tombstones').
delta.logRetentionDuration in the table configuration is 7 days, and delta.autoOptimize.autoCompact is true. No other table configuration is returned in metadata.
Error seems to be thrown here:
.map_err(|_| ProtocolError::Generic("filed to get tombstones".into()))?
, seems it's trying to create a new checkpoint during the vacuuming, not sure why....
I have no clue why this happens or how to prevent it, does anyone have any suggestions?
reacted with thumbs up emoji reacted with thumbs down emoji reacted with laugh emoji reacted with hooray emoji reacted with confused emoji reacted with heart emoji reacted with rocket emoji reacted with eyes emoji
-
Hi, using deltalake 0.22.3 (and 0.21.0 before that), I'm running a daily job that merges some data to a table, after which I z-order (table.optimize.z_order(attributes)) the table and then vacuum (table.vacuum(dry_run=False) it.
This mostly works fine, but a few times a month, it throws the exception "Delta protocol violation: Generic action error: filed to get tombstones" (the typo not made by me, I assume they mean to throw with message 'failed to get tombstones').
delta.logRetentionDuration in the table configuration is 7 days, and delta.autoOptimize.autoCompact is true. No other table configuration is returned in metadata.
Error seems to be thrown here:
delta-rs/crates/core/src/protocol/checkpoints.rs
Line 182 in 4ef9fb3
I have no clue why this happens or how to prevent it, does anyone have any suggestions?
Beta Was this translation helpful? Give feedback.
All reactions