Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Doc: Add DELETE ORPHAN-FILES example to Hive page #11896

Merged
merged 1 commit into from
Jan 15, 2025

Conversation

ebyhr
Copy link
Contributor

@ebyhr ebyhr commented Jan 1, 2025

Relates to apache/hive#4897

Tested with apache/hive:4.0.1:

0: jdbc:hive2://localhost:10000/> ALTER TABLE employee EXECUTE DELETE ORPHAN-FILES;
INFO  : Compiling command(queryId=hive_20250101074601_ee489833-e643-4cc9-9c0d-338192d5270f): ALTER TABLE employee EXECUTE DELETE ORPHAN-FILES
INFO  : Semantic Analysis Completed (retrial = false)
INFO  : Created Hive schema: Schema(fieldSchemas:null, properties:null)
INFO  : Completed compiling command(queryId=hive_20250101074601_ee489833-e643-4cc9-9c0d-338192d5270f); Time taken: 0.03 seconds
INFO  : Concurrency mode is disabled, not creating a lock manager
INFO  : Executing command(queryId=hive_20250101074601_ee489833-e643-4cc9-9c0d-338192d5270f): ALTER TABLE employee EXECUTE DELETE ORPHAN-FILES
INFO  : Starting task [Stage-0:DDL] in serial mode
INFO  : Completed executing command(queryId=hive_20250101074601_ee489833-e643-4cc9-9c0d-338192d5270f); Time taken: 0.061 seconds
No rows affected (0.101 seconds)
0: jdbc:hive2://localhost:10000/> ALTER TABLE employee EXECUTE DELETE ORPHAN-FILES OLDER THAN ('2021-12-09 05:39:18.689000000');
INFO  : Compiling command(queryId=hive_20250101075012_80a3545a-8dab-4441-8e02-b572b503ccf3): ALTER TABLE employee EXECUTE DELETE ORPHAN-FILES OLDER THAN ('2021-12-09 05:39:18.689000000')
INFO  : Semantic Analysis Completed (retrial = false)
INFO  : Created Hive schema: Schema(fieldSchemas:null, properties:null)
INFO  : Completed compiling command(queryId=hive_20250101075012_80a3545a-8dab-4441-8e02-b572b503ccf3); Time taken: 0.028 seconds
INFO  : Concurrency mode is disabled, not creating a lock manager
INFO  : Executing command(queryId=hive_20250101075012_80a3545a-8dab-4441-8e02-b572b503ccf3): ALTER TABLE employee EXECUTE DELETE ORPHAN-FILES OLDER THAN ('2021-12-09 05:39:18.689000000')
INFO  : Starting task [Stage-0:DDL] in serial mode
INFO  : Completed executing command(queryId=hive_20250101075012_80a3545a-8dab-4441-8e02-b572b503ccf3); Time taken: 0.044 seconds
No rows affected (0.08 seconds)

@github-actions github-actions bot added the docs label Jan 1, 2025
@ebyhr ebyhr changed the title Doc: Add DELETE ORPHAN-FILES example Doc: Add DELETE ORPHAN-FILES example to Hive page Jan 1, 2025
@ebyhr
Copy link
Contributor Author

ebyhr commented Jan 15, 2025

@pvary Can you take a look at this PR?

@pvary
Copy link
Contributor

pvary commented Jan 15, 2025

@ebyhr: Is it available in Hive 4.0.0? Do we have other maintenance features, like compaction, manifest rewrite, expire snapshot that are available in Hive as well?

@ebyhr
Copy link
Contributor Author

ebyhr commented Jan 15, 2025

Is it available in Hive 4.0.0?

Yes. Fix version in https://issues.apache.org/jira/browse/HIVE-27906 is 4.0.0. I confirmed the procedure with docker just in case.

Do we have other maintenance features, like compaction, manifest rewrite, expire snapshot that are available in Hive as well?

The compaction (OPTIMIZE TABLE table REWRITE DATA) and expiring snapshots ALTER TABLE table EXECUTE EXPIRE_SNAPSHOTS(...) are supported. They are already documented. Manifest rewrite is unsupported as far as I know.

@pvary pvary merged commit 101e650 into apache:main Jan 15, 2025
2 checks passed
@pvary
Copy link
Contributor

pvary commented Jan 15, 2025

Merged to main.
Thanks for the PR @ebyhr!

@ebyhr ebyhr deleted the ebi/doc-hive-delete-orphan-files branch January 15, 2025 11:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants