Commit daed157
authored
Logging failures to refresh metadata (#385)
## Summary
problem - when an exception throws under
`super.refreshFromMetadataLocation` , there is no failure log which
shows how long it took to fail. Adding logging here for the total
duration Helps us measure how long a request takes when it fails to
retrieve metadata from hdfs. This is a common cause for 504 gateway
timeouts. Without this, we can only guess the total duration by
correlating it with metrics. The metrics don’t have a failure/success
dimension, and also cannot be isolated to a single request/table
metadata file.
solution - try/catch and log before throwing the existing error. There
is another PR which aims to reduce the total duration via better
configured timeouts+retries at hdfs layer
#386
At the same time, adding a missing log message for catalog update, for
future operations
## Changes
- [ ] Client-facing API Changes
- [ ] Internal API Changes
- [X] Bug Fixes
- [ ] New Features
- [ ] Performance Improvements
- [ ] Code Style
- [ ] Refactoring
- [ ] Documentation
- [ ] Tests
## Testing Done
existing tests should cover this
- [ ] Manually Tested on local docker setup. Please include commands
ran, and their output.
- [ ] Added new tests for the changes made.
- [ ] Updated existing tests to reflect the changes made.
- [X] No tests added or updated. Please explain why. If unsure, please
feel free to ask for help.
- [ ] Some other form of testing like staging or soak time in
production. Please explain.
# Additional Information
- [ ] Breaking Changes
- [ ] Deprecations
- [ ] Large PR broken into smaller PRs, and PR plan linked in the
description.1 parent 0960244 commit daed157
File tree
1 file changed
+38
-14
lines changed- iceberg/openhouse/internalcatalog/src/main/java/com/linkedin/openhouse/internal/catalog
1 file changed
+38
-14
lines changedLines changed: 38 additions & 14 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
127 | 127 | | |
128 | 128 | | |
129 | 129 | | |
130 | | - | |
131 | | - | |
132 | | - | |
133 | | - | |
134 | | - | |
| 130 | + | |
| 131 | + | |
| 132 | + | |
| 133 | + | |
| 134 | + | |
| 135 | + | |
| 136 | + | |
| 137 | + | |
| 138 | + | |
| 139 | + | |
| 140 | + | |
| 141 | + | |
| 142 | + | |
| 143 | + | |
| 144 | + | |
| 145 | + | |
| 146 | + | |
| 147 | + | |
135 | 148 | | |
136 | | - | |
137 | | - | |
138 | | - | |
139 | | - | |
140 | 149 | | |
141 | 150 | | |
142 | 151 | | |
| |||
284 | 293 | | |
285 | 294 | | |
286 | 295 | | |
287 | | - | |
288 | | - | |
289 | | - | |
290 | | - | |
291 | | - | |
| 296 | + | |
| 297 | + | |
| 298 | + | |
| 299 | + | |
| 300 | + | |
| 301 | + | |
| 302 | + | |
| 303 | + | |
| 304 | + | |
| 305 | + | |
| 306 | + | |
| 307 | + | |
| 308 | + | |
| 309 | + | |
| 310 | + | |
| 311 | + | |
| 312 | + | |
| 313 | + | |
| 314 | + | |
| 315 | + | |
292 | 316 | | |
293 | 317 | | |
294 | 318 | | |
| |||
0 commit comments