[read-unfinalized-object] [part1] Relax file-cache checks for unfinalized objects

[gargnitingoogle]

Description

The change is based on design doc

Changes in behavior for unfinalized objects:

• Don't invalidate file-cache entry if size of completed download entry >= intended downloaded size
• Don't fall back to GCS from file-cache read if size of completed download entry >= intended upper limit of offset to be read.
• Fallback to GCS while reading from cache_handle if requested read isn't available from the cached file-info data size. If partial read is possible from the cache, then go ahead and start the download job.

Additional changes

• Added a simple utility for easily checking if an object is unfinalized.
• Enable zonal bucket unit tests for TestFileCacheReader, TestRandomReader and TestReadManager.
• Add new unit tests corresponding to the above code changes.
• Update existing unit tests corresponding to the above code changes.

Dependencies

• This builds on top of zero byte read to fetch size for unfinalized obj #3387 and is thus blocked for merge until that PR is merged

Out of scope:

• This does not add support for reading unfinalized objects itself. That's covered in the in [read-unfinalized-object] [part2] Enable read of unfinalized objects while being written via streaming #3405 .
• The changes in this don't touch any non-zonal non-unfinalized objects in any way.

Link to the issue in case of a bug fix.

b/421294827

Testing details

1. Manual - Tested manually.
2. Unit tests - Tested automatically by this PR. Add more UTs for the changes in the code.
3. Integration tests - ran with presubmit (only zonal, e2e zonal, non-zonal and perf test run)
4. Perf tests results

+--------+------------+--------------+------------+--------------+--------------+
| Branch | File Size  |   Read BW    |  Write BW  | RandRead BW  | RandWrite BW |
+--------+------------+--------------+------------+--------------+--------------+
| Master |  0.25MiB   | 555.31MiB/s  |  1.2MiB/s  |  77.32MiB/s  |  1.17MiB/s   |
|   PR   |  0.25MiB   | 581.98MiB/s  | 1.21MiB/s  |  77.26MiB/s  |   1.2MiB/s   |
|        |            |              |            |              |              |
|        |            |              |            |              |              |
| Master | 48.828MiB  | 3900.45MiB/s | 85.61MiB/s | 1554.19MiB/s |  84.39MiB/s  |
|   PR   | 48.828MiB  | 3849.86MiB/s | 85.42MiB/s | 1499.03MiB/s |  84.85MiB/s  |
|        |            |              |            |              |              |
|        |            |              |            |              |              |
| Master | 976.562MiB | 3742.3MiB/s  | 79.22MiB/s | 763.35MiB/s  |  39.24MiB/s  |
|   PR   | 976.562MiB | 3711.34MiB/s | 77.93MiB/s | 1296.53MiB/s |  38.71MiB/s  |
|        |            |              |            |              |              |
|        |            |              |            |              |              |
+--------+------------+--------------+------------+--------------+--------------+

Any backward incompatible change? If so, please explain.

[codecov]

Codecov Report

Attention: Patch coverage is 84.00000% with 4 lines in your changes missing coverage. Please review.

Project coverage is 79.00%. Comparing base (94b76ce) to head (45305ac).
Report is 1 commits behind head on master.

Files with missing lines	Patch %	Lines
internal/cache/file/cache_handle.go	81.81%	3 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #3401      +/-   ##
==========================================
- Coverage   79.07%   79.00%   -0.07%     
==========================================
  Files         140      140              
  Lines       18520    18539      +19     
==========================================
+ Hits        14644    14647       +3     
- Misses       3392     3404      +12     
- Partials      484      488       +4     

Flag	Coverage Δ
unittests	79.00% <84.00%> (-0.07%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.