feat(virtio-block): Add fallocate support#5511
feat(virtio-block): Add fallocate support#5511LDagnachew wants to merge 4 commits intofirecracker-microvm:mainfrom
Conversation
|
also disclaimer this still needs to be style checked, but this should be the near final implementation, other than writing a more rigorous testing suite. |
41e7941 to
abda29f
Compare
|
@bchalios @ShadowCurse whenever you have time do y'all mind taking a look? I really want to bring this issue to the finish line, thanks! |
ShadowCurse
left a comment
There was a problem hiding this comment.
Sorry for late review. Did a first pass and left some comments. One thing we need to think about is the fact that fallocate is not supported on all filesystems, so either we need to make this feature optional for the device, or make failures to fallocate to just print error logs and not return actual errors.
| if end_sector > num_disk_sectors { | ||
| return Err(VirtioBlockError::BeyondDiskSize); | ||
| } |
There was a problem hiding this comment.
Instead of returnning an error, you can just ignore invalid segment and print an error log.
There was a problem hiding this comment.
My bad, this was incorrect suggestion, we better to abort the whole request if we find invalid segment, so Err(..) this should be.
|
Hi, sorry for the late response, but I will get this handled within the next day or two and mark for another review. Thanks! |
8fd863f to
dacdcff
Compare
- Updated unmap/fstrim/discard deprecated logic. Signed-off-by: LDagnachew <leulmdagnachew@gmail.com>
- fallocate unblocked in seccomp, thus allowing discard requests. Signed-off-by: LDagnachew <leulmdagnachew@gmail.com>
Update Python files to comply with Black and pylint checks. These changes are formatting-only and do not alter functionality. Signed-off-by: LDagnachew <leulmdagnachew@gmail.com>
- added safety comments + minor error handling. Signed-off-by: LDagnachew <leulmdagnachew@gmail.com>
f23cb9b to
dd0a7e1
Compare
ShadowCurse
left a comment
There was a problem hiding this comment.
On top of everything, we need to figure out the way to test if we can even call fallocate on the file since some filesystems do not support this operation. Most likely we should do a dummy fallocate call that does not actually change the file as a check.
Also this feature should be user configurable, just like others, so the block API should have a new field to enable this feature.
| @@ -27,7 +27,7 @@ use crate::devices::virtio::block::CacheType; | |||
| use crate::devices::virtio::block::virtio::metrics::{BlockDeviceMetrics, BlockMetricsPerDevice}; | |||
There was a problem hiding this comment.
commit: style: fix Python formatting and lint issues
everything here belongs to the first implementation commit. Please don't do separate "running a formatter" type of commits
| @@ -110,6 +110,7 @@ impl AsyncFileEngine { | |||
| Ok(()) | |||
There was a problem hiding this comment.
commit: style: replace errors for fallocate/discard with error logs
same here, merge this into main implementation commit.
| opcode: OpCode::Fallocate, | ||
| addr: None, | ||
| len: Some(len), | ||
| flags: 0, |
There was a problem hiding this comment.
libc::FALLOC_FL_PUNCH_HOLE | libc::FALLOC_FL_KEEP_SIZE should be in flags
| if req.r#type != RequestType::Flush { | ||
| return Err(VirtioBlockError::DescriptorChainTooShort); | ||
| } | ||
| data_desc = desc; |
| status_desc = data_desc | ||
| .next_descriptor() | ||
| .ok_or(VirtioBlockError::DescriptorChainTooShort)?; | ||
| if !status_desc.is_write_only() { | ||
| return Err(VirtioBlockError::UnexpectedReadOnlyDescriptor); | ||
| } |
There was a problem hiding this comment.
status_desc is already obtained from the data_desc above and there is a check for status_desc just bellow.
| let off_i64 = i64::try_from(offset).map_err(|_| { | ||
| SyncIoError::Discard(std::io::Error::new( | ||
| std::io::ErrorKind::InvalidInput, | ||
| "offset overflow", | ||
| )) | ||
| })?; |
There was a problem hiding this comment.
I wonder if there is a way to remove this. The syscall wants, i64, but in reality it is just 64 bit value which will be checked by the kernel in any case.
| vm.ssh.run("mount -o remount,discard /") | ||
| exit_code, stdout, _ = vm.ssh.run("mount | grep /dev/vda") | ||
| assert ( | ||
| exit_code == 0 | ||
| ), f"Failed to remount root filesystem with discard option: {stdout}" | ||
|
|
||
| exit_code, stdout, _ = vm.ssh.run("fstrim -v /") | ||
| assert exit_code == 0, f"fstrim -v failed: {stdout}" | ||
| assert "trimmed" in stdout, f"Unexpected fstrim output: {stdout}" |
There was a problem hiding this comment.
- You test wrong block device (should be
vdbfor thetest_diskyou added). - There is no check for the actual
discardcapability aftermount | grep ...command. - What should happen after
fstrimis called? How do you test that thediscrardwas correctly executed? Maybe you can check FC metrics and check the underlyingtest_diskwithstator smth to see if the hole was successfully created.
| exit_code, stdout, _ = vm.ssh.run("fstrim -v /") | ||
| assert exit_code == 0, f"fstrim -v failed: {stdout}" | ||
| assert "trimmed" in stdout, f"Unexpected fstrim output: {stdout}" | ||
| vm.kill() |
There was a problem hiding this comment.
There is no need to explicitly kill the vm. It will be killed automatically at the end of the test
| import host_tools.drive as drive_tools | ||
|
|
||
|
|
||
| def test_discard_support(uvm_plain_rw): |
There was a problem hiding this comment.
Because discard is implemented in both sync and async engines, you need to parametrize the test.
| if end_sector > num_disk_sectors { | ||
| return Err(VirtioBlockError::BeyondDiskSize); | ||
| } |
There was a problem hiding this comment.
My bad, this was incorrect suggestion, we better to abort the whole request if we find invalid segment, so Err(..) this should be.
Changes
fallocate.Reason
Attempting to resolve Issue #2708.
License Acceptance
By submitting this pull request, I confirm that my contribution is made under
the terms of the Apache 2.0 license. For more information on following Developer
Certificate of Origin and signing off your commits, please check
CONTRIBUTING.md.PR Checklist
tools/devtool checkbuild --allto verify that the PR passesbuild checks on all supported architectures.
tools/devtool checkstyleto verify that the PR passes theautomated style checks.
how they are solving the problem in a clear and encompassing way.
in the PR.
CHANGELOG.md.Runbook for Firecracker API changes.
integration tests.
TODO.rust-vmm.