Skip to content

Resizing storage on new images #5096

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 5 commits into
base: master
Choose a base branch
from

Conversation

antonlacon
Copy link
Contributor

@antonlacon antonlacon commented Feb 5, 2021

This series works to eliminate the usage of populatefs in editing the storage partition prior to deployment on devices. Populatefs, to my knowledge, is limited to ext2-4 filesystems. Every example I've seen of its usage was for ext4 filesystems. Removing it is a stepping stone to using other filesystems for /storage.

The series works by shifting the check into mount_storage() of busybox's init script. It checks to see if the /storage partition is still the original 32MiB size done at image creation, and if so, sets the .please_resize_me marker. blockdev and findfs are added to busybox:init to do this check (about +15KiB, iirc). The check is ignored when STORAGE_NETBOOT=yes, done when cmdline has disk={ISCI,NBD,NFS}=....

Alternative approaches considered:

Instead of checking if /storage was the original size, it would instead check for something like storage-payload.tar existing in /flash, and untarring it when found. It would then be deleted once untarred. The tarball would only be created when building .img's. This would be able to hold whatever could fit into /storage's default size, which is I suppose more flexible.The tarball would be ignored when updating systems by way of an img file. However, populatefs was supposed to give the same flexibility when it was set up, and no further files have been included since, so that flexibility may be unnecessary.

Using a systemd generator to do one of the above. This seems more complicated, and not doing so saves a reboot.

Prebuilding the /storage partition blocks for the buildsystem to stitch together when creating the .img. Again, this seemed more complicated when compared to one of the first two.

Other changes in here (if this PR ends up LE11 (expected), these commits should be split):

Refresh of the busybox configs.

Dropping of pivot_root from busybox:init. Documentation for this says it's intended for linux 2.4 systems using initrd. init is using switch_root, which is intended for linux >=2.6 using initramfs.

Tested on a new RPi3 image (this is the UUID path). Generic/virtual (for /* disks) and whatever uses /dev/ubi need testing.

Open questions from doing this:

busybox:target has switch_root. I'm not sure the use case for this. Remove?

I believe e2fsprogs:host would be orphaned. Remove?

@antonlacon
Copy link
Contributor Author

antonlacon commented Aug 29, 2021

Some updates pushed. Mainly, it moves the STORAGE_SIZE variable out of the image script and into distro options. It then uses that for the resize marker in busybox's init. It updates the show_config function to give information about the filesystems of the image being made.

Two minors are the subshell removal in show_config and having a warning in busybox's init refer to SYSTEM_SIZE.

Still draft to see if I can find a way to check the storage drive for unallocated space as part of its condition in creating the resize marker., and probably look to add a check to abort the build if STORAGE_SIZE isn't set now that it's not set in stone by the image script.

@CvH
Copy link
Member

CvH commented Dec 25, 2021

@antonlacon this needs a rebase

anyone else, pls review after rebase

@antonlacon
Copy link
Contributor Author

I've rebased and squashed some. I've tested booting an active install, but need to test a clean image again, so draft is still on.

I would like to drop the last two commits that include parted into init (formerly optional). It's intended as a safeguard against a potential bootloop if someone were to find themselves in a situation where SYSTEM_SIZE + SYSTEMS_SIZE_OFFSET + STORAGE_SIZE == the size of their install media. I think this can be solved with a "don't be foolish" comment where STORAGE_SIZE is set (that's added), but wanted to show that checking unallocated would be done.

Otherwise, this should be reviewable/RFC.

@antonlacon
Copy link
Contributor Author

antonlacon commented Dec 31, 2021

Looking to replace findfs. I need to get to the parent device node of $disk instead of $disk's device node for the parted portion.

@antonlacon
Copy link
Contributor Author

antonlacon commented Apr 16, 2023

Rebased on #7036; last tested 2023.04.15. I'm still thinking regarding the inclusion of parted. May opt for putting something on the kernel command line that skips the resize check instead.

These are used to determine whether to add a resize marker.

Signed-off-by: Ian Leonard <[email protected]>
Add STORAGE_SIZE as part of the general buildsystem, like SYSTEM_SIZE.

Signed-off-by: Ian Leonard <[email protected]>
Signed-off-by: Ian Leonard <[email protected]>
This check is to prevent a boot loop should someone set
SYSTEM_SIZE + STORAGE_SIZE to equal their memory device, or
/storage could not otherwise expand.

Determine parent node by taking STORAGE_TARGET, and removing the last
character. It then checks if the new string is a valid block device.
It continues to do so until the string is reduced to "/dev/", in the
event that a disk has many partitions.

A GPT disk could have up to 128 partitions (possibly more) and breaking
the loop early may yeild a false positive. /dev/sda100 would also trigger
on /dev/sda10 and /dev/sda1 if broken early, while we want /dev/sda.

We're interested in the unallocated space at the end of the device, so
be sure to take the last "Free Space" line output from parted when
parsing its output.

Signed-off-by: Ian Leonard <[email protected]>
@chewitt
Copy link
Member

chewitt commented May 2, 2024

I believe this has been superseded by #8817 ?

@antonlacon
Copy link
Contributor Author

Not superseded. This is changing how /storage/.please_resize_me gets created.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants