[Rebuild only] e2fsdroid: Could not allocate block in ext2 filesystem while populating file system

[ypid]

I now hit this issue the third time. This only happens when I make changes in the source code and the then do make build again. Any idea how to solve this instead of deleting out? A clean build does not have this issue.

2019-11-22 18:45:03 - build_image.py - ERROR   : Failed to build out/target/product/sargo/product.img from out/target/product/sargo/product
Out of space? Out of inodes? The tree size of out/target/product/sargo/product is 214323200 bytes (204 MB), with reserved space of 0 bytes (0 MB).
The max image size for filesystem files is 232579072 bytes (221 MB), out of a total partition size of 232579072 bytes (221 MB).
Traceback (most recent call last):
  File "./build/tools/releasetools/build_image.py", line 789, in <module>
    main(sys.argv[1:])
  File "./build/tools/releasetools/build_image.py", line 781, in main
    BuildImage(in_dir, image_properties, out_file, target_out)
  File "./build/tools/releasetools/build_image.py", line 423, in BuildImage
    BuildImageMkfs(in_dir, prop_dict, out_file, target_out, fs_config)
  File "./build/tools/releasetools/build_image.py", line 334, in BuildImageMkfs
    mkfs_output = common.RunAndCheckOutput(build_command)
  File "/home/build/build/base/build/make/tools/releasetools/common.py", line 249, in RunAndCheckOutput
    args, proc.returncode, output))
common.ExternalError: Failed to run command '['mkuserimg_mke2fs', '-s', 'out/target/product/sargo/product', 'out/target/product/sargo/product.img', 'ext4', 'product', '232579072', '-j', '0', '-D', 'out/target/product/sargo/system', '-L', 'product', '-i', '510', '-M', '0', '-c', '--inode_size', '256', 'out/target/product/sargo/obj/ETC/file_contexts.bin_intermediates/file_contexts.bin']' (exit code 4):
18:44:43 mkuserimg_mke2fs.py INFO: Env: {'MKE2FS_CONFIG': './system/extras/ext4_utils/mke2fs.conf'}
18:44:43 mkuserimg_mke2fs.py INFO: Running: mke2fs -O ^has_journal -L product -N 510 -I 256 -M /product -m 0 -E android_sparse -t ext4 -b 4096 out/target/product/sargo/product.img 56782
18:44:43 mkuserimg_mke2fs.py INFO: Env: {}
18:44:43 mkuserimg_mke2fs.py INFO: Running: e2fsdroid -p out/target/product/sargo/system -s -S out/target/product/sargo/obj/ETC/file_contexts.bin_intermediates/file_contexts.bin -f out/target/product/sargo/product -a /product out/target/product/sargo/product.img
18:45:03 mkuserimg_mke2fs.py ERROR: Failed to run e2fsdroid_cmd: __populate_fs: Could not allocate block in ext2 filesystem while writing file "Launcher3QuickStep.apk"
e2fsdroid: Could not allocate block in ext2 filesystem while populating file system

mke2fs 1.44.4 (18-Aug-2018)
Creating filesystem with 56782 4k blocks and 512 inodes
Filesystem UUID: 5c5096c4-7682-48b8-aa09-be6035cdfd33
Superblock backups stored on blocks: 
	32768

Allocating group tables: 0/2   done                            
Writing inode tables: 0/2   done                            
Writing superblocks and filesystem accounting information: 0/2   done


__populate_fs: Could not allocate block in ext2 filesystem while writing file "Launcher3QuickStep.apk"
e2fsdroid: Could not allocate block in ext2 filesystem while populating file system

depmod: WARNING: could not open modules.order at /home/build/build/base/out/target/product/sargo/obj/PACKAGING/depmod_vendor_intermediates/lib/modules/0.0: No such file or directory
depmod: WARNING: could not open modules.builtin at /home/build/build/base/out/target/product/sargo/obj/PACKAGING/depmod_vendor_intermediates/lib/modules/0.0: No such file or directory
19:02:23 ninja failed with: exit status 1
Makefile:37: recipe for target 'build' failed
make: *** [build] Error 1

[ypid]

I hit this issue again (very similar) with the 2020-01 security patches and this time a clean build did not help. If anybody either runs into this same issue or if you are not affected, please comment so that I can either check what I am doing wrong or so that we can together search for a solution.

Build logs: https://gist.github.com/ypid/580160332bf465ee9aa74336e748ea70

$ grep 'failed' ...
FAILED: out/target/product/sargo/obj/PACKAGING/target_files_intermediates/aosp_sargo-target_files-01142217.zip
2020-01-19 00:59:59 - common.py - WARNING : Failed to read PRODUCT_SERVICES/build.prop
2020-01-19 00:59:59 - common.py - WARNING : Failed to read PRODUCT_SERVICES/etc/build.prop
2020-01-19 00:59:59 - common.py - WARNING : Failed to read ODM/build.prop
2020-01-19 00:59:59 - common.py - WARNING : Failed to read ODM/etc/build.prop
01:00:17 mkuserimg_mke2fs.py ERROR: Failed to run e2fsdroid_cmd: __populate_fs: Could not allocate block in ext2 filesystem while writing file "android.hardware.automotive.vehicle@2.0.so"
ExternalError: Failed to run command '['mkuserimg_mke2fs', '-s', '/home/build/build/base/out/soong/.temp/tmporKDRT', '/home/build/build/base/out/target/product/sargo/obj/PACKAGING/target_files_intermediates/aosp_sargo-target_files-01142217/IMAGES/system.img', 'ext4', '/', '564908032', '-j', '0', '-T', '1230768000', '-C', '/home/build/build/base/out/soong/.temp/merged_fs_config1DoORN.txt', '-B', '/home/build/build/base/out/target/product/sargo/obj/PACKAGING/target_files_intermediates/aosp_sargo-target_files-01142217/IMAGES/system.map', '-L', '/', '-i', '3377', '-M', '0', '-U', '4729639d-b5f2-5cc1-a120-9ac5f788683c', '-S', 'fd77d4fd-bb91-5ef5-9533-d5d806da85a3', '-c', '--inode_size', '256', '/home/build/build/base/out/target/product/sargo/obj/PACKAGING/target_files_intermediates/aosp_sargo-target_files-01142217/META/file_contexts.bin']' (exit code 4):
01:00:17 mkuserimg_mke2fs.py ERROR: Failed to run e2fsdroid_cmd: __populate_fs: Could not allocate block in ext2 filesystem while writing file "android.hardware.automotive.vehicle@2.0.so"

�[K
�[K01:00:17 ninja failed with: exit status 1
Makefile:47: recipe for target 'build' failed

cc: @ubergeek77

[ubergeek77]

Does this only happen when you try to do a "re" build against a new security patch? Or is it happening for any rebuild, even if you haven't done a new repo sync?

I've been doing local rebuilds to test some cherry-picks for my device, and I haven't run into anything like this. A full build takes about 3-4 hours on my hardware, but a rebuild with cherry picks takes ~15 minutes. Since my cherry-picked rebuilds are done in that much shorter time frame of 15 minutes, and the resulting images flash just fine, I know for certain that my rebuilds are working properly. However, I haven't gone from one upstream build reference to another, so I'll be able to comment on that when the February one rolls around.

I'm building for crosshatch, so do you think this is device-specific? It looks like you're building for sargo.

I know this is a longshot, but you do have enough disk space, right? That particular error looks like it could happen on lack of space.

[ubergeek77]

That error output from your first comment caught my eye:

Out of space? Out of inodes?

I'm not aware of this, but is there a limit on the number of inodes a Linux filesystem can have, regardless of location? It's a stretch, but perhaps you're hitting this limit. If that was what was going on, deleting out would obviously reduce the number of inodes on your system, so this error wouldn't be hit.

Or, if they aren't talking about your system, but instead the filesystem that the build process is trying to create, maybe there's a bug in the python script that's over-active to the number of files/inodes in the target-files directory, even if ignoring that "limit" would produce a build with no issue.

If you're sure your local filesystem has no issues, then my recommendation would be to see if "e2fsdroid" or "e2fsprogs" has any notable changes between device trees (such as on crosshatch), and if there are, modify your build manifest to pull down crosshatch's branch versions of those instead. If there really is a bug there, considering this isn't happening to me, maybe that will fix it.

[ypid]

I checked the obvious things already I hope.

/dev/vda2        ext4       20G  7.4G   12G  40% /
tmpfs            tmpfs     7.2G     0  7.2G   0% /tmp
/dev/vdb1        btrfs     481G  353G  119G  75% /media/data

aren't talking about your system, but instead the filesystem that the build process is trying to create

That is also my understanding.

I guess I will need to investigate further. Thank you already for your help 👍

[ubergeek77]

Devil's advocate, if you wanted to be extra sure about that inode thing

[ypid]

Thanks, btrfs does not use inodes :)

/dev/vda2        ext4       1.3M  314K  967K   25% /
tmpfs            tmpfs      1.8M    15  1.8M    1% /tmp
/dev/vdb1        btrfs         0     0     0     - /media/data

[ubergeek77]

Oh cool, that's a TIL for me, I didn't know that.

[ypid]

I fixed it in the third clean build. I changed two things:

• Update from 278337a0cf60d223852e8e515cc7f3d53bd73998 to a205d2281b7ef49d3e8e66a38155f58ce234aed2 of kernel/build. It has some abi changes so that might be related.
• make clean. I am not sure if I called that before.

I will leave the issue open for now because the original one has probably a different cause.

[Manouchehri]

I'm running into this issue on coral as well when building AOSP (not the kernel). I'll try m clean and report back when it finally finishes building. =)

2020-06-15 10:25:54 - build_image.py - ERROR   : Failed to build out/target/product/coral/obj/PACKAGING/systemimage_intermediates/system.img from out/target/product/coral/system
Out of space? Out of inodes? The tree size of /mnt/builds/android/out/soong/.temp/tmptr9suL is 496683008 bytes (473 MB), with reserved space of 0 bytes (0 MB).
The max image size for filesystem files is 513454080 bytes (489 MB), out of a total partition size of 513454080 bytes (489 MB).
Traceback (most recent call last):
  File "build/make/tools/releasetools/build_image.py", line 789, in <module>
    main(sys.argv[1:])
  File "build/make/tools/releasetools/build_image.py", line 781, in main
    BuildImage(in_dir, image_properties, out_file, target_out)
  File "build/make/tools/releasetools/build_image.py", line 423, in BuildImage
    BuildImageMkfs(in_dir, prop_dict, out_file, target_out, fs_config)
  File "build/make/tools/releasetools/build_image.py", line 334, in BuildImageMkfs
    mkfs_output = common.RunAndCheckOutput(build_command)
  File "/mnt/builds/android/build/make/tools/releasetools/common.py", line 252, in RunAndCheckOutput
    args, proc.returncode, output))
common.ExternalError: Failed to run command '['mkuserimg_mke2fs', '-s', '/mnt/builds/android/out/soong/.temp/tmptr9suL', 'out/target/product/coral/obj/PACKAGING/systemimage_intermediates/system.img', 'ext4', '/', '513454080', '-j', '0', '-D', 'out/target/product/coral/system', '-L', '/', '-i', '3048', '-M', '0', '-c', '--inode_size', '256', 'out/target/product/coral/obj/ETC/file_contexts.bin_intermediates/file_contexts.bin']' (exit code 4):
10:25:51 mkuserimg_mke2fs.py INFO: Env: {'MKE2FS_CONFIG': './system/extras/ext4_utils/mke2fs.conf'}
10:25:51 mkuserimg_mke2fs.py INFO: Running: mke2fs -O ^has_journal -L / -N 3048 -I 256 -M / -m 0 -E android_sparse -t ext4 -b 4096 out/target/product/coral/obj/PACKAGING/systemimage_intermediates/system.img 125355
10:25:51 mkuserimg_mke2fs.py INFO: Env: {}
10:25:51 mkuserimg_mke2fs.py INFO: Running: e2fsdroid -p out/target/product/coral/system -s -S out/target/product/coral/obj/ETC/file_contexts.bin_intermediates/file_contexts.bin -f /mnt/builds/android/out/soong/.temp/tmptr9suL -a / out/target/product/coral/obj/PACKAGING/systemimage_intermediates/system.img
10:25:54 mkuserimg_mke2fs.py ERROR: Failed to run e2fsdroid_cmd: __populate_fs: Could not allocate block in ext2 filesystem while writing file "android.hardware.wifi@1.0.so"
e2fsdroid: Could not allocate block in ext2 filesystem while populating file system

mke2fs 1.44.4 (18-Aug-2018)
Creating filesystem with 125355 4k blocks and 3072 inodes
Filesystem UUID: 1d03f927-293c-4e89-b19f-1405b0cdb64d
Superblock backups stored on blocks: 
        32768, 98304

Allocating group tables: done                            
Writing inode tables: done                            
Writing superblocks and filesystem accounting information: done


__populate_fs: Could not allocate block in ext2 filesystem while writing file "android.hardware.wifi@1.0.so"
e2fsdroid: Could not allocate block in ext2 filesystem while populating file system


10:25:55 ninja failed with: exit status 1

#### failed to build some targets (04:15 (mm:ss)) ####

[kheaactua]

I have the same problem as above, except it fails on vendor.img instead of system.img, I've made clean and re-tried several times now to no avail. There's a thread here that suggests increasing the _PARITION_SIZE settings in BoardConfig.mk, but for x86_64 there is no explicitly VENDOR_PARTITION_SIZE. I gave it a try but nothing seemed to help.

My error:

2020-07-13 00:19:22 - build_image.py - ERROR   : Failed to build out/target/product/generic_x86_64/vendor.img from out/target/product/generic_x86_64/vendor
Out of space? Out of inodes? The tree size of out/target/product/generic_x86_64/vendor is 68457472 bytes (65 MB), with reserved space of 0 bytes (0 MB).
The max image size for filesystem files is 85237760 bytes (81 MB), out of a total partition size of 85237760 bytes (81 MB).
Traceback (most recent call last):
  File "build/make/tools/releasetools/build_image.py", line 789, in <module>
    main(sys.argv[1:])
  File "build/make/tools/releasetools/build_image.py", line 781, in main
    BuildImage(in_dir, image_properties, out_file, target_out)
  File "build/make/tools/releasetools/build_image.py", line 423, in BuildImage
    BuildImageMkfs(in_dir, prop_dict, out_file, target_out, fs_config)
  File "build/make/tools/releasetools/build_image.py", line 334, in BuildImageMkfs
    mkfs_output = common.RunAndCheckOutput(build_command)
  File "/src/aosp/build/make/tools/releasetools/common.py", line 252, in RunAndCheckOutput
    args, proc.returncode, output))
common.ExternalError: Failed to run command '['mkuserimg_mke2fs', 'out/target/product/generic_x86_64/vendor', 'out/target/product/generic_x86_64/vendor.img', 'ext4', 'vendor', '85237760', '-j', '0', '-D', 'out/target/product/generic_x86_64/system', '-L', 'vendor', '-i', '541', '-M', '0', '-c', '--inode_size', '256', 'out/target/product/generic_x86_64/obj/ETC/file_contexts.bin_intermediates/file_contexts.bin']' (exit code 4):
00:19:20 mkuserimg_mke2fs.py INFO: Env: {'MKE2FS_CONFIG': './system/extras/ext4_utils/mke2fs.conf'}
00:19:20 mkuserimg_mke2fs.py INFO: Running: mke2fs -O ^has_journal -L vendor -N 541 -I 256 -M /vendor -m 0 -t ext4 -b 4096 out/target/product/generic_x86_64/vendor.img 20810
00:19:20 mkuserimg_mke2fs.py INFO: Env: {}
00:19:20 mkuserimg_mke2fs.py INFO: Running: e2fsdroid -e -p out/target/product/generic_x86_64/system -s -S out/target/product/generic_x86_64/obj/ETC/file_contexts.bin_intermediates/file_contexts.bin -f out/target/product/generic_x86_64/vendor -a /vendor out/target/product/generic_x86_64/vendor.img
00:19:22 mkuserimg_mke2fs.py ERROR: Failed to run e2fsdroid_cmd: __populate_fs: Could not allocate block in ext2 filesystem while writing file "libGLESv1_CM_swiftshader.so"
e2fsdroid: Could not allocate block in ext2 filesystem while populating file system

mke2fs 1.44.4 (18-Aug-2018)
Creating filesystem with 20810 4k blocks and 544 inodes

Allocating group tables: done
Writing inode tables: done
Writing superblocks and filesystem accounting information: done


__populate_fs: Could not allocate block in ext2 filesystem while writing file "libGLESv1_CM_swiftshader.so"
e2fsdroid: Could not allocate block in ext2 filesystem while populating file system

[ypid]

Jup, that also happened to me this month.

ExternalError: Failed to run command '['mkuserimg_mke2fs', '-s', '/home/build/build/base/out/soong/.temp/tmpx7dyzJ', '/home/build/build/base
/out/target/product/sargo/obj/PACKAGING/target_files_intermediates/aosp_sargo-target_files-07072122/IMAGES/system.img', 'ext4', '/', '552378
368', '-j', '0', '-T', '1230768000', '-C', '/home/build/build/base/out/soong/.temp/merged_fs_configj4qel7.txt', '-B', '/home/build/build/bas
e/out/target/product/sargo/obj/PACKAGING/target_files_intermediates/aosp_sargo-target_files-07072122/IMAGES/system.map', '-L', '/', '-i', '3
384', '-M', '0', '-U', '4729639d-b5f2-5cc1-a120-9ac5f788683c', '-S', 'fd77d4fd-bb91-5ef5-9533-d5d806da85a3', '-c', '--inode_size', '256', '/
home/build/build/base/out/target/product/sargo/obj/PACKAGING/target_files_intermediates/aosp_sargo-target_files-07072122/META/file_contexts.
bin']' (exit code 4):
13:16:37 mkuserimg_mke2fs.py INFO: Env: {'E2FSPROGS_FAKE_TIME': '1230768000', 'MKE2FS_CONFIG': './system/extras/ext4_utils/mke2fs.conf'}
13:16:37 mkuserimg_mke2fs.py INFO: Running: mke2fs -O ^has_journal -L / -N 3384 -I 256 -M / -m 0 -U 4729639d-b5f2-5cc1-a120-9ac5f788683c -E
android_sparse,hash_seed=fd77d4fd-bb91-5ef5-9533-d5d806da85a3 -t ext4 -b 4096 /home/build/build/base/out/target/product/sargo/obj/PACKAGING/
target_files_intermediates/aosp_sargo-target_files-07072122/IMAGES/system.img 134858
13:16:38 mkuserimg_mke2fs.py INFO: Env: {'E2FSPROGS_FAKE_TIME': '1230768000'}
13:16:38 mkuserimg_mke2fs.py INFO: Running: e2fsdroid -T 1230768000 -C /home/build/build/base/out/soong/.temp/merged_fs_configj4qel7.txt -B /home/build/build/base/out/target/product/sargo/obj/PACKAGING/target_files_intermediates/aosp_sargo-target_files-07072122/IMAGES/system.map
-s -S /home/build/build/base/out/target/product/sargo/obj/PACKAGING/target_files_intermediates/aosp_sargo-target_files-07072122/META/file_co
ntexts.bin -f /home/build/build/base/out/soong/.temp/tmpx7dyzJ -a / /home/build/build/base/out/target/product/sargo/obj/PACKAGING/target_fil
es_intermediates/aosp_sargo-target_files-07072122/IMAGES/system.img
13:16:42 mkuserimg_mke2fs.py ERROR: Failed to run e2fsdroid_cmd: __populate_fs: Could not allocate block in ext2 filesystem while writing f$
le "libvintf.so"                                                                       
e2fsdroid: Could not allocate block in ext2 filesystem while populating file system

The second build attempt was successful. But I can not see anything being wrong with my first build attempt either. Both were clean builds.

My hope is that those errors go away with #4 (GrapheneOS also lists Debian buster as supported). You might try that.

[kheaactua]

For me, this was a result of attempting to write to a zfs drive. As soon as I pointed to an ext4 drive the build succeeded.

[Manouchehri]

I should also mention that I always have this failure on a NFS drive too (backed by ZFS). Occasionally I've had the issue on a a non-clean build (as mentioned in #9 (comment)) on ext4.