Description
Tell us more about this new feature.
When I tried to run the fio command with numjobs option for write operaiton type, it is not allowing. If I remove that then it works fine. Need to know how we can send multiple threads to file write operations to s3-mountpoint and how we can achieve as stated here:
https://awslabs.github.io/mountpoint-s3/dev/bench/ ==>Here the benchmark graphs stated sequential_write-four_threads using the scripts but the seq_write.fio doesn't have any numjobs parameter which means it is single thread only.
https://github.com/awslabs/mountpoint-s3/blob/main/mountpoint-s3/scripts/fs_bench.sh
https://github.com/awslabs/mountpoint-s3/blob/main/mountpoint-s3/scripts/fio/write/seq_write.fio
Error:
[ ec2-user]# fio --name=fs_bench --bs=1M --runtime=120s --time_based --group_reporting --size=100G --rw=write --ioengine=sync --direct=1 --fallocate=none --numjobs=8 --create_on_open=1 --fsync_on_close=1 --unlink=1 --filename=/s3mount/testfile_new
fs_bench: (g=0): rw=write, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=sync, iodepth=1
...
fio-3.32
Starting 8 processes
fio: pid=22450, err=1/file:filesetup.c:805, func=open(/s3mount/testfile_new), error=Operation not permitted
fio: pid=22448, err=1/file:filesetup.c:805, func=open(/s3mount/testfile_new), error=Operation not permitted
fio: pid=22452, err=1/file:filesetup.c:805, func=open(/s3mount/testfile_new), error=Operation not permitted
fio: pid=22446, err=1/file:filesetup.c:805, func=open(/s3mount/testfile_new), error=Operation not permitted
fio: pid=22449, err=1/file:filesetup.c:805, func=open(/s3mount/testfile_new), error=Operation not permitted
fio: pid=22445, err=1/file:filesetup.c:805, func=open(/s3mount/testfile_new), error=Operation not permitted
fio: pid=22447, err=1/file:filesetup.c:805, func=open(/s3mount/testfile_new), error=Operation not permitted
fio: io_u error on file /s3mount/testfile_new: File too large: write offset=83886080000, buflen=1048576
fio: pid=22451, err=27/file:io_u.c:1846, func=io_u error, error=File too large
fs_bench: (groupid=0, jobs=8): err= 1 (file:filesetup.c:805, func=open(/s3mount/testfile_new), error=Operation not permitted): pid=22445: Thu Jun 19 22:42:24 2025
write: IOPS=1702, BW=1703MiB/s (1786MB/s)(78.1GiB/46977msec); 0 zone resets
clat (usec): min=347, max=41880, avg=520.12, stdev=385.59
lat (usec): min=363, max=41984, avg=585.05, stdev=387.90
clat percentiles (usec):
| 1.00th=[ 371], 5.00th=[ 383], 10.00th=[ 400], 20.00th=[ 433],
| 30.00th=[ 457], 40.00th=[ 490], 50.00th=[ 510], 60.00th=[ 529],
| 70.00th=[ 562], 80.00th=[ 586], 90.00th=[ 619], 95.00th=[ 660],
| 99.00th=[ 865], 99.50th=[ 922], 99.90th=[ 1483], 99.95th=[ 2147],
| 99.99th=[13960]
bw ( MiB/s): min= 389, max= 1896, per=100.00%, avg=1709.42, stdev=170.26, samples=93
iops : min= 389, max= 1896, avg=1709.37, stdev=170.28, samples=93
lat (usec) : 500=45.18%, 750=52.92%, 1000=1.59%
lat (msec) : 2=0.25%, 4=0.04%, 10=0.01%, 20=0.01%, 50=0.01%
cpu : usr=11.59%, sys=2.74%, ctx=80049, majf=0, minf=123
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.1%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=0,80001,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=1
Run status group 0 (all jobs):
Need to understand how to send multiple threads to single when writing and does it fall under feature request ?