zfs trimming a seagate HDD

[mabod]

This is on endeavouros with latest zfs 2.3.4.

I have a RAIDZ2 pool consisting of 5 HDD. Today I realized that two of the HDD (Seagate ST4000VN006-3CW104) show that they can be trimmed. And I can successfully trim them. The trim takes 1-2 minutes while showing the progress in percent and finishes successful. What does that mean? How can that be?

This is a pool on LUKS. The LUKS open command is with --allow-discards --perf-no_read_workqueue --perf-no_write_workqueue

If I remove the --allow-discards I can not execute the trim but it still shows the trim status for these two Seagate drives: (100% trimmed, completed at Thu Sep 11 08:07:02 2025)

The other two Seagate drives ST4000VN008-2DR166 are correctly shown as (trim unsupported)

Status before trim:

# zpool status -t zsea
  pool: zsea
 state: ONLINE
  scan: scrub repaired 0B in 10:23:29 with 0 errors on Tue Jun 24 23:41:46 2025
config:

        NAME                               STATE     READ WRITE CKSUM
        zsea                               ONLINE       0     0     0
          raidz2-0                         ONLINE       0     0     0
            ST4000VN008-2DR166-ZDHB956C    ONLINE       0     0     0  (trim unsupported)
            ST4000VN008-2DR166-ZDHB95H0    ONLINE       0     0     0  (trim unsupported)
            WD40EZRZ-00GXCB0-WCC7K7EK9VC4  ONLINE       0     0     0  (trim unsupported)
            ST4000VN006-3CW104-ZW62YK8E    ONLINE       0     0     0  (untrimmed)
            ST4000VN006-3CW104-ZW62YHYW    ONLINE       0     0     0  (untrimmed)

Status after trim:

# zpool status -t zsea
  pool: zsea
 state: ONLINE
  scan: scrub repaired 0B in 10:23:29 with 0 errors on Tue Jun 24 23:41:46 2025
config:

        NAME                               STATE     READ WRITE CKSUM
        zsea                               ONLINE       0     0     0
          raidz2-0                         ONLINE       0     0     0
            ST4000VN008-2DR166-ZDHB956C    ONLINE       0     0     0  (trim unsupported)
            ST4000VN008-2DR166-ZDHB95H0    ONLINE       0     0     0  (trim unsupported)
            WD40EZRZ-00GXCB0-WCC7K7EK9VC4  ONLINE       0     0     0  (trim unsupported)
            ST4000VN006-3CW104-ZW62YK8E    ONLINE       0     0     0  (100% trimmed, completed at Thu Sep 11 08:07:02 2025)
            ST4000VN006-3CW104-ZW62YHYW    ONLINE       0     0     0  (100% trimmed, completed at Thu Sep 11 08:07:02 2025)

[bknl]

What is the defect here ? TRIM is a performance advisory to a disk (be it an SSD or a SMR drive or whatever can benefit from the additional information). When a physical drive in a multidrive VDEV says "don't bother telling me, not interested" why would it be a defect to not tell those drives in the setup that say "yes please, inform me of what is unused" ? I have similar multidrive setups (albeit just mirrors) with a mix of Seagate (no trim) and Western Digital (trim) drives and it just works that way.

[mabod]

In only read about trimming in the context of SSD and NVME drives, where it makes sense to prevent performance degradation. Never heard about it for rotational discs. Its the first time I see this. I guess I learned something new. Although I do not understand what the trim command is doing on a CMR drive.

[amotin]

TRIM definitely makes sense for SMR HDDs to reduce their random rewrite performance penalties. I do have some of those TRIM-capable SMR drives somewhere in my lab. Though if the drives in question are really an CMR, then it is surprising. It would be interesting to see what drive itself tells about this capability when asked directly. I may equally believe in either of firmware, kernel or ZFS bugs.

[mabod]

The drives in question are ST4000VN006-3CW104. Their siblings, ST4000VN008-2DR166, do not show a trim support with zfs.

According to the seagate datasheets all seagate ironwolf are CMR drives. In particular ST4000VN006 and ST4000VN008.

https://www.seagate.com/content/dam/seagate/migrated-assets/www-content/datasheets/pdfs/ironwolf-18tb-DS1904-21-2207US-en_US.pdf

When I query the drives with hdparm or lsblk I get confusing results.

lsblk --discard is supposed to show trim support if the values DISC-GRAN and DISC-MAX are not null. This is only true for one of the ST4000VN006 drives.

# lsblk --discard /dev/sdf /dev/sdg /dev/sdi /dev/sdj
NAME                            DISC-ALN DISC-GRAN DISC-MAX DISC-ZERO
sdf                                    0        4K       4G         0
├─sdf1                                 0        4K       4G         0
│ └─ST4000VN006-3CW104-ZW62YK8E        0        4K       4G         0
└─sdf2                                 0        4K       4G         0
sdg                                    0        0B       0B         0
├─sdg1                                 0        0B       0B         0
│ └─ST4000VN006-3CW104-ZW62YHYW        0        0B       0B         0
└─sdg2                                 0        0B       0B         0
sdi                                    0        0B       0B         0
├─sdi1                                 0        0B       0B         0
│ └─ST4000VN008-2DR166-ZDHB956C        0        0B       0B         0
└─sdi9                                 0        0B       0B         0
sdj                                    0        0B       0B         0
├─sdj1                                 0        0B       0B         0
│ └─ST4000VN008-2DR166-ZDHB95H0        0        0B       0B         0
└─sdj9                                 0        0B       0B         0

hdparm is showing no trace of trim support

# hdparm -I /dev/sdf

/dev/sdf:

ATA device, with non-removable media
        Model Number:       ST4000VN006-3CW104
        Serial Number:      ZW62YK8E
        Firmware Revision:  SC60
        Transport:          Serial, ATA8-AST, SATA 1.0a, SATA II Extensions, SATA Rev 2.5, SATA Rev 2.6, SATA Rev 3.0
Standards:
        Used: unknown (minor revision code 0x006d)
        Supported: 10 9 8 7 6 5
        Likely used: 10
Configuration:
        Logical         max     current
        cylinders       16383   16383
        heads           16      16
        sectors/track   63      63
        --
        CHS current addressable sectors:    16514064
        LBA    user addressable sectors:   268435455
        LBA48  user addressable sectors:  7814037168
        Logical  Sector size:                   512 bytes
        Physical Sector size:                  4096 bytes
        Logical Sector-0 offset:                  0 bytes
        device size with M = 1024*1024:     3815447 MBytes
        device size with M = 1000*1000:     4000787 MBytes (4000 GB)
        cache/buffer size  = unknown
        Form Factor: 3.5 inch
        Nominal Media Rotation Rate: 5400
Capabilities:
        LBA, IORDY(can be disabled)
        Queue depth: 32
        Standby timer values: spec'd by Standard, no device specific minimum
        R/W multiple sector transfer: Max = 16  Current = 16
        Recommended acoustic management value: 128, current value: 128
        DMA: mdma0 mdma1 mdma2 udma0 udma1 udma2 udma3 udma4 udma5 *udma6
             Cycle time: min=120ns recommended=120ns
        PIO: pio0 pio1 pio2 pio3 pio4
             Cycle time: no flow control=120ns  IORDY flow control=120ns
Commands/features:
        Enabled Supported:
           *    SMART feature set
                Security Mode feature set
           *    Power Management feature set
           *    Write cache
           *    Look-ahead
           *    Host Protected Area feature set
           *    WRITE_BUFFER command
           *    READ_BUFFER command
           *    DOWNLOAD_MICROCODE
                Power-Up In Standby feature set
           *    SET_FEATURES required to spinup after power up
                SET_MAX security extension
           *    48-bit Address feature set
           *    Device Configuration Overlay feature set
           *    Mandatory FLUSH_CACHE
           *    FLUSH_CACHE_EXT
           *    SMART error logging
           *    SMART self-test
           *    Media Card Pass-Through
           *    General Purpose Logging feature set
           *    64-bit World wide name
           *    WRITE_UNCORRECTABLE_EXT command
           *    {READ,WRITE}_DMA_EXT_GPL commands
           *    Segmented DOWNLOAD_MICROCODE
                unknown 119[6]
           *    unknown 119[7]
           *    Gen1 signaling speed (1.5Gb/s)
           *    Gen2 signaling speed (3.0Gb/s)
           *    Gen3 signaling speed (6.0Gb/s)
           *    Native Command Queueing (NCQ)
           *    Phy event counters
           *    READ_LOG_DMA_EXT equivalent to READ_LOG_EXT
                DMA Setup Auto-Activate optimization
                Device-initiated interface power management
           *    Software settings preservation
                unknown 78[7]
           *    SMART Command Transport (SCT) feature set
           *    SCT Write Same (AC2)
           *    SCT Error Recovery Control (AC3)
           *    SCT Features Control (AC4)
           *    SCT Data Tables (AC5)
                unknown 206[7]
                unknown 206[12] (vendor specific)
                unknown 206[13] (vendor specific)
                unknown 206[14] (vendor specific)
           *    DOWNLOAD MICROCODE DMA command
Security:
        Master password revision code = 65534
                supported
        not     enabled
        not     locked
        not     frozen
        not     expired: security count
                supported: enhanced erase
        468min for SECURITY ERASE UNIT. 468min for ENHANCED SECURITY ERASE UNIT.
Logical Unit WWN Device Identifier: 5000c500e94aca6d
        NAA             : 5
        IEEE OUI        : 000c50
        Unique ID       : 0e94aca6d
Checksum: correct

[bknl]

I have no personal experience with your specific drives, but according to this there seems to be something fishy about their "CMRness".

[mabod]

I have never experienced slowness of these drives during daily operations or scrub. A scrub takes about 10 hours. This is a reasonable time for a 5 discs RAIDZ2 with 6 TB of data.

And the read / write performance is excellent from my point of view. fio telling me:

   READ: bw=377MiB/s (395MB/s), 377MiB/s-377MiB/s (395MB/s-395MB/s), io=64.0GiB (68.7GB), run=173976-173976msec
  WRITE: bw=269MiB/s (282MB/s), 269MiB/s-269MiB/s (282MB/s-282MB/s), io=47.3GiB (50.8GB), run=180025-180025msec

I would not take too much notice of this one report in your link.

[jxdking]

I have never experienced slowness of these drives during daily operations or scrub.

Scrub is read heavy operation. SMR is fine for reading. Also, it may have hundreds GB CMR region for caching. You don't really notice difference during the benchmark. It really becomes issue when you try to resilver a SMR drive without the sequential flag.

[mabod]

I am not sure in which direction this is going here. Official Seagate technical specs says these are CMR drives. Are you suggesting Seagate is lying with the technical specs?

[amotin]

Are you suggesting Seagate is lying with the technical specs?

WD did at some point. Or rather omitted it. But I don't insist on analogy. But since you see DISC-GRAN and DISC-MAX reported by OS, then it is likely outside of ZFS responsibility.

[mabod]

But since you see DISC-GRAN and DISC-MAX reported by OS, then it is likely outside of ZFS responsibility.

Only one of the two ST4000VN006 shows that.

[amotin]

But since you see DISC-GRAN and DISC-MAX reported by OS, then it is likely outside of ZFS responsibility.

Only one of the two ST4000VN006 shows that.

Until we know why it is reported for one, we can not trust others. It could have changed since pool import.

[mabod]

I will contact the Seagate support and see what they say.

In the meantime I found this page which says that the ST4000VN006-3CW104 has no trim support. Weird.

https://smarthdd.com/database/ST4000VN006-3CW104/SC60/

[mabod]

Another weird thing. When I trim the pool today, only one of the devices gets trimmed by zfs. The second devices stays on status from yesterday:

# zpool status -t zsea
  pool: zsea
 state: ONLINE
  scan: scrub repaired 0B in 10:23:29 with 0 errors on Tue Jun 24 23:41:46 2025
config:

        NAME                               STATE     READ WRITE CKSUM
        zsea                               ONLINE       0     0     0
          raidz2-0                         ONLINE       0     0     0
            ST4000VN008-2DR166-ZDHB956C    ONLINE       0     0     0  (trim unsupported)
            ST4000VN008-2DR166-ZDHB95H0    ONLINE       0     0     0  (trim unsupported)
            WD40EZRZ-00GXCB0-WCC7K7EK9VC4  ONLINE       0     0     0  (trim unsupported)
            ST4000VN006-3CW104-ZW62YK8E    ONLINE       0     0     0  (100% trimmed, completed at Thu Sep 11 08:07:02 2025)
            ST4000VN006-3CW104-ZW62YHYW    ONLINE       0     0     0  (100% trimmed, completed at Fri Sep 12 10:14:53 2025)

errors: No known data errors

How can that be? I thought all devices are trimmed at the same time? Or is there a way that I can trim only a specific device in a pool?

[Momi-V]

How can that be? I thought all devices are trimmed at the same time? Or is there a way that I can trim only a specific device in a pool?

You showed yourself that one of the drives did and one didn't announce trim support (any more?)

That is indicative of the drives reporting trim support at pool import time, responding to trim requests successfully and one of them then just stopping it's advertisement of trim support (leaving all attempts after that time unsuccessful).

Maybe you can get it back with a full export + shutdown + PSU disconnect + boot + import to try "restarting" the HDD firmware?