Latency differences between ACT 5.2 and 6.4 versions

[phil12ip]

I am running ACT benchmark on an NVMe SSD, and seeing odd results between different versions of ACT. Specifically, after act_prep has run, and the drive has reached a steady state, running 40x scaling load via ACT version 5.2 vs. version 6.4 produce very different latency results, running each on the same system and disk. Here is a snippet of the latency results between the two versions. I am using the default config for the specific version in each run, changing only the target read-reqs-per-sec and write-reqs-per-sec to match the target load.

6.4 ACT Results @40x

        reads
        %>(ms)
slice        1      2      4      8     16     32     64
-----   ------ ------ ------ ------ ------ ------ ------
    1    70.96  28.44   3.76   0.05   0.00   0.00   0.00
    2    71.34  28.79   3.93   0.06   0.00   0.00   0.00
    3    71.18  28.79   3.87   0.05   0.00   0.00   0.00
    4    71.13  28.71   3.84   0.05   0.00   0.00   0.00
    5    71.25  28.90   3.99   0.08   0.00   0.00   0.00
    6    71.16  28.69   3.88   0.08   0.01   0.00   0.00
    7    71.17  28.63   3.76   0.05   0.00   0.00   0.00
    8    71.27  28.79   3.88   0.05   0.00   0.00   0.00
    9    71.26  28.94   4.01   0.06   0.00   0.00   0.00
   10    71.19  28.80   3.94   0.06   0.00   0.00   0.00
-----   ------ ------ ------ ------ ------ ------ ------
  avg    71.19  28.75   3.89   0.06   0.00   0.00   0.00
  max    71.34  28.94   4.01   0.08   0.01   0.00   0.00

5.2 ACT Results @40x

        reads                                              device-reads
        %>(ms)                                             %>(ms)
slice        1      2      4      8     16     32     64        1      2      4      8     16     32     64
-----   ------ ------ ------ ------ ------ ------ ------   ------ ------ ------ ------ ------ ------ ------
    1     0.33   0.02   0.01   0.00   0.00   0.00   0.00     0.07   0.01   0.01   0.00   0.00   0.00   0.00
    2     0.33   0.02   0.01   0.00   0.00   0.00   0.00     0.08   0.02   0.01   0.00   0.00   0.00   0.00
    3     0.26   0.00   0.00   0.00   0.00   0.00   0.00     0.04   0.00   0.00   0.00   0.00   0.00   0.00
    4     0.28   0.02   0.01   0.00   0.00   0.00   0.00     0.06   0.02   0.01   0.00   0.00   0.00   0.00
    5     0.27   0.01   0.01   0.00   0.00   0.00   0.00     0.05   0.01   0.01   0.00   0.00   0.00   0.00
    6     0.29   0.02   0.01   0.00   0.00   0.00   0.00     0.07   0.02   0.01   0.00   0.00   0.00   0.00
    7     0.28   0.00   0.00   0.00   0.00   0.00   0.00     0.05   0.00   0.00   0.00   0.00   0.00   0.00
    8     0.29   0.01   0.01   0.00   0.00   0.00   0.00     0.07   0.01   0.00   0.00   0.00   0.00   0.00
    9     0.30   0.02   0.01   0.00   0.00   0.00   0.00     0.07   0.02   0.01   0.00   0.00   0.00   0.00
   10     0.27   0.01   0.00   0.00   0.00   0.00   0.00     0.05   0.01   0.00   0.00   0.00   0.00   0.00
-----   ------ ------ ------ ------ ------ ------ ------   ------ ------ ------ ------ ------ ------ ------
  avg     0.29   0.01   0.01   0.00   0.00   0.00   0.00     0.06   0.01   0.01   0.00   0.00   0.00   0.00
  max     0.33   0.02   0.01   0.00   0.00   0.00   0.00     0.08   0.02   0.01   0.00   0.00   0.00   0.00

I have attached the full results output for both the 5.2 and 6.4 run. I am looking for some pointers on what might be the cause for the significant difference in latency being reported between the two test runs on the versions noted.
ACT-5.2-40x-results.txt
ACT-6.4-40x-results.txt

[kportertx]

ACT 5.2 used 4 * <number of cores> threads for the transaction queues by default.. The equivalent tin 6.4 is the service-threads which now uses 5* <number of cores> by default. Could you try manually tuning the number of threads down in 6.4?