loading signal_pa from pod5 is very slow for some files

[lpryszcz]

Hi, with some files (typically originating from last few hours of sequencing) I'm experiencing very slow access to signal_pa.

For example, it takes over 1 minute to load signal from a file of 2.3GB with 1.7k reads. I tried this both, with HDD and SSD, the loading time is similar.
Note, for other pod5 files it takes 10-20 seconds to load signal from 100k reads (file size 1.6G)!

reader = pod5.DatasetReader(fn)
for ri, read in enumerate(reader, 1):
    sig = read.signal_pa

I'm using the latest version of pod5 v0.3.15.

I'm interested in the pA signal from the beggining of the read. Is there a way to access it faster?

[0x55555555]

Hi @lpryszcz,

This isnt something we expect, what is the length of reads in the file? What platform is the code running on, and where is the data stored (local or network disk?)

Would you be able to send us a file to investigate?

[lpryszcz]

Hi @0x55555555 , added two files to https://public-docs.crg.es/enovoa/public/lpryszcz/bugs/pod5/171/

• slow: PBE05494_skip_609f0f66_c409c7a5_17.pod5
• normal: PBE05494_skip_609f0f66_c409c7a5_1.pod5

This isnt something we expect, what is the length of reads in the file? What platform is the code running on, and where is the data stored (local or network disk?)

• Haven't looked at read lengths.
• Ubuntu 22.04 x64.
• I tested both, with NFS and local SSD, with similar run times (SSD being a few seconds faster understandably).

[0x55555555]

Thanks - are you able to provide me permission to download files?

[lpryszcz]

ah sorry, fixed

[0x55555555]

Hi @lpryszcz ,

Running this script:

if __name__ == "__main__":

    for file in sys.argv[1:]:
        begin = time.time()
        print(f"Processing {file}...")
        reader = pod5.DatasetReader(file)
        read_count = 0
        for ri, read in enumerate(reader, 1):
            sig = read.signal_pa
            read_count += 1

        end = time.time()
        print(f"Processed {read_count} reads in {end - begin:.2f} seconds, for {read_count / (end - begin):.2f} reads/sec")

For me, it takes approximately the same time to process both files - ~8s (but this will depend heavily on decompress performance and disk speed).

I would note the 1 (fast) file has ~100k reads, while the 17 file has ~1000 reads - so the slower file reads are significantly longer.

I don't see it take significantly longer to process the 17 file, as you report... it could be related to decompress performance - what CPU/memory are you running on? If you view the cpu usage of your machine when running is it maxing out a core, or several cores?

[lpryszcz]

thank you for checking it!

I don't see it take significantly longer to process the 17 file, as you report... it could be related to decompress performance - what CPU/memory are you running on? If you view the cpu usage of your machine when running is it maxing out a core, or several cores?

No, I don't see high CPU usage, I'm reading data from NFS, I suspect I/O may be a bottleneck for very long reads.