(Clickbaity?) talk claims Haskell implementation beats fd

[gendx]

I didn't dig into the details, but just noticed a talk published a few days ago with a clickbait title "Blazing-Fast Directory Tree Traversal: Haskell Beats Rust". Looking at the slides, they used fd v8.7.1 as representative code for Rust, so I thought you should know.

I have no idea if the benchmarks were fair or not, nor if there are potential performance improvements on the table.

• Conference link: https://confengine.com/conferences/functional-conf-2025/proposal/21325/blazing-fast-directory-tree-traversal-haskell-streamly-beats-rust
• Slides: https://github.com/composewell/fnconf-2025-directory-traversal/releases/download/fnconf-2025/directory-traversal.pdf
• Video: https://www.youtube.com/watch?v=voy1iT2E4bk
• Code: https://github.com/composewell/streamly-examples/blob/master/examples/ListDir.hs

[tavianator]

they used fd v8.7.1 as representative code for Rust

Well it would have been nice of them to measure fd after I made it 6-13× faster lol.

I haven't looked at the talk yet but it would be interesting to redo the benchmarks with newer fd.

[tmccombs]

I haven't looked very deeply into this, but some thoughts I have:

• it looks like they are just using time to measure the performance, which isn't sufficient for multiple reasons.
  • it only runs the program once, so doesn't account for noise in the run
  • it doesn't account for the OS disk cache. So whichever program is run first will probably be much slower
• It's comparing the full fd program to something that just recursively lists all files in the directory. There is going to be some overhead to the additional functionality fd has

[tavianator]

• it doesn't account for the OS disk cache. So whichever program is run first will probably be much slower

The "Comparative Performance" slide does mention they are taking this into account.