File locked/in use issue

[stl3]

I'm having somewhat of a unique issue that I am unable to figure out on my own so I was hoping there's an answer to it. For a bit of background, I am using this project specifically only for playing videos, and discarding other files I consider useless (or at least have priority set to none). This worked fine for the most part up until v1.58.1 where I am able to delete the files that have None as priority:

	for _, file := range t.Files() {
		lowerPath := strings.ToLower(file.Path())

		switch {
		case strings.HasSuffix(lowerPath, ".srt"),
			strings.HasSuffix(lowerPath, ".sub"),
			strings.HasSuffix(lowerPath, ".ass"),
			strings.HasSuffix(lowerPath, ".vtt"):
			file.SetPriority(torrent.PiecePriorityHigh) // Subtitles first
		case file == largestFile:
			file.SetPriority(torrent.PiecePriorityNormal) // Main video normal
		default:
			file.SetPriority(torrent.PiecePriorityNone) // Everything else not important
		}
	}

When I upgraded to v1.60.0, I've had to set the following in order for the files to keep original names while downloading and not have .part as extension:

"github.com/anacrolix/torrent/storage"
"github.com/anacrolix/torrent/generics"
	clientConfig.DefaultStorage = storage.NewFileOpts(storage.NewFileClientOpts{
		ClientBaseDir: dataDir,
		UsePartFiles:  generics.Some(false),
	})

Now with those above, when I'm on v1.60.0 files that are set as file.SetPriority(torrent.PiecePriorityNone) cannot be deleted as they are locked/in use. How can I make it so it works like back on v1.58.1 where the None priority files can be deleted so when their files are recreated they are 0bytes like this:

What am I doing wrong or missing?

Note: Yes I am aware there are cross "pollinated" boundary pieces shared - I was aware of this when I had been doing it on previous versions, but now in v1.60.0 all files in torrent are locked/in use no matter what unlike in versions below v1.58.1

[anacrolix]

Interesting. I think it's because the default file storage implementation now uses a mix of mmap and persistent file handles to improve performance. Does this match your usage? Are you on Windows? If so I think there's 2 ways to handle this: 1) Expose a method to flush/close persistent handles for the file so you can delete them, or use a storage/wide flush method to close out those handles (this currently somewhat exists for closing storage when the Client/Torrent are closed). However it's worth noting that deleting files from under the implementation even in your old case wasn't ideal: If the torrent attempts to seed or access that data and discovers it is missing, it's not ideal. I think it handles it correctly for most cases. So a 3rd case exists which is to add a delete method for files to inform the storage that the data isn't required anymore and can be deleted. That's a lot more work.

What do you think?

[stl3]

I tried a few of the suggestions but they failed (possibly through my implementation):
Track all FileEntry readers and explicitly close them before deletion

type Client struct {
    activeReaders map[string]io.Closer
    readersMutex  sync.RWMutex
}

func (client *Client) trackReader(filePath string, reader io.Closer) {
    client.readersMutex.Lock()
    defer client.readersMutex.Unlock()
    client.activeReaders[filePath] = reader
}

func (client *Client) CloseAllReaders() error {
    for filePath, reader := range client.activeReaders {
        reader.Close()
        delete(client.activeReaders, filePath)
    }
}

Even with all tracked readers closed, files remain locked

Then I tried to flush storage handles before deletion

func (client *Client) FlushStorage() error {
    // Try direct storage flush interface
    if storage, ok := client.ClientConfig.DefaultStorage.(interface{ Flush() error }); ok {
        return storage.Flush()
    }
    
    // Force garbage collection
    for i := 0; i < 3; i++ {
        runtime.GC()
        time.Sleep(200 * time.Millisecond)
    }
    return nil
}

No flush interface available, garbage collection doesn't release handles

Then I tried to Drop torrent and Close client before deletion attempts

func (client *Client) PrepareForDeletion() error {
    client.CloseAllReaders()
    
    if client.Torrent != nil {
        client.Torrent.Drop()
    }
    
    if client.Client != nil {
        client.Client.Close()
    }
    
    time.Sleep(3 * time.Second)
    runtime.GC()
    return nil
}

Files still locked even after client closure

I also did try temporarily shutting *Client, then delete files and then recreate the client but I am still unable to release and delete the files until my program fully exits. I'm kind of out of ideas at this point lol. Am I completely out of luck with this then, or is there something else I can still do?

[anacrolix]

Sorry, did you try on main? The Latest release significantly lags and a lot of things were tightened up on main in this regard.

Your testing looks very thorough. I believe only the Close/Drop should actually have an affect with the code in the current form, so if main works when you Close/Drop the next step would be to add something to flush handles or release/delete files so you don't need to Close/Drop to achieve that, and don't race against the client trying to reopen them again.

[anacrolix]

I just released v1.61.0. However the changes to closing handles are only on main, so I think you need to try that.

[stl3]

Good news!

go: upgraded github.com/anacrolix/torrent v1.58.1 => v1.61.1-0.20260113032004-afa3e1129e5f

On my old code with the old deletion method:

INFO Download progress reached 5.19%, starting deletion of unselected files 
[i] Stopping processes...[ 6.4 GB ] 8.85%  Download Speed: 1.1 MB/s

[i] Exiting...

[i] Deleting downloads...T:\stream\Example - Season 3 (2020) [1080p]

Success for deletions! (I know it didn't really have much feedback in the text, but I was looking at file explorer and the files did indeed get deleted).

On my newer code that handles deletions more fancy (messy):

INFO Download progress reached 5.15%, starting deletion of unselected files 
INFO Starting deletion of unselected files, preserving: Example - S03E05 - Example.mkv 
DEBU Disabling download for unselected files...
INFO Found 7 files to delete, 1 actually exist on disk ed: 1.1 MB/s
DEBU Disabling download for unselected files...
INFO Found 7 files to delete, 1 actually exist on disk ed: 1.1 MB/s
DEBU Disabling download for unselected files...
INFO Found 7 files to delete, 1 actually exist on disk ed: 1.1 MB/s
INFO Using radical deletion approach (client recreation)
DEBU Disabling download for unselected files...
INFO Found 7 files to delete, 1 actually exist on disk ed: 1.1 MB/s
INFO Using radical deletion approach (client recreation)
DEBU Disabling download for unselected files...
INFO Found 7 files to delete, 1 actually exist on disk ed: 1.1 MB/s
INFO Using radical deletion approach (client recreation)
DEBU Disabling download for unselected files...
INFO Found 7 files to delete, 1 actually exist on disk ed: 1.1 MB/s
DEBU Disabling download for unselected files...
DEBU Disabling download for unselected files...
DEBU Disabling download for unselected files...
INFO Found 7 files to delete, 1 actually exist on disk ed: 1.1 MB/s
INFO Using radical deletion approach (client recreation)
INFO Attempting file deletion with temporary client recreation...
DEBU Saving state: progress=5.54%, magnet=Unknown
DEBU Temporarily closing torrent client to release file handles...
INFO Successfully deleted file: T:\stream\Example - Season 3 (2020) [1080p]\Example - S03E07 - Example.mkv
INFO Client recreation deletion: 1 deleted, 0 failed
INFO Recreating torrent client and resuming streaming...
INFO Client recreated and streaming resumed successfully
[i] Stopping processes... [ 6.4 GB ] 61.71%  Download Speed: 1.1 MB/s

[i] Exiting...
[i] Starting enhanced cleanup process...  
[i] Performing graceful client shutdown...
INFO Stopping client and cleaning up resources... 
DEBU Attempting to flush storage handles...       
DEBU Flushing torrent pieces...
DEBU Storage flush completed
INFO Client stopped successfully
[i] Waiting for file handles to be released...
[i] Deleting downloads...T:\stream\Example - Season 3 (2020) [1080p]
[i] Successfully deleted directory:T:\stream\Example - Season 3 (2020) [1080p]
[i] Cleanup completed

Also I wasn't sure which one you were referring to as main, but when I was using master it wasn't working but go get github.com/anacrolix/torrent@anacrolix was the one that worked, I did look at the last commit times (looked for mmapFileIo related ones) so wasn't sure but in any case it worked (so I hope that is the one you meant by main).

Now all I have to do is some code cleanup 👍