Open
Description
Problem:
fs.get copies source dir inside target dir but expected to copy content of source dir into target dir.
case 1e of the doc
Script to reproduce error:
import os
from fsspec import filesystem
from fsspec.implementations.dirfs import DirFileSystem
DATA_PATH = "dir"
REMOTE_PATH = "data/remote"
LOCAL_PATH = "data/local"
def init_test():
# remote
os.makedirs(REMOTE_PATH, exist_ok=True)
data_dir = os.path.join(REMOTE_PATH, DATA_PATH)
os.makedirs(data_dir, exist_ok=True)
with open(os.path.join(data_dir, "f.txt"), "w") as f:
f.write("hello world")
# local
os.makedirs(LOCAL_PATH, exist_ok=True)
def test():
fs = filesystem("file")
fs = DirFileSystem(REMOTE_PATH, fs)
# copy from data/remote/dir to data/local/dir
# first leads to data/local/dir/dir/f.txt if called 2 times
fs.get(DATA_PATH, os.path.join(LOCAL_PATH, DATA_PATH), recursive=True)
init_test()
test()
After calling script 2 times I have following dir structure:
data
├── local
│ └── dir
│ └── dir
│ └── f.txt
└── remote
└── dir
└── f.txt
Expected to have following:
data
├── local
│ └── dir
│ └── f.txt
└── remote
└── dir
└── f.txt
Extra info:
fsspec 2025.5.1
Metadata
Metadata
Assignees
Labels
No labels