Skip to content

Raise FileNotFoundError from MemoryFileSystem.rm for a missing path - #2140

Merged
martindurant merged 1 commit into
fsspec:masterfrom
itzzdev09:fix/memory-rm-missing-path
Sep 14, 2026
Merged

Raise FileNotFoundError from MemoryFileSystem.rm for a missing path#2140
martindurant merged 1 commit into
fsspec:masterfrom
itzzdev09:fix/memory-rm-missing-path

Conversation

@itzzdev09

Copy link
Copy Markdown
Contributor

Problem

MemoryFileSystem.rm silently succeeds for a path that does not exist:

import fsspec

fs = fsspec.filesystem("memory")
fs.rm("/missing")                    # no error
fs.rm_file("/missing")               # FileNotFoundError
fs.rm("/missing", recursive=True)    # FileNotFoundError

LocalFileSystem raises in all three cases, as does the base AbstractFileSystem.rm, which calls rm_file on every expanded path.

It is visible through the mapper too. FSMap.__delitem__ turns the filesystem error into KeyError, so on a memory-backed mapper a missing key deletes without error, unlike dict and a file-backed mapper:

m = fsspec.get_mapper("memory://demo")
del m["missing"]                     # no error; a local mapper raises KeyError

Cause

A non-recursive expand_path keeps a literal path whether or not it exists. MemoryFileSystem.rm then skips any expanded path that does not exist:

elif not self.exists(p):
    continue

The comment above that branch explains its purpose: a directory that is not in pseudo_dirs only exists while files sit under it, so deleting those files earlier in the loop makes it vanish. That case is real — rm("/a", recursive=True) after pipe("/a/b/c", ...) depends on it — but the same branch also swallows paths that never existed.

Change

Record which expanded paths exist before deleting anything. A path that no longer exists is still skipped if it existed at the start of the call — the vanishing implicit directory — and otherwise raises FileNotFoundError, like rm_file.

The raise happens at the same point in the reversed loop where AbstractFileSystem.rm would raise, so the ordering matches the other filesystems. Recursive and glob calls are unchanged: expand_path already drops non-existent paths for them.

Tests

In test_memory.py:

  • test_rm_missing_path_raises
  • test_rm_list_with_missing_path_raises
  • test_mapper_delitem_missing_key_raises_keyerror
  • test_rm_recursive_still_removes_implicit_parents — a guard for the case the skip exists for, so this fix cannot regress it

Against master's memory.py, the first three fail and the guard passes. With the change, all four pass.

test_memory.py, test_mapping.py, test_core.py, test_local.py and test_utils.py together: 368 passed before, 372 after, no failures. A wider run found nothing new; every non-passing test also failed on master and came from a dependency I don't have installed (aiohttp, pytest-mock).

ruff 0.14.3 (the pre-commit pin), ruff format and codespell are clean. I added a changelog entry under Dev → Fixes.

🤖 Generated with Claude Code

A non-recursive expand_path keeps a literal path whether or not it
exists, and MemoryFileSystem.rm skipped any expanded path that did not
exist. That skip is meant for directories missing from pseudo_dirs,
which vanish once the files under them are deleted earlier in the same
call, but it also swallowed paths that never existed. So rm("/missing")
succeeded silently, where rm_file, rm(recursive=True) and the other
filesystems raise. Because FSMap.__delitem__ turns that error into
KeyError, `del mapper["missing"]` on a memory-backed mapper did not
raise either.

Record which expanded paths exist before deleting anything, and only
skip a path that existed at the start of the call.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@martindurant
martindurant merged commit 55384a1 into fsspec:master Sep 14, 2026
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants