Skip to content

linux: the read-deny glob walk has a budget - #607

Open
ronleizrowice-ant wants to merge 20 commits into
mainfrom
fix/linux-read-deny-glob-bound
Open

ronleizrowice-ant wants to merge 20 commits into
mainfrom
fix/linux-read-deny-glob-bound

Conversation

@ronleizrowice-ant

@ronleizrowice-ant ronleizrowice-ant commented Sep 25, 2026 •

Copy link
Copy Markdown
Collaborator

Human: @ronleizrowice-ant

Requested by Ron · project thread

What it does

On Linux the globs of one read configuration (allowRead, denyRead, the globs among credentials.files) are expanded on one shared budget: directory entries read from disk, and wall time. When it runs out, wrapWithSandbox(), wrapWithSandboxArgv() and getFsReadConfig() fail with LinuxSandboxProfileError, code deny_glob_too_large, and a GlobWalkBudgetError on .cause (type exported). Nothing shortened is returned.

Size

+585 −18 in 8 files: 175 lines of source, 407 of tests, 3 of README. It was +1,485 −41 before the simplification pass.

Defaults and option

20,000,000 entries and 60,000 ms. filesystem.denyReadGlobBudget: { maxEntries?, timeoutMs? } (positive integers, no other key) sets others, from initialize(), updateConfig() or one wrap's customConfig, which replaces only the limit it names. The values are checked against the schema where the budget is made, since only the CLI runs the schema.

Who sees a difference

Only a configuration whose patterns read past the budget: it used to wait, now it is refused.

What changed on 2026-10-20, after a review of the whole diff against main

  • Only an entry read from disk is counted. Before, an entry was also charged when it came from a listing another pattern had made, so the tree that fits shrank with every pattern written over it (ten ** patterns over 2.5 million entries were refused). Going through a listing again now costs time alone. This reverses a choice the earlier integration suite pinned ("charges each pattern for it").
  • The allowRead walk is on the same budget. It had none.
  • The clock is also read before an untyped entry's type is asked for, and a spent budget is no longer taken for a directory that could not be listed.
  • A limit that is no positive integer, a key that is neither of the two, or a budget that is no object fails the wrap on every route. A limit that a wrap's own configuration holds as undefined leaves the session's in place.
  • A budget that runs out while the types of a listing are asked for is no longer taken for a real path that could not be read (found by a second review, of the simplified diff).
  • The counters directoriesListed and entriesExamined, and the longer SRT_DEBUG line that was their only reader, are gone. The debug line is main's again.
  • The case "a signal wins over a spent budget" is gone: it pinned an order nothing depends on.
  • The two suites are one, over one planted tree and a clock that moves 1 ms a reading. 36 mutations of the source were tried against it and each fails at least one case.

Known limits

initialize() and updateConfig() accept an over-budget configuration, and nothing is kept between wraps: every refused wrap spends the budget again. One blocking filesystem call or one slow match is not cut short. All patterns share one deadline, and turns the wrap gives to other work count against it: that can refuse a wrap earlier and never shortens a list.

Merge order

Conflicts with #693 in read-deny-glob.ts and sandbox-manager.ts (one options object, { anchor, budget, listings } here and { anchor, listings, matchedLinks } there): whichever lands second takes both. No conflict with #573 or #594 as they stand.

main merged in on 2026-11-06

main at 27fce0d is merged in at 369c275. One conflict, in the ISandboxManager interface: main gave checkDependencies() a second parameter on the lines just above the comment this pull request adds to getFsReadConfig(). Both are kept. What this pull request changes is the same as before, +585 −18 in 8 files.

main merged in again on 2026-11-09

main at 951e641 is merged in at dbd699b, without a conflict. What this pull request changes is the same as before.

Claude session #01FydxqU


Generated by Claude Code

bubblewrap has no patterns, so a denyRead entry with glob syntax is
expanded on the host into the paths to mount over, on every wrap. That
walk listed through any symbolic link to a directory, wherever it led;
the only link it declined was one leading back up the tree. With a link
from a project to /usr and `**/.env` denied, each expansion listed
57,000 entries: 260 to 320 ms per command warm, seconds cold, and no
limit for a larger target. A sandboxed command can plant such a link.

A link whose target lies outside the pattern's tree is no longer listed
through. The tree is the pattern's literal starting directory and what
is under it, and a link is judged by where it resolves. Links that stay
inside are followed as before, and a link whose own name matches still
masks what it leads to. What this gives up is a file the pattern matches
only by a name that passes through such a link: macOS never covered it,
since patterns are matched against resolved paths there.

Where a matched link hides a directory outside the tree that the pattern
would have carried on into, nothing is bound back beneath it: it is
reported through unlistableDenyDirs like a directory the walk cannot
list. Unlisted, an allowed path beneath it would otherwise come back
with nothing masked.

The walk takes a budget, shared by all the denyRead patterns of one
configuration: 2,000,000 directory entries and 10 seconds, the clock
read before each listing and at each entry. Out of budget it throws and
hands nothing back, and the wrap is refused with LinuxSandboxProfileError
`deny_glob_too_large`, the walk's error as its cause. getFsReadConfig()
throws the same. A deny list cut short would leave readable what the
pattern was written to hide.

Each expansion logs one line under SRT_DEBUG: milliseconds, matches,
mounts, directories listed, entries looked at, links left unfollowed.
Ten existing cases pinned the listing through a link out of the tree.
Five now expect the rule. Five are about something else (a chain of
links, a path too long to name, `**` written against text, a bracket
expression, a carve-out): they keep their expectations under a base
widened to hold the link, and assert the original pattern under the new
rule beside it. None is removed.

New cases pin the rule from both sides: a link out is never listed, by
a count of listings, and a link inside the tree that is the only route
to a match is still followed. A link is judged by where it resolves; the
tree is the pattern's and not the project's; a matched link still masks
its target. Nothing is bound back beneath an unlisted target, shown by a
real bubblewrap run in which the file beneath the allowed path is
unreadable under both names. The budget: the exact entry boundary, the
deadline before and during the walk and inside a small directory, one
budget shared by the patterns of a wrap, never a partial result. At the
manager the wrap rejects and returns no command, the getter throws the
same, and the fields of the cause are what a caller words its own
message from.
@ronleizrowice-ant ronleizrowice-ant mentioned this pull request Sep 25, 2026
…llowed

A link out of a denyRead pattern's tree is not listed through, and what
the pattern would have matched beneath it is not denied. That was
recorded on the walk and written to the debug log; a caller had no way
to tell its user. getFsReadConfig() now lists each such link in
`unfollowedDenyLinks`, with the pattern that came to it, the link and
where it resolves. It informs and restricts nothing: no backend reads
it. The type is exported as UnfollowedDenyLink.
It lists the links left unfollowed for leading out of a pattern's tree.
A link that leads back up the tree, and any link under a pattern that
cannot be followed one path component at a time, is not listed through
either and is not in the list. The field's comment and the README read
as if the list were of every link not followed.
Where a link whose own name matches hides a directory outside the
pattern's tree, the expansion reported that directory as one it could
not list, so that nothing was bound back beneath it. The wrapper then
restores no allowed path at or beneath it, the paths that are always
writable included: with a link to a directory above the command's
temporary directory, every command failed, and an allowed write path
beneath such a directory stopped applying. Through the ancestor that
hides it, a second link could take the working directory along.

The report is taken out again. A directory outside the tree is never
listed, whether or not the link that leads there matches and whatever
is allowed beneath it; a matched link still hides its target whole; an
allowed path beneath it is bound back as written, as beneath a directory
denied literally. `unlistableDenyDirs` again names only a directory the
walk could not list, and the code that fills it is what it was.

What that gives up is what such a link used to add: what the pattern
matched beneath the allowed path through the link's name is not masked
again. A link out of the tree can only add to what a pattern covers. A
file whose real path is in the tree and matches is found by walking the
tree itself, so no link a sandboxed command creates takes a mask away.

The unfollowed links are handed back sorted by name, so that the list
and the debug line do not depend on the order a directory is read in.
The cases written for the report of an unlisted directory now state the
rule as it is: what a matched link out of the tree leads to is hidden
whole and not listed, with nothing allowed beneath it and with an
allowed path at it, beneath it or written through the link; nothing is
handed back as unlistable; a link inside such a directory is never come
to. Two real bubblewrap runs pin what follows from it: beneath such a
directory an allowed path is readable and the rest stays hidden, under
either name, and a write allowed there succeeds. The unfollowed links
come back in the same order whichever way the directory is read.
The walk stopped at a symbolic link that leaves the pattern's tree.
Every release follows such a link and masks what it finds behind it:
with `project/out` a link to `../outside` and `project/**/.env` denied,
`outside/deep/.env` is unreadable under both names on 0.0.75, 0.0.76 and
0.0.77. Stopping there left it readable, so the rule lowered what a
pattern covers and is taken out. The messages of the earlier commits on
this branch that describe it as what other releases and platforms do
were wrong on that point.

Link handling in walkGlobPattern is what it was. `GlobWalk.unfollowedLinks`,
the out-parameter of expandReadDenyGlobLinux, and `unfollowedDenyLinks`
with its exported type are removed.

What stays is what the walk never had: a budget shared by all the
denyRead patterns of one configuration, 2,000,000 directory entries and
10 seconds, the clock read before each listing and at each entry. Out of
budget the walk throws and hands nothing back, the wrap is refused with
LinuxSandboxProfileError `deny_glob_too_large`, and getFsReadConfig()
throws the same. What lies behind a link counts against it like anything
else. Each expansion logs one line under SRT_DEBUG with what it cost.
…budget

The two existing glob suites are as they were before this branch. The
new file keeps the cases for the budget, for the refusal at the manager
and for the debug line, and loses the ones written for the rule. Added:
a match behind a link to a directory beside the project is denied where
it really is, and a real bubblewrap run shows it unreadable under both
names; what such a link leads to is spent from the budget, at the walk
and at the manager.
@ronleizrowice-ant ronleizrowice-ant changed the title linux: a read-deny glob stays in its own tree, and its walk has a budget linux: the read-deny glob walk has a budget Sep 26, 2026
Comments only: the JavaScript emitted with comments removed is
byte-identical for every file, and the test names are unchanged.

Kept: every invariant and what could be done without it, the
documentation of exported names, and the reasons for what is not done
the obvious way. Cut: how the code came to be as it is, narration of
the lines that follow, and explanations repeated at several sites.
…to change it

The default of 2,000,000 entries and 10 seconds refused every command
in a large monorepo: one `**/.env` pattern over 2.5 million entries
walks in about 7 seconds. Such a user waited before; refused, they
could not run anything, and nothing could raise the limit.

The defaults are now 20,000,000 entries and 60 seconds, and
`filesystem.denyReadGlobBudget` (`maxEntries`, `timeoutMs`) sets
others, from the initialized configuration, updateConfig() or one
wrap's customConfig. The refusal names the option.

README: wrapWithSandboxArgv() rejects the same way; the patterns share
one deadline; an over-budget configuration is accepted at initialize()
and every wrap of it spends the budget again.
A misspelt `maxEntires` was accepted and ignored, and the default
applied in silence.
ronleizrowice-ant added a commit that referenced this pull request Sep 30, 2026
Both it and the walk's budget (#607) hand every pattern of one read
configuration something to share, so the listings go where the budget already
is: made in readDenyGlobExpander(), and handed on in the options of
expandReadDenyGlobLinux(). The walk's record of a directory gives up its own
copy of the entries for the shared map.

An entry is paid for before it is skipped: the budget bounds the entries
looked at, and one the fast path passes over was looked at too.
ronleizrowice-ant added a commit that referenced this pull request Sep 30, 2026
The walk's budget (#607), its shared listings (#627) and its steps (#628)
rewrite one code path, and the path entries of #620 to #622 sit on top of it.

Everything above the walk is a generator now: the read-deny expansion, the
function readDenyGlobExpander() returns, the allowRead expansion, the
resolution of read path entries, and withOtherReadings(), which calls the
expander once for each reading of an entry. The synchronous names finish
those on the spot.

In the walk the step comes first and the budget's clock is read after it,
before the next listing. The deadline is wall time, so turns given to other
work count against it: a wrap can be refused earlier for that, and a list is
never cut short.

The budget error is given its code around the delegated generator. An abort
is raised by the driver and does not pass through there, so a wrap with a
spent budget and an aborted signal rejects with the signal's reason.

A wrap that starts over makes a new budget and new listings, and reads the
new configuration's denyReadGlobBudget.

In the scan's catch the abort is looked at first, then what ripgrep listed.
The walk, the expansion and the manager have all been rewritten on main
since this branch left it, so in the five files git marks the tree is
main's with the budget put back in, as it was worked out on the
integration branch:

- One options object for the expansion, `{ anchor?, budget?, listings? }`,
  all three passed on. A walk beneath an anchor draws on the budget too.
- The shared listings are made beside the budget, one of each per read
  configuration. The wrap makes its map, hands it over and empties it
  before the scan; the budget's clock starts after the wait in line.
- An entry is charged before the fast path skips it: the budget bounds the
  entries looked at, also those read from a listing another pattern made.
- In the walk the step comes first, and the clock is read after it and
  before the next listing. Turns given to other work count against the
  deadline, which can refuse a wrap earlier and never shortens a list.
- The budget error gets its code around the delegated generator. An abort
  is raised by the driver and does not pass through there, so with a spent
  budget and an aborted signal the wrap rejects with the signal's reason.
- The denyRead patterns and the credential denies go through one expander:
  one budget and one set of listings for both lists.
- A wrap that starts over makes a new budget and new listings.
- `directoriesListed` counts the records listed for a position, not
  `records.size`, which also counts directories that could not be listed.

test/sandbox/read-deny-walk-joins.test.ts comes from the integration branch:
the choices that only exist between the budget, the listings and the steps.

Co-Authored-By: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FydxqUESqxwSCVFnhk3gfv
Co-authored-by: ronleizrowice <ronleizrowice@anthropic.com>
claude and others added 5 commits October 20, 2026 11:42
- An entry counts against `maxEntries` when it is read from disk. One that
  another pattern has read, and that comes from the shared listings, costs
  time alone. Charged like the rest, it made the tree that fits 20,000,000
  divided by the number of `**` patterns: a 2.5 million entry repository
  under nine patterns was refused after about 12 of its 60 seconds, and it
  runs on main. This turns round a choice the join suite asserted.
- allowRead patterns draw on the same budget. They ran the same walk with
  none.
- A wrap's `denyReadGlobBudget` replaces the limit it names, not the whole
  object: naming one used to reset the other to its default.
- The limits are checked where the budget is made, by the schema. Only the
  CLI ran it, so through initialize(), updateConfig() or customConfig a NaN
  turned a limit off, a misspelt key left the default, '1000' was
  concatenated.
- On a file system that lists names without types, the clock is read before
  each lstat. 200,000 such entries at 1 ms held the thread for 200 s.
- The refusal no longer says "denyRead pattern" of a credentials.files or
  allowRead one. The type of its `.cause` is exported.
- The budget keeps `timeoutMs` as given, instead of a deadline it is rebuilt
  from by subtraction and rounding. The options of the expansion are declared
  once. The directories are counted as they are listed.
- Tests: an anchored walk, a credentials.files pattern and an allowRead one
  are each held to the budget; each new case fails with the line it guards
  taken out. "A signal wins over a spent budget" goes: it aborted in the
  allowRead walk, before the budget was drawn on, and nothing but a test
  can abort in the middle of a step. No temp path goes into a RegExp, and
  the join suite has the platform gate its sibling has.

Co-Authored-By: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FydxqUESqxwSCVFnhk3gfv
Co-authored-by: ronleizrowice <ronleizrowice@anthropic.com>
A simplification pass over the whole diff against main. No behaviour of the
budget changes.

- A spent budget is turned into the profile error in one place, where the
  read entries are resolved, and the two expander factories are gone.
- GlobWalkBudgetError works out which limit was reached and has one message.
- The walk no longer counts directories listed and entries looked at, which
  only the debug line read; that line is main's again.
- The two suites are one, over one planted tree and one clock that moves a
  millisecond a reading: 1,231 lines of tests are 392.
- The README says it in one paragraph.

Co-Authored-By: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FydxqUESqxwSCVFnhk3gfv
Co-authored-by: ronleizrowice <ronleizrowice@anthropic.com>
- A budget that runs out while the types of a listing are asked for was taken
  for a real path that could not be read where a shorter name leads to the
  directory: it was listed once more, and had that name answered with nothing
  by then the directory was covered whole and the wrap went on.
- A limit a wrap's own configuration holds as undefined no longer resets the
  session's, and a budget that is no object is refused like any other value.
- A wrap that starts over is shown to start with nothing spent.

Co-Authored-By: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FydxqUESqxwSCVFnhk3gfv
Co-authored-by: ronleizrowice <ronleizrowice@anthropic.com>
Co-Authored-By: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FydxqUESqxwSCVFnhk3gfv
Co-authored-by: ronleizrowice <ronleizrowice@anthropic.com>
Co-Authored-By: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FydxqUESqxwSCVFnhk3gfv
Co-authored-by: ronleizrowice <ronleizrowice@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants