Skip to content

Judge the age gate, region block and rate limit checks - #21

Merged
arrufat merged 3 commits into
mainfrom
judge-presets
Nov 5, 2026
Merged

arrufat merged 3 commits into
mainfrom
judge-presets

Conversation

@arrufat

@arrufat arrufat commented Nov 3, 2026

Copy link
Copy Markdown
Member

Follows lightpanda-io/browser#4428, which makes classify ask the same twelve presets as the local page classifier.

PageVerdict gets is_age_gate, is_region_blocked and is_rate_limited. A 429 now sets is_rate_limited instead of is_blocked, matching page-classifier's LABELLING.md; code that checked is_blocked on a 429 will see the change.

Needs a browser with #4428: on an older binary the three attributes are never set, so reading them raises AttributeError. triage_crawl.py keeps its own rate_limited question until a release ships the preset.

The browser's classify now asks the same twelve presets the local page
classifier was trained on. A 429 names is_rate_limited rather than
is_blocked, as the classifier's labelling guide does.
Lists the three new classify presets, and picks up structured_data's
microdata section from browser main.
The browser now asks whether a page says to slow down as one of judge()'s
checks, and a 429 sets it too, so the example drops its own question and
the helper that combined the two.
@arrufat
arrufat marked this pull request as draft November 3, 2026 15:53
@arrufat

arrufat commented Nov 3, 2026

Copy link
Copy Markdown
Member Author

Draft until a browser release ships lightpanda-io/browser#4428. Every new check here, and triage_crawl.py's back-off, read presets the browser bundled in the current release doesn't ask yet, so on it verdict.is_rate_limited raises AttributeError. CI's _methods.py diff will also fail until nightly has #4428. Once a release has it: re-run triage_crawl.py and classify_pages.py, refresh the sample output in examples/README.md, then mark this ready.

@arrufat
arrufat marked this pull request as ready for review November 5, 2026 10:06
@arrufat
arrufat merged commit 5f508e1 into main Nov 5, 2026
15 of 19 checks passed
@arrufat
arrufat deleted the judge-presets branch November 5, 2026 10:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant