[koala73]

Problem

Production /api/health (2026-10-31 03:17Z) reported SEED_ERROR on two tender sources that were still serving data:

• globalTendersKenyaPpip: one HTTP 429 at 02:33Z, 100 retained records, last success 61 min earlier. tenders.go.ke answers 200 again.
• globalTendersBulgariaEop: HTTP 403 three runs in a row from 00:33Z. storage.eop.bg publishes day D's bucket around 06:00Z on D+1 and returns 403 AccessDenied until then. Probe at 03:20Z: open-data-2026-10-29/ 200, open-data-2026-10-30/ 403. The adapter reads now - 1 day, so it failed from 00:00 to 06:00 UTC every day.

Tender data that is up to a day old is acceptable, including weekend gaps. The pipeline had a 180-min limit in three places: producer retention, status-key TTL and health budget.

Change

• Bulgaria adapter (scripts/_global-tender-feeds.mjs): on 403/404 for yesterday's bucket, read the day before. Other errors still throw.
• Retention (scripts/_global-tenders.mjs): a failed source keeps its last-good open tenders for 24h (TENDER_SOURCE_RETENTION_MS) instead of 180 min. Rows whose deadline has passed still drop.
• Status TTL (scripts/seed-global-tenders.mjs): per-source status and seed-meta keys live 26h, so the key ages into STALE_SEED before it expires to EMPTY.
• Health (api/health.js): 15 per-source tender checks get maxStaleMin: 1440 and sourceFailureWithinBudgetOk. While the last success is under 24h old, a failed attempt is OK only if the source still serves something true: retained rows (stale with recordCount > 0) or a verified empty result (error + confirmedEmpty). An error that serves nothing still warns. Past 24h it warns again.
• Contracts Finder is unchanged on the health side. It keeps its own first-failure containment (composeContractsFinderHealth), which recounts usable canonical rows. The aggregate globalTenders check also stays at 180 min, so a fully stopped seeder still alarms.

Verification

• New failing-first test: Bulgaria falls back to D-2 on 403 and rethrows a 500 (tests/global-tenders-additional-feeds.test.mjs).
• Retention and health tests moved to the 24h boundary: 23h59 retained/OK, 24h expired/not OK. Added per-source checks for a verified empty result (OK) and a failure serving nothing (not OK).
• All suites whose names match health/seed-meta/tender/procurement: 68 files, 1,013 pass, 0 fail.

Not verified: live behaviour after deploy (next Railway tick plus Vercel deploy).

https://claude.ai/code/session_01PScvxj8Yb7wKE4NzHeCmg3

Summary by CodeRabbit

• Improvements
  • Global tender listings can remain available for up to 24 hours after a source refresh fails, when previously fetched records are retained.
  • Source health may remain healthy during this period when usable records are retained or an empty result has been confirmed; warnings return when the retention window expires.
• Bug Fixes
  • Bulgaria tender imports now retry the previous day’s listing when the current day’s listing is unavailable. Other listing errors continue to be reported.

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated
worldmonitor	Ready	Preview	Oct 31, 2026 3:40am UTC

[strix-security]

Strix Security Review

✅ No open security findings.

PR summary: The PR gives each global tender source a 24-hour last-good budget: a failed source keeps its open tenders for a day, its status keys live 26 hours, two tender health checks are re-graded against a 1440-minute budget, and the Bulgaria adapter falls back to the day-before bucket while yesterday's is still unpublished.

Review summary

Reviewed the four production files changed by this PR — the new sourceFailureWithinBudgetOk branch and 1440-minute budgets in api/health.js, the 24-hour failed-source retention in scripts/_global-tenders.mjs, the 26-hour status TTL in scripts/seed-global-tenders.mjs, and the 403/404 day-before fallback in fetchBulgaria. No security vulnerability was identified. The health-grading fields are written only by the trusted seeder, the suppression is hard-bounded by the same 24-hour last-success budget, and per-source staleness remains visible in the published snapshot and RPC freshness layer. The retained tender path re-applies the deadline, active-status, source-scoped HTTPS allowlist and shape checks on every merge, and the Bulgaria fallback builds its retry URL from a fixed host and internally derived date while neutralising the listing key and rethrowing every non-403/404 error. The remaining effects are deliberate data-freshness and monitoring trade-offs that no attacker can induce.

Ruled out (3)

• Does extending a failed source's retention from 180 minutes to 24 hours let stale, expired, or unsafe tender rows reach API consumers?
  Every integrity guard is re-evaluated on each merge — the future-deadline check, active/open status, the source-scoped HTTPS host allowlist, and the id/title shape — and a source removed from the adapter list stops being retained immediately.
• Can the new sourceFailureWithinBudgetOk clause make a broken or compromised tender source report healthy?
  The clause is gated on the data key being inside the same 24-hour budget and only trusts seed-meta fields written by the trusted seeder, so it neither outlives that budget nor accepts input from any untrusted caller.
• Can the Bulgaria 403/404 fallback be steered to another host or hide a real failure?
  The retried URL is a fixed storage.eop.bg template built from the internal clock, the listing key is passed through encodeURIComponent so hostile keys stay in the bucket, and every error other than a numeric 403/404 is rethrown immediately.

_{Review 1 on this PR · 4330d18 · requested Oct 31 03:33 UTC when the PR was opened}

---

Reviewed by Strix
_{Re-run review · Configure security review settings}

[coderabbitai]

Walkthrough

Global tender sources now retain tracked data for up to 24 hours and use a matching health budget. Source status metadata persists for 26 hours. The Bulgaria adapter retries an earlier listing bucket after HTTP 403 or 404 responses.

Changes: Tender retention and health

Layer / File(s)	Summary
Retention windows scripts/_global-tenders.mjs, scripts/seed-global-tenders.mjs, tests/global-tenders-additional-feeds.test.mjs, tests/global-tenders.test.mjs, tests/list-global-tenders.test.mts	Tracked-source freshness changes to 24 hours, and source status metadata uses a 26-hour TTL. Tests cover retention and expiry.
Source health within the retention budget api/health.js, tests/global-tender-health.test.mjs, tests/global-tenders-world-bank-pipeline.test.mts	Global tender probes use a 1,440-minute stale limit. Qualifying failed attempts remain healthy when records or a confirmed-empty result exist and seed metadata is fresh. Contracts Finder keeps separate health handling.

Bulgaria listing fallback

Layer / File(s)	Summary
Earlier listing bucket fallback scripts/_global-tender-feeds.mjs, tests/global-tenders-additional-feeds.test.mjs	The adapter retries the earlier listing bucket after HTTP 403 or 404. Tests cover fallback success and propagation of other errors.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Change: Bug fix

Possibly related issues

• Data sources: procurement (29 new feeds + 7 optional debarment lists) #9840 — The issue proposes adding the Bulgaria EOP daily OCDS feed, and this change updates the Bulgaria adapter’s listing behavior.

Possibly related PRs

• koala73/worldmonitor#9883 — Both changes modify global ingestion health behavior in api/health.js.
• koala73/worldmonitor#9856 — That PR adds Global Tenders adapters, including Bulgaria; this change updates their retention and health budgets and adds the Bulgaria listing fallback.

Merge Risk: Minimal · up to 4330d

The change extends tender-source retention and the health budget to 24 hours and adds an earlier-bucket fallback for the Bulgaria listing. No concrete merge-blocking risk is established. The one remaining concern, an older bucket replacing newer records, is speculative and depends on a published bucket later becoming unavailable.

Pre-merge checks: 3 · 2

Failed checks: 2 warnings

Check name	Status	Explanation	Resolution
Description check		The description gives detailed problem, change, and verification information, but it does not follow the required template. It omits the required Summary, Type of change, Affected areas, Checklist, Do…	Reformat the description using the repository template. Add the required headings and complete the relevant checkboxes, mark non-applicable items clearly, and retain the existing problem, change, verification, and live-deployment verificati…
Docstring Coverage		Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documentat…	Write docstrings for the functions missing them to satisfy the coverage threshold.

Passed checks: 3 passed

Check name	Status	Explanation
Title check		The title clearly and concisely describes the main change: extending per-source tender health to a 24-hour last-good budget.
Linked Issues check		Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check		Check skipped because no linked issues were found for this pull request.

Full details: Description check

Explanation

The description gives detailed problem, change, and verification information, but it does not follow the required template. It omits the required Summary, Type of change, Affected areas, Checklist, Documentation Alignment Checklist, and Screenshots sections.

Resolution

Reformat the description using the repository template. Add the required headings and complete the relevant checkboxes, mark non-applicable items clearly, and retain the existing problem, change, verification, and live-deployment verification details.

Full details: Docstring Coverage

Explanation

Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documentation-only edits and deletions within surviving functions are included; unchanged diff context and fully deleted functions are excluded. Analyzed 3 functions across 9 files.

Resolution

Write docstrings for the functions missing them to satisfy the coverage threshold.

Finishing Touches

Generate docstrings

• Commit to this branch
• Create a new PR

Generate unit tests (beta)

• Commit to this branch
• Create a new PR

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

• X
• Mastodon
• Reddit
• LinkedIn

[greptile-apps]

Confidence Score: 5/5

[Medium impact] The PR appears safe to merge; no actionable issues were found.

Summary

Gives per-source tender health a 24-hour last-good budget.

• Failed tender feeds keep eligible open tenders for 24 hours.
• Bulgaria's tender feed falls back to the prior day's bucket.

No actionable issues found. Checks were based on source inspection; tests and live deployment behavior were not independently run.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart TD
  A[Source refresh fails] --> B{Last success within budget?}
  B -->|No| C[Health warns]
  B -->|Yes| D{Retained rows or confirmed empty?}
  D -->|Yes| E[Per-source health OK]
  D -->|No| C
  F[Aggregate snapshot] --> G[Separate 180-minute health check]

Loading

_{Reviews (1) · Last reviewed commit: "fix(global-tenders): give per-source ten..." · Reviewed by Greptile}

[coderabbitai]

Additional comments: 1

scripts/_global-tender-feeds.mjs-213-214: 1

213-214: Major · Data Integrity & Integration · Heavy lift

Unverified finding
Verification ran but could not confirm this finding. It is shown for review, not as a verified issue.

Do not replace a newer snapshot with the fallback bucket.

If yesterday’s bucket succeeds and later returns 403 or 404, this fallback reads the older bucket. mergeTenderSourceResults then replaces the newer Bulgarian records and marks the source fresh. New tenders can disappear, and cancelled tenders can reappear. (raw.githubusercontent.com)

Track the selected bucket date and retain a newer previous snapshot. Alternatively, treat this case as a failed attempt so the existing retention path applies.

---

Review info

Run configuration

• Source mode: Auto
• Repository: Unavailable
• Branch: Unavailable
• File: No configuration file from this repository applied
• Commit: Unavailable
• Layers: defaults
• Review profile: CHILL
• Plan: Advanced
• Run ID: d42b33a8-e655-4559-beb8-6d24251d2f3d

Commits

Reviewing files that changed from the base of the PR and between 26bd7c2 and 4330d18.

Files selected for processing: 9

• api/health.js
• scripts/_global-tender-feeds.mjs
• scripts/_global-tenders.mjs
• scripts/seed-global-tenders.mjs
• tests/global-tender-health.test.mjs
• tests/global-tenders-additional-feeds.test.mjs
• tests/global-tenders-world-bank-pipeline.test.mts
• tests/global-tenders.test.mjs
• tests/list-global-tenders.test.mts

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 6 remain after this review.

[koala73]

Re CodeRabbit's unverified finding on scripts/_global-tender-feeds.mjs:213-214 ("Do not replace a newer snapshot with the fallback bucket"): no change made.

The target bucket is isoDay(now - DAY), which only changes at UTC midnight. From 00:00 to about 06:00 on D+1, bucket D doesn't exist yet and returns 403. The fallback reads D-1, the same bucket the adapter served successfully all through day D. So the fallback never replaces a newer read. It keeps serving the last published day instead of failing for six hours. Probe at 03:20Z today: open-data-2026-10-29/ 200 with LastModified 2026-10-30T06:00Z, open-data-2026-10-30/ 403.

That leaves one case: a bucket that was readable and later gets withdrawn. There's no evidence the publisher does that. If it did, D-1 is still that portal's last published day, and closed tenders are filtered by deadline in finish() anyway.