Skip to content

fix(kb): add kb_documents.world_id migration and KB schema-drift test - #9

Merged
Ian-Tharp merged 3 commits into
developfrom
claude/kb-upload-world-id-migration-z0pekg
Nov 3, 2026
Merged

Ian-Tharp merged 3 commits into
developfrom
claude/kb-upload-world-id-migration-z0pekg

Conversation

@Ian-Tharp

Copy link
Copy Markdown
Owner

What Changed

  • backend/app/migrations/sql/005_kb_documents_world_id.sql (new): ADD COLUMN IF NOT EXISTS world_id TEXT on kb_documents, plus idx_kb_documents_world_id. If world_id already exists as UUID, it is converted in place to TEXT. The migration also drops the legacy NOT NULL on kb_chunks.embedding, and only when that column still exists.
  • backend/app/migrations/sql/003b_knowledgebase_tables.sql (new): idempotent baseline for kb_documents, kb_chunks and kb_activity (CREATE TABLE IF NOT EXISTS + ADD COLUMN IF NOT EXISTS for every column added later). It sorts before 004 on purpose.
  • backend/app/dependencies.py: setup_db_schema() now guards ALTER COLUMN embedding DROP NOT NULL with an information_schema check, and creates world_id as TEXT to match the migration.
  • backend/tests/test_kb_schema_drift.py (new): schema-drift test. It needs no Postgres.

Why

Only backend/app/migrations/sql/ is applied at startup, by app/migrations/runner.py from the main.py lifespan. backend/migrations/ is never applied automatically, so this PR adds nothing there.

I reproduced both failures against a real Postgres 16 + pgvector database:

  1. The reported 500 (world_id): world_id was only ever added inside setup_db_schema(), which runs all of its DDL in one transaction. On a DB whose legacy kb_chunks.embedding column was already dropped (for example by backend/migrations/003_remove_openai_embeddings.sql), the unguarded ALTER TABLE kb_chunks ALTER COLUMN embedding DROP NOT NULL raised. That rolled back the whole transaction, world_id included. main.py logs the error and then skips every later ensure_*_tables() call as well.
  2. A second drift bug, found during the audit: on a fresh DB, migration 004_knowledge_attribution runs before the KB tables exist and fails. The first boot is left without instance_name, instance_timestamp and source_discussion, so uploads 500 on instance_name until a restart. 003b fixes the ordering.

Other tables the repository touches: kb_activity matches the code. kb_chunks had the legacy NOT NULL embedding problem, fixed in 005. The attribution functions are placeholders and run no SQL.

world_id is TEXT because it is an opaque scope key supplied by callers outside the kernel (world logic is archived, see ADR 006). It is not a foreign key to worlds.

How to Test

cd backend
uv run pytest tests/test_kb_schema_drift.py -q   # 13 tests, no DB needed
uv run pytest -q                                  # full suite

What the drift test does:

  • Replays app/migrations/sql/*.sql in the runner's order into a {table: {column: type}} model.
  • Calls every public knowledgebase_repository function against a fake asyncpg pool that records the SQL.
  • Asserts that every column referenced is created by some migration.
  • Also checks that no migration ALTERs a KB table before a migration creates it, that setup_db_schema() and the migrations agree on column types, and that the legacy ALTER is guarded.
  • A coverage test fails when a new repository function is not added to the call table.

Checks that fail without the fix:

  • Removing 005 → create_document: kb_documents.world_id and list_documents_by_world: world_id
  • Removing 003b → ordering errors for 004
  • Reverting dependencies.py → the type-mismatch and unguarded-ALTER tests fail

Manual verification (real Postgres 16 + pgvector):

  • Ran the actual startup sequence (runner → setup_db_schema()) twice each on three databases:
    • a brand-new DB
    • a DB with world_id missing and kb_chunks.embedding dropped (the broken state; before this fix, setup failed there with column "embedding" ... does not exist)
    • a DB where world_id already existed as UUID
  • All three ended with world_id text plus the index, and both boots succeeded each time.
  • On each DB, every repository function ran, including create_document(world_id=...) and list_documents_by_world.

Applying to an existing local DB

Option A (recommended): restart the backend. The runner applies 003b and 005 automatically at startup. Both are idempotent, so this works whether or not your DB already ran 004. Look for Applying migration: 005_kb_documents_world_id in the logs.

Option B: apply by hand with psql, for example when the backend isn't running:

docker compose exec -T postgres psql -U core_user -d core_db < backend/app/migrations/sql/003b_knowledgebase_tables.sql
docker compose exec -T postgres psql -U core_user -d core_db < backend/app/migrations/sql/005_kb_documents_world_id.sql
# optional: record them so the runner won't re-run them (re-running is harmless)
docker compose exec -T postgres psql -U core_user -d core_db -c \
  "INSERT INTO schema_migrations(version) VALUES ('003b_knowledgebase_tables'),('005_kb_documents_world_id') ON CONFLICT DO NOTHING"

Check with \d kb_documents: world_id should be text, and idx_kb_documents_world_id should exist.

Checklist

  • Tests pass (2840 passed, 1 skipped)
  • Code follows project style guidelines (black --check app/ tests/ clean)
  • Documentation updated (if applicable): migration headers explain the reasons; no docs page changes
  • Conventional commit messages used

Known gaps

  • The drift test's SQL extractor is a heuristic tuned to this repository's SQL style. It covers only knowledgebase_repository, not every repository.
  • backend/migrations/ (not auto-applied) still duplicates part of app/migrations/sql/. Consolidating or removing it is follow-up work.
  • setup_db_schema() is still one large transaction, so any other failing statement could cause the same kind of silent rollback. This PR only fixes the KB statement that is known to fail.
  • 004_knowledge_attribution.sql contains BEGIN/COMMIT inside the runner's transaction. It works today, but it is fragile. I left the file unchanged because it has already been applied on existing DBs.

Generated by Claude Code

setup_db_schema() runs all DDL in one transaction. On databases that
already dropped the legacy kb_chunks.embedding column, the unguarded
'ALTER COLUMN embedding DROP NOT NULL' raised and rolled back the whole
transaction, including 'ADD COLUMN world_id' and every table created
after it. Guard the ALTER with an information_schema check.

Also create kb_documents.world_id as TEXT (an opaque scope key from
callers outside the kernel) to match migration 005.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019KQ4WmeseA49GumEaZfYyj
- 003b_knowledgebase_tables: idempotent CREATE/ADD COLUMN for
  kb_documents, kb_chunks and kb_activity. Sorts before 004, which
  failed on a fresh DB's first boot because the migration runner runs
  before setup_db_schema() creates the tables.
- 005_kb_documents_world_id: ADD COLUMN IF NOT EXISTS world_id TEXT,
  converts an existing UUID column, adds idx_kb_documents_world_id,
  and drops the legacy NOT NULL on kb_chunks.embedding when present.

Fixes POST /knowledgebase/upload 500: column world_id of relation
kb_documents does not exist.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019KQ4WmeseA49GumEaZfYyj
Replays app/migrations/sql into a column model, captures the SQL every
knowledgebase_repository function sends through a fake asyncpg pool, and
asserts each referenced column exists. Also checks migration ordering
for KB tables, setup_db_schema type agreement, and the guarded legacy
ALTER. Runs without Postgres.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019KQ4WmeseA49GumEaZfYyj
@Ian-Tharp
Ian-Tharp marked this pull request as ready for review November 3, 2026 14:34
@Ian-Tharp
Ian-Tharp merged commit a1e3c5d into develop Nov 3, 2026
2 checks passed
Ian-Tharp added a commit that referenced this pull request Nov 3, 2026
Brings in #8 (fail-safe fault capture wiring), #9 (kb world_id migration),
#10 (coverage pass) and #11 (comprehension gate Phase 3).

Conflict resolutions:
- backend/app/evaluation/__init__.py: union of exports (develop's
  fault_reasons / maybe_capture_fault plus the claims API).
- backend/app/evaluation/capture.py: kept develop's fault_reasons /
  state_failed / maybe_capture_fault (CORE_EVAL_CAPTURE_FAULTS gate,
  never raises); the branch's unsupported-claim check becomes a new
  "unsupported_claims" fault reason instead of an early return in
  state_failed. Branch's dispatched_tool filter and refusal /
  blocked_actions / unsupported_claims metadata kept in
  fault_report_from_state.
- backend/app/models/core_state.py: kept both field sets (refusal,
  blocked_actions, claim_report from the branch; step_checks from
  develop).
- docs/evaluation_spec.md: auto-merged; added the unsupported_claims
  fault reason and the extra metadata keys to "Live fault capture".
- tests/evaluation/test_claims.py: asserts fault_reasons ==
  ["unsupported_claims"] for the unsupported-claim run.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants