Skip to content

feat(cli): accept VERYFRONT_API_KEY as the one credential name - #5514

Open
mattboon wants to merge 14 commits into
veryfront:mainfrom
mattboon:fix/cli-api-key-hint
Open

mattboon wants to merge 14 commits into
veryfront:mainfrom
mattboon:fix/cli-api-key-hint

Conversation

@mattboon

@mattboon mattboon commented Nov 11, 2026 •

Copy link
Copy Markdown
Contributor

Needs Koji's security review. This changes which environment variables the CLI and SDK accept as a credential.

Description

A Veryfront API key now has one user-facing name: VERYFRONT_API_KEY. That is the name Studio's "Get API key" page, the AI Gateway snippets and the docs already print. The CLI and SDK now read it everywhere they read VERYFRONT_API_TOKEN. VERYFRONT_API_TOKEN keeps working for existing setups and takes precedence when both are set.

Problem

The two names were never different credentials for a user: the same vf_ key works in either one.

Decisions

  1. VERYFRONT_API_KEY is the user-facing name, and the code keeps VERYFRONT_API_TOKEN internally. The token slot also holds the session after veryfront login, so it stays the CLI's single place for "the credential in use". The key is what users set.
  2. VERYFRONT_API_TOKEN wins when both are set. Existing CI secrets, scripts and .env files behave exactly as before.
  3. A blank VERYFRONT_API_TOKEN counts as unset, the same rule the CLI already applies to a blank token.
  4. The key is copied once, at load time, never read as a fallback afterwards. After each env load, if the token slot is unset or blank, the key is copied into it with the key's provenance: a .env file and its $NAME expansion taint, or the shell. A read-time fallback would be unsafe. The CLI deliberately deletes VERYFRONT_API_TOKEN when the credential must not reach a repository-steered API URL, and a fallback would bring the credential back through the key.
  5. Plain SDK scripts that never load env files read the key through getHostEnv. Once the env loader has run in a process, getHostEnv stops falling back (settleVeryfrontApiKeyAlias).
  6. Project-reachable getEnv is unchanged. In Cloud, getEnv reads the project's own env snapshot, and a project's gateway key must not become a server-side identity. Only the host-level credential resolution changes.
  7. Messages name the variable the user actually set. whoami says "(via VERYFRONT_API_KEY)" or "(via VERYFRONT_API_TOKEN)". Static help and errors say VERYFRONT_API_KEY.
  8. Platform-injected and operator names are unchanged. That covers VERYFRONT_API_TOKEN_FILE, the Cloud bootstrap variables, and the agent-service registration config, which reads its own config record. Those docs still say VERYFRONT_API_TOKEN.

Behavior change to confirm: a developer who exported a gateway key as VERYFRONT_API_KEY and is logged in as a different account now has the CLI use that key. An exported VERYFRONT_API_TOKEN already beats the stored login today, so this matches it.

Security

The trust rules are unchanged and now also apply to the key:

  • A key in the same .env file as a repository-steered VERYFRONT_API_URL is accepted only if written literally, exactly as a VERYFRONT_API_TOKEN there would be.
  • A key that $NAME expansion copied out of the shell is refused for a repository-steered endpoint, and the secret is not echoed.
  • A shell key is never sent to a repository-steered endpoint.
  • A .env key carries the env-file value marker, so getHostEnvExcludingEnvFile and the sandbox's "URL and token from matching sources" check judge it exactly like a .env token.

Each rule has a VERYFRONT_API_KEY test next to the existing VERYFRONT_API_TOKEN tests in cli/shared/config.test.ts.

Evidence

User level, staging. veryfront whoami with a clean HOME (no stored login) and a scrubbed environment, VERYFRONT_API_URL=https://api.veryfront.org:

main,   only VERYFRONT_API_KEY set:   ✗ Not logged in / Run 'veryfront login' to authenticate
branch, only VERYFRONT_API_KEY set:   ✓ Authenticated with an API key (via VERYFRONT_API_KEY)
branch, valid VERYFRONT_API_TOKEN and a junk VERYFRONT_API_KEY:  ✓ Authenticated with an API key (via VERYFRONT_API_TOKEN)

SDK, Sandbox.create() with only VERYFRONT_API_KEY set. The project validation runs after auth, so this costs no credits:

main:   Sandbox auth not configured. Set VERYFRONT_API_TOKEN, provide request-scoped Veryfront credentials, or pass authToken explicitly.
branch: Failed to create sandbox: 400 {... "slug":"validation-failed", "detail":"project_id (or an environment_id that belongs to a project) is required ..."}

On the branch the key authenticates and the request reaches the API's project validation.

Tests, written first. Regression proof: with src/platform/compat/process/env.ts and src/utils/env-loader.ts reset to main and the tests kept, every behavior test below fails. The security guards pass both before and after, as they must.

  • src/utils/env-loader.test.ts, "VERYFRONT_API_KEY as the user-facing credential name":
    • .env key with the file's provenance
    • exported key with process provenance
    • token wins
    • .env token beats a shell key
    • blank token
    • blank key
    • expansion taint carried
    • a project directory loaded after startup
    • a token removed after loading does not come back
  • src/platform/compat/process/env.test.ts, "VERYFRONT_API_KEY for processes that never load env files": key read, token wins, blank cases, a registered host credential wins, no fallback once settled, getEnv unchanged.
  • src/sandbox/sandbox.test.ts: Sandbox.create() sends Bearer <API_KEY> with only the key set, and the token wins when both are set.
  • cli/shared/config.test.ts, "VERYFRONT_API_KEY under the same repository trust rules": a literal .env key is accepted beside the URL, an expanded .env key is refused, a shell key is never sent to a steered endpoint, and an exported key works on the default host.
  • cli/auth/login.test.ts: whoami names VERYFRONT_API_KEY for a .env key and for an exported key, and never prints the key.
  • cli/mcp/tools/deploy-tool.test.ts: the MCP deploy tool still classifies the CLI's missing-credential error after the wording change. Self-review found that the tool matched on the old message text, a real regression the copy change would have caused. It was red before the fix and now matches an exported constant.

Each touched test file passed on two consecutive runs.

Checks, all on the pinned Deno 2.9.7.

  • The pre-push hook passed: fmt:check, deno lint, deno check src/index.ts and the full test:unit.
  • The CI hygiene gates passed: semantic dispositions, cwd-relative reads, sanitizer baseline, testing front door, skipped tests, test typecheck and anti-slop.
  • generate:manifests:check, docs:api-reference:check, docs:errors:check, docs:public:check, docs:snippets:check and docs:validate passed.
  • Broader suites pass:
Suite Tests
src/agent/runtime 876
cli/mcp/tools 14
cli/help, router, schema, cli/commands/integration pass
tests/docs 67
cli/auth/exit-code.integration.test.ts pass
tests/integration/cli/agent-ux-contract.integration.test.ts pass

Generated files (regenerated on Deno 2.9.7, with only the expected lines changing): docs/guides/errors.md, docs/api-reference/veryfront/{runs,conversations}.md (JSDoc samples) and cli/skills/core-skills.generated.ts.

Changes

  • Core. src/platform/compat/process/env.ts adds VERYFRONT_API_KEY_ENV, settleVeryfrontApiKeyAlias and the SDK-only fallback in getHostEnv. src/utils/env-loader.ts adds the one-time copy with provenance, isVeryfrontApiTokenFromKey and veryfrontApiCredentialEnvName.
  • CLI copy. login/whoami messages, help for login, logout, deploy, env, pull and merge, the config errors, the deploy readiness message, and the MCP standalone and deploy tool.
  • SDK and runtime copy. The sandbox and blob-storage auth errors, the error registry (eval auth), the veryfront-api MCP message, and the JSDoc samples in veryfront/runs and veryfront/conversations.
  • Docs. 17 guides now say VERYFRONT_API_KEY where a user sets or passes the credential. The providers guide states that VERYFRONT_API_TOKEN still works and wins.
  • Docs validator. The deploy-page rule accepts either name.

Out of scope

  • The docs IA prototype in veryfront-studio#8463 still teaches VERYFRONT_API_TOKEN in about 22 places. Switch it after a veryfront release contains this change, because the current release reads only the token.
  • The sandbox platform stripping VERYFRONT_API_TOKEN, and blocking staging's private-address hostnames, is API and platform behavior (fix(docs): validate public documentation destinations #4275).
  • The internal eval harness's VERYFRONT_TOKEN is unchanged.

Refs veryfront/veryfront-issue-inbox#4088
Refs veryfront/veryfront-issue-inbox#4220

Type of Change

  • New feature (non-breaking change that adds functionality)
  • Documentation update

Checklist

  • I have made corresponding changes to the documentation (if applicable)
  • I have added tests that prove my fix is effective or that my feature works

Behavior change, the .env case (from review): a project .env that sets VERYFRONT_API_KEY for the app's own gateway calls now also beats veryfront login for management commands such as deploy, pull and env. A .env VERYFRONT_API_TOKEN does the same today, and the key only goes to the default host. Because VERYFRONT_API_KEY is the usual name in app .env files, a developer with a gateway key there who is logged in as another account gets that key's account; veryfront whoami shows it as "(via VERYFRONT_API_KEY from .env)".

@kwakayama kwakayama left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Security review: no defects found at 66ad121

What I checked

  1. Shell or stored credential to a repo-chosen host. An exported key is copied with process provenance (src/utils/env-loader.ts:384-386). isRepositorySuppliedCredential (cli/shared/config.ts:432-437) reads getEnvSource("VERYFRONT_API_TOKEN"), so a shell key is refused for a .env-steered URL. A repo .env key cannot shadow an exported key, because the primary load keeps non-empty existing values. A blank repo VERYFRONT_API_TOKEN only lets the shell key in with process provenance, which is still refused.
  2. .env provenance. A .env key gets markEnvFileValue plus the same file and expandedFromProcessEnv record as a .env token (env-loader.ts:374-383). So the same-file literal rule, the $NAME taint, getHostEnvExcludingEnvFile and the sandbox "matching sources" check all judge it like a token. Windows lowercase spellings resolve through findAliasedEnvSource.
  3. A deleted token coming back. The first load settles the alias (env-loader.ts:370). After that, getHostEnv no longer falls back (src/platform/compat/process/env.ts:252). The only later re-copy is loadAdditionalEnv in dev, and it runs before applyQualifiedRuntimeAuth deletes anything. Server bootstrap skips markEnvLoaded once the CLI has loaded. A child process that inherits the key re-reads veryfront.json and reaches the same refusal.
  4. SDK and sandbox. The read-time fallback only applies in processes that never run the env loader. Those processes have no env-file marks, so nothing is misclassified. Hosted pods do not inject VERYFRONT_API_KEY: I found no hits in infrastructure or server. Project-injected task env goes to ctx.env, not the process env, so a project's gateway key never reaches the host token slot.
  5. Exported key beating a stored login. This is surfaced. whoami prints "(via VERYFRONT_API_KEY)", or "(via VERYFRONT_API_KEY from .env)" for a file key (cli/auth/login.ts:74-83). The 401 hint and the probe remedy name the key too.
  6. Tests fail without the change. At the head, the 6 touched suites pass (640 steps). With the alias copy and the read-time fallback stubbed out, 11 behavior tests fail across env-loader, env, sandbox, config and login. Three more mutations each turned tests red:
    • Dropping the taint: 2 fail.
    • Giving a shell key env-file provenance: 4 fail, including "keeps an exported key on the default host".
    • Removing settleVeryfrontApiKeyAlias(): "does not bring back a token the CLI removed after loading" fails.

Non-blocking notes (no change required for security)

  • The behavior-change note covers only an exported key. A project .env that carries VERYFRONT_API_KEY for the app's gateway calls now also beats veryfront login for management commands such as deploy, pull and env. That matches a .env VERYFRONT_API_TOKEN today, and the key only goes to the default host. Still, that name is the common one in app .env files, so please add it to the behavior-change note.
  • RESERVED_TASK_ENV_KEYS (src/runs/runtime-env.ts:12) and the sandbox -u list still leave out VERYFRONT_API_KEY. This is intended for #4275. The key a caller passes can only act as that caller's own identity.

@mattboon

Copy link
Copy Markdown
Contributor Author

Thanks. Added the .env VERYFRONT_API_KEY case to the behavior-change note in the description. The RESERVED_TASK_ENV_KEYS and sandbox -u gap stays with veryfront/veryfront-issue-inbox#4275.

@mattboon
mattboon marked this pull request as ready for review November 11, 2026 14:01
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 11, 2026 •

Copy link
Copy Markdown
Codex Review Summary: ✅ Completed

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-13T01:09:33.574883Z 87b04ec Manual request
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@github-actions

Copy link
Copy Markdown

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 66ad121f4d

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread src/platform/compat/process/env.ts Resolved
Comment thread cli/auth/login.ts Resolved
@mattboon

Copy link
Copy Markdown
Contributor Author

Fix for the two Codex threads on 66ad121f, pushed as 59743fb75.

Evidence (red without the fix, green with it):

cli/auth/login.test.ts            without fix: FAILED | 2 passed | 1 failed   with fix: ok | 3 passed | 0 failed
scripts/test/suites.test.ts       new "scrubs the VERYFRONT_API_KEY alias" test: red -> green
scripts/test/runtime-env.test.ts  new test: red -> green
src/testing/sanitize-process-env.test.ts  new test: red -> green
runtime-auth.test.ts with VERYFRONT_API_KEY exported: ok | 1 passed | 0 failed

Pre-push hooks (fmt, lint, type check, unit suite) pass on the pinned Deno 2.9.7.

@github-actions

Copy link
Copy Markdown

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 59743fb75c

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread src/utils/env-loader.ts Resolved
Comment thread src/platform/compat/process/env.ts Resolved
@kwakayama

Copy link
Copy Markdown
Contributor

Not approving yet: Codex left two open threads on 59743fb7.

The 66ad121f..59743fb7 delta itself is good:

  • VERYFRONT_API_KEY is scrubbed in the Deno suite, Node/Bun runtime env, preload sanitizer, runtime E2E helper and binary-server setup.
  • Reverting each scrubber line and login.ts makes its new test fail, so the tests guard the fix.
  • The login hints now name the variable you set, and the key value is never printed.

Open threads:

  • P1, token file vs key provenance (thread). The claim holds. VERYFRONT_API_TOKEN_FILE wins the value, but the token keeps the .env key's provenance, so a repo-steered URL can receive the token-file secret. main has the same gap with a literal VERYFRONT_API_TOKEN in a project .env, so this PR does not widen it. Fix it here (skip the alias while a token file is set) or track it in a follow-up issue, then resolve.
  • P2, remaining VERYFRONT_API_TOKEN hints (thread). Valid but not a blocker. A follow-up is fine; please resolve the thread with a link.

I'll approve once both threads are resolved and Codex is clean on the head.

@github-actions

Copy link
Copy Markdown

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 7258c0c907

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread src/utils/env-loader.ts Resolved
@mattboon

Copy link
Copy Markdown
Contributor Author

Fix for the Codex P1 on 7258c0c9, pushed as 8d2ce9c82.

src/utils/env-loader.test.ts  new "clears an earlier key alias when a later load brings a token file"
  without fix: FAILED | 0 passed | 1 failed   with fix: ok | 1 passed | 0 failed
src/utils: ok | 83 passed | 0 failed      cli/auth: ok | 10 passed | 0 failed
runtime-auth.test.ts: ok | 1 passed | 0 failed

Pre-push hooks (fmt, lint, type check, unit suite) pass on Deno 2.9.7.

@github-actions

Copy link
Copy Markdown

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 8d2ce9c827

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread src/platform/compat/process/env.ts Resolved
@kwakayama

Copy link
Copy Markdown
Contributor

@mattboon #5515 just merged and touches cli/shared/config.ts, so this branch now conflicts with main. Please merge main and keep both changes: #5515 refuses a token file for a repository-chosen API URL, and this PR's alias handling. I'll approve once the new head has green CI and a clean Codex review.

@github-actions

Copy link
Copy Markdown

@codex review

@mattboon

Copy link
Copy Markdown
Contributor Author

Fixes for the remaining Codex findings, pushed as cd4777615. Both earlier deferrals are now fixed here.

P2 token file vs key fallback  src/platform/compat/process/env.test.ts  without fix: FAILED | 1 failed   with fix: ok
P2 config reports the key name cli/commands/config/handler.test.ts       without fix: FAILED | 1 failed   with fix: ok
P2 remaining hints             up, remote file tools, runs client, cloud/local providers, AG-UI, error registry, docs-agent template
Affected suites: up, ag-ui handler, chat fallback, runs client, providers, cli/mcp, errors, process env, sandbox, config, env-loader, cli/auth: all ok
Generated: docs:errors:check, docs:api-reference:check, generate:manifests:check all current

Pre-push hooks (fmt, lint, type check, unit suite) pass on Deno 2.9.7.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Something went wrong. Try again later by commenting “@codex review”.

Failed to sample tokens
ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

@github-actions

Copy link
Copy Markdown

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: f6e4ce3bb5

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread src/utils/env-loader.ts Resolved
@kwakayama

Copy link
Copy Markdown
Contributor

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: ad24df7753

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/utils/env-loader.ts Resolved
@github-actions

Copy link
Copy Markdown

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: ad24df7753

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread src/utils/env-loader.ts Resolved
@github-actions

Copy link
Copy Markdown

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Something went wrong. Try again later by commenting “@codex review”.

Failed to sample tokens
ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

@mattboon

Copy link
Copy Markdown
Contributor Author

Reproduced in the docs flows pilot (11 Nov 2026, veryfront 0.1.1288, AI Agent template, npm run dev, no TTY):

Env /api/ag-ui answer
no key NO_MODEL_CREDENTIALS
VERYFRONT_API_KEY=<staging key> NO_MODEL_CREDENTIALS (the name every docs page uses)
VERYFRONT_API_TOKEN=<same key> calculator called, "An 18% tip on £84.50 is £15.21."

This PR fixes the second row. Results: veryfront/veryfront-studio#8496 (comment), board veryfront/veryfront-issue-inbox#4318.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Something went wrong. Try again later by commenting “@codex review”.

Failed to sample tokens
ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

@kwakayama

Copy link
Copy Markdown
Contributor

CI on c1bafe9 (verification PR #5521): ci (lint) still fails at lint:test-semantic-dispositions:

  • src/config/environment-config.test.ts: disposition is missing the filesystem-write effect.
  • src/testing/sanitize-process-env.test.ts: no semantic disposition.
    The new commit only touches docs, so this is unchanged from d3035f4. Everything else is green. Once both have dispositions (or the tests are made hermetic), I'll move Verification only for #5514 — do not merge #5521 to the new head and approve.

Studio, the gateway snippets and the docs tell users to export
VERYFRONT_API_KEY, but the CLI and SDK only read VERYFRONT_API_TOKEN, so
'veryfront whoami' said 'Not logged in' with the key the user just set.

The CLI and SDK now read VERYFRONT_API_KEY too:
- After each env load, a key is copied into the VERYFRONT_API_TOKEN slot
  once, only when that slot is unset or blank, and with the key's
  provenance, so every repository trust rule judges a .env key exactly
  like a .env token (including $NAME expansion taint).
- Processes that never load env files (SDK scripts) read the key through
  getHostEnv. Once the loader has run, getHostEnv never falls back, so a
  token the CLI removed on purpose cannot come back through the key.
- VERYFRONT_API_TOKEN keeps working and wins when both are set.

Help, errors, docs and SDK examples now name VERYFRONT_API_KEY; messages
about an active credential name the variable the user actually set. The
MCP deploy tool classifies the missing-credential error by an exported
constant instead of copying its wording.

Refs veryfront/veryfront-issue-inbox#4088
Refs veryfront/veryfront-issue-inbox#4220
… hints

Codex review on 66ad121:
- P1: every test environment scrubber removed VERYFRONT_API_TOKEN but not
  VERYFRONT_API_KEY, so an exported key became an active credential in test
  children. Add the key to the Deno suite, Node/Bun runtime, preload, runtime
  E2E and binary-server scrubbers.
- P2: login messages told you to unset VERYFRONT_API_TOKEN even when the key
  supplied the credential. Name the variable you set in the rejected,
  unavailable, existing-session and invalid-warning messages, and fix the
  runs target README sample.
A token file supplies the credential ahead of VERYFRONT_API_TOKEN. Copying a
project .env key into the token slot attached the key's env-file provenance to
the token file's secret, so a repository-steered API URL could receive it.
Skip the alias while VERYFRONT_API_TOKEN_FILE is set.
…ater load

veryfront dev --project loads the cwd .env, then the project's. A key aliased
from the first load kept its env-file provenance after the second load set
VERYFRONT_API_TOKEN_FILE, so the token file's secret could be treated as
repository-supplied. Revoke the alias and its provenance whenever a token file
is set, instead of only skipping a new alias.
…in every hint

Codex review on 8d2ce9c and 59743fb:
- getHostEnv no longer falls back to VERYFRONT_API_KEY while a nonblank
  VERYFRONT_API_TOKEN_FILE is set, so an SDK process (for example
  Sandbox.create) uses the renewable token file instead of a stale key.
- The remaining missing-credential hints now say VERYFRONT_API_KEY: up,
  remote file tools, runs client, Veryfront Cloud and local providers, the
  AG-UI no-credentials error and the authentication-required suggestion.
  'veryfront config' reports apiToken (VERYFRONT_API_KEY) when the key
  supplied the token. The docs-agent template exports the key.
- Regenerate docs/guides/errors.md, the RSC bundle and the templates manifest.
withoutHostBinaryInfraEnv removed VERYFRONT_API_TOKEN but not the key. The
VFS proxy-mode E2E reloads .env after the suite scrubber, so a developer key
became the token in compiled children pointed at the real API. Add the key
to the blocklist that the VFS and compiled-binary E2E suites share.
An additional env load keeps values defined before it, and it counted the
token copied from VERYFRONT_API_KEY as one. With veryfront dev --project and
an exported key, the project's explicit VERYFRONT_API_TOKEN was skipped, which
broke token-over-key precedence and rejected same-file self-hosted pairs. The
precedence check now treats the alias as unset, so only values the user set
keep precedence.
RESERVED_TASK_ENV_KEYS stripped VERYFRONT_API_TOKEN but not the key that can
now supply it, so a task started with VERYFRONT_API_KEY copied the developer
credential into ctx.env, and verified skill scripts, which reuse the same set,
could receive it. Reserve the key alongside the token.
In a process that never loads env files, only getHostEnv saw
VERYFRONT_API_KEY. The environment snapshot carried no token, so integration
discovery and dependency write-back took the host-private branch, which
resolves VERYFRONT_API_URL first instead of VERYFRONT_API_BASE_URL. With
separate GraphQL and REST hosts, a key-only script reached a different
backend than a token-only one.

The snapshot now reads the key through readUnloadedVeryfrontApiKey, the rule
getHostEnv already used: only before the env loader resolves the key, and
never while a token file is set.
…blank token as unset

Codex review on f391d47:
- P1: the snapshot read the key fallback through the project-scoped reader,
  so inside a project env scope a project's VERYFRONT_API_KEY became the
  snapshot token and could authenticate a dependency write-back. The
  fallback now reads the host environment and never applies inside a project
  scope.
- P2: a whitespace-only VERYFRONT_API_TOKEN shadowed a valid key. A blank
  token now gives way to the key, as getHostEnv and the env loader already
  treat it. A blank token with no key is returned as before.
The bundled integrations reference still named VERYFRONT_API_TOKEN as the
platform credential for a connect. Regenerated core-skills.generated.ts.
The environment-config test wrote a temporary token file and the
sanitize-process-env test mutated Deno.env, so both needed a new
semantic disposition. The token-file rule is already covered in
env.test.ts, and the sanitize test now checks the exported key list.
@mattboon
mattboon force-pushed the fix/cli-api-key-hint branch from c1bafe9 to 742d577 Compare November 12, 2026 12:16
@mattboon

Copy link
Copy Markdown
Contributor Author

@kwakayama Fixed in 742d5775b, rebased onto main (082d0f8b5). The audit allowlist does not grow; both tests are hermetic now:

Local, Deno 2.9.7: lint:test-semantic-dispositions ok, docs:api-reference:check current, focused tests 17 passed (218 steps), full pre-push suite passed.

@github-actions

Copy link
Copy Markdown

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 742d5775b2

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread src/runs/runtime-env.ts Resolved
@kwakayama

Copy link
Copy Markdown
Contributor

Three failures at 742d5775b, from CI on the verification PR #5521. All three are in files this PR changes:

  1. ci (lint), testing front door: new raw primitives, which the baseline does not allow.
    • cli/auth/login.test.ts:3427, 3452, 3487, 3513, 3639, 3664, 3681, 3703 assign globalThis.fetch directly. Use withMockFetch or installMockFetch from src/testing/mock-fetch.ts.
    • cli/commands/config/handler.test.ts:29, 63, 306 call raw Deno.makeTempDir. Use makeTempDir or withTempDir from src/testing/deno-compat.ts.
  2. tests (bun): src/testing/sanitize-process-env.ts:29 calls Deno.env.delete(key) at module load, and Bun throws ReferenceError: Deno is not defined. Use the runtime-neutral env helper, or guard for Deno.
  3. tests (node shard 1/4): probably the same Deno reference under Node. Please confirm once (2) is fixed.

On Windows the env loader case-folds names, so a project .env entry
such as veryfront_api_key supplies the credential under a lower-case
name and passed the exact-match reserved check into task and skill
script env. The check now compares the upper-cased name.

Also keep the new tests on the shared test primitives: withMockFetch
and makeTempDir instead of a raw globalThis.fetch assignment and
Deno.makeTempDir, and move the sensitive key list into a module without
side effects so its test runs under Bun and Node.
@mattboon

Copy link
Copy Markdown
Contributor Author

@kwakayama All three CI failures from #5521 at 742d5775b, plus the open Codex P1, are fixed in df8757b47:

  1. ci (lint), testing front door: the new key-hint test in cli/auth/login.test.ts uses withMockFetch, and cli/commands/config/handler.test.ts uses makeTempDir from src/testing/deno-compat.ts. Back at the baseline (fetch-assignment 138, temp-dir 2), no baseline edit.
  2. tests (bun) and 3. tests (node shard 1/4): the same cause. The Node shard's only failure was src/testing/sanitize-process-env.test.ts, which loaded the module that calls Deno.env.delete. The key list now lives in src/testing/test-sensitive-env-keys.ts, which has no side effects, and its test imports only that module.
  3. Codex P1 (Windows key casing): isReservedTaskEnvKey compares the upper-cased name, so veryfront_api_key and Veryfront_Api_Token are reserved in task ctx.env and skill script env. The new test failed before the change:
reserves credential names in any letter case ... FAILED
AssertionError: Values are not equal: veryfront_api_key

Local, Deno 2.9.7:

deno task lint:testing-front-door             exit 0
deno task lint:test-semantic-dispositions     ok: 2579 unit executable(s), 732 disposed
deno task docs:api-reference:check            current (49 files)
deno task test:file <8 touched test files>    ok | 8 passed (241 steps) | 0 failed
pre-push suite                                All pre-push checks passed

@github-actions

Copy link
Copy Markdown

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: df8757b47c

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread tests/_helpers/proxy-mode.ts Resolved
withoutHostBinaryInfraEnv deleted blocked names by exact case. On
Windows a project .env can supply veryfront_api_key, which a compiled
child process reads as the credential. Match by folded name, as
buildTestProcessEnv and buildRuntimeTestProcessEnv already do.
@mattboon

Copy link
Copy Markdown
Contributor Author

@kwakayama The head is now 87b04eceb: df8757b47 (the three CI failures and the Codex P1, details above) plus the follow-up Codex P2 (case-folded names in tests/_helpers/proxy-mode.ts). All threads are resolved and the local pre-push suite passed. Could you move #5521 to 87b04eceb when you have a moment?

@github-actions

Copy link
Copy Markdown

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Something went wrong. Try again later by commenting “@codex review”.

Failed to sample tokens
ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

@mattboon

Copy link
Copy Markdown
Contributor Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 87b04ecebd

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

readEnv: (key: string) => string | undefined,
): string | undefined {
if (veryfrontApiKeyAliasSettled) return undefined;
if (isNonBlank(readEnv(VERYFRONT_API_TOKEN_FILE_ENV))) return undefined;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Keep blank token-file selectors authoritative

When VERYFRONT_API_TOKEN_FILE is defined as empty or whitespace and VERYFRONT_API_KEY is valid, this check (and the same nonblank guard in src/utils/env-loader.ts:389) treats the file selector as absent and returns or aliases the key. resolveSandboxAuthToken() consumes that environment token at src/sandbox/config.ts:95-113 before reaching the file resolver, even though readVeryfrontApiTokenFile() rejects every defined empty path (src/platform/cloud/credential-file.ts:40-43) and tests/integration/platform/renewable-runtime-credential.test.ts:61-69 codifies fail-closed behavior. Fresh evidence beyond the earlier nonblank-selector fix is this blank-selector contract; treat any defined selector as authoritative so a misconfigured renewable credential cannot silently fall back to the key.

Useful? React with 👍 / 👎.

Comment thread docs/guides/providers.md
Comment on lines +76 to +77
The examples read it from `VERYFRONT_API_KEY`, the same variable the Veryfront
CLI and SDK read, described above.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Accept the key in agent-service configuration

When a standalone cloud agent service is started with only the newly documented VERYFRONT_API_KEY, this SDK claim does not hold: startAgentService() calls its separate loadAgentServiceEnvFiles() (src/agent/hosted/veryfront-cloud-agent-service.ts:250-258), while parseAgentServiceConfig() retains only VERYFRONT_API_TOKEN (src/agent/service/config.ts:77-130). Registration therefore sees no token (src/agent/service/registration.ts:322-335), so auto silently skips registration and enabled fails startup. Fresh evidence after the prior hint sweep is this separate parser and loader path; accept the key there with token-over-key precedence too.

AGENTS.md reference: AGENTS.md:L108-L108

Useful? React with 👍 / 👎.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants