Skip to content

fix(sdk/node): diagnose misrouted HTML responses on Connect streams - #2344

Open
fxia0018 wants to merge 3 commits into
TencentCloud:masterfrom
fxia0018:fix/sdk-node-misrouted-connect-streams
Open

fxia0018 wants to merge 3 commits into
TencentCloud:masterfrom
fxia0018:fix/sdk-node-misrouted-connect-streams

Conversation

@fxia0018

Copy link
Copy Markdown

Summary

Node.js counterpart of #2336 (Python) and #1823 (Go). It already includes the changes requested in the review of #2336.

When a data-plane request is misrouted (for example CUBE_PROXY_NODE_IP is not set, or the endpoint points at the Web UI), the server returns an HTML page. With HTTP 200, readConnectFrames reads <!DOC as an envelope header (flags 0x3c, length 558124867), and commands.run, files.watchDir and pty.create fail with:

Connect stream message too large: 558124867 bytes

With a 404/502 HTML page, the error is the raw page with no hint about routing.

Change (src/commands.ts, used by commands.ts, filesystem.ts, pty.ts)

  • htmlResponseError(): an HTTP 200 text/html response is rejected before the stream is read, with the Go SDK's message and hint.
  • readConnectFrames(): flag bits other than compressed (0x01) / end-stream (0x02) are rejected before the length is read, so an HTML, JSON or gzip body without an HTML content type reports response is not a Connect envelope (first 5 bytes: 3c21444f43); ....
  • streamHttpErrorDetail(): on HTTP ≥ 400, a body that is text/html or starts with <!doctype / <html is reported from a bounded (4 KiB) prefix, truncated to 200 characters, followed by the same status-specific hint as the Go SDK's validateConnectResponse (401/403, 5xx, other). A JSON error document keeps the server's message even when labelled text/html; other bodies are reported as before.
  • Hint text lives in two constants, ROUTING_HINT and NOT_AN_ENVELOPE_HINT.

Error classes are unchanged (Error for commands/watch, CubeSandboxError / ApiError for PTY). files.watchDir now includes the response body in its HTTP error, like the other two callers.

Tests

New test/connect-routing.test.ts (52 cases, local HTTP server, no network):

  • valid multi-frame commands.run (stdout + stderr + exit code) and pty.create / pty.connect streams, also written 3 bytes at a time so envelope headers span chunks;
  • readConnectFrames rejects HTML, BOM, leading newline, JSON and gzip prefixes, and accepts flags 0x00–0x03;
  • the four callers on a 200 HTML page, with and without an HTML content type;
  • the four callers on 404 / 502 / 401 HTML pages, an HTML page without a content type, a 100 KB page (truncated), a JSON error labelled text/html, and a plain-text error (no hint).

On master, 36 of the 52 fail. Full suite: npx vitest run → 333 passed, 2 skipped. npm run typecheck and npm run build pass.

The troubleshooting page is updated in #2336; once that lands I can add the Node.js method names there too.

When a data-plane request is misrouted (for example CUBE_PROXY_NODE_IP
unset or pointing at the Web UI), the server returns an HTML page with
HTTP 200. readConnectFrames read "<!DOC" as an envelope header and
commands.run, files.watchDir and pty.create failed with

    Connect stream message too large: 558124867 bytes

TencentCloud#1823 fixed the same diagnosis in the Go SDK. Port its checks:

- htmlResponseError(): a 200 text/html response is rejected before the
  stream is read, with the CubeProxy routing hint;
- readConnectFrames() rejects flag bits other than compressed and
  end-stream before interpreting the length, so HTML, JSON or gzip
  bodies without an HTML content type report "response is not a
  Connect envelope" with the first five bytes;
- streamHttpErrorDetail(): a non-200 HTML page gets the Go SDK's
  status-specific routing hint and is reported from a bounded,
  truncated prefix of the body. A JSON error document keeps the
  server's message.

Valid Connect streams are unaffected.

Signed-off-by: Jace Xia <fxia0018@student.monash.edu>
Comment thread sdk/node/src/commands.ts Outdated
const hint = routingPageHint(resp.status);
return text ? `${truncateDiagnostic(text)}; ${hint}` : hint;
}
for (;;) {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unbounded error-body read for non-routing bodies. For a body that is not a routing page this loop reads the response to EOF and keeps every chunk in chunks, then embeds the full text in the thrown message. The Go SDK caps HTTP error diagnostics at 64 KiB (maxErrorBodyBytes / " [truncated]" in sdk/go/errors.go), and this SDK's control-plane helper caps non-JSON bodies at 500 chars (errorMessageFromResponse in sandbox.ts).

commands.run and pty.openStream already did resp.text() on this path (and both clear their idle timer before calling this helper, so it is not bounded by it either). filesystem.watchDir is the new case: it previously aborted immediately without reading the error body, has no idle/body timeout, and now controller.abort() runs only after this read. A misrouted endpoint that returns a very large — or never-ending — non-HTML error body therefore makes watchDir() buffer and block before throwing. Consider bounding the total buffered prefix (e.g. 64 KiB) and appending a truncation marker, matching the Go behaviour.

Comment thread sdk/node/src/commands.ts Outdated
* Whether an HTTP error body looks like a Web UI or gateway page. A JSON
* error document keeps the server's message even when labelled text/html.
*/
function isRoutingPage(type: string, body: Buffer): boolean {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Routing-page detection diverges from the Go isRoutingPage this PR mirrors (sdk/go/connect.go):

  1. Any body starting with { is accepted as a genuine JSON error document without validating it. Go's looksLikeJSONErrorDocument requires valid JSON with a non-empty message/detail/error; otherwise a text/html body that merely starts with { counts as a routing page. Here such a response gets no hint.
  2. Go also treats a 404 with a non-empty, non-JSON text/* body (e.g. Go's net/http 404 page not found) as a routing page. Here text/plain never matches, so that common gateway page still produces a bare HTTP 404: 404 page not found with no CUBE_PROXY_NODE_IP hint.

If parity with #1823 is intended, mirroring both (plus a 404 text/plain test) would close the gap; otherwise it is worth documenting the narrower HTML-only scope.

@cubesandboxbot

cubesandboxbot Bot commented Nov 19, 2026 •

Copy link
Copy Markdown

Automated review — #2344 fix(sdk/node): diagnose misrouted HTML responses on Connect streams

Verdict: no blocking issues found. The change is well scoped, closely mirrors the Go SDK's validateConnectResponse / readConnectEnvelope diagnostics, and the new tests cover the failure modes that motivated it (200 text/html, non-envelope bodies without an HTML content type, 4xx/5xx HTML pages, bounded/released error bodies).

AI-generated review. This is not a human approval — a maintainer must still review and sign off. I reviewed the prefetched diff and the base-branch sources statically; I did not run the test suite.

What I checked

  • readConnectFrames (sdk/node/src/commands.ts):
    • Unknown flag bits are rejected before the length is read, matching Go's readConnectEnvelope; the reported first 5 bytes hex and hint match the Go wording. Valid flag values 0x00–0x03 still parse, so existing frame tests are unaffected.
    • The new finally reader.cancel() releases the HTTP connection both when the consumer breaks early and when the flag/size check throws; the existing "partial message" error still comes from the post-finally check.
  • htmlResponseError (sdk/node/src/connect.ts): only fires on HTTP 200 + media type text/html (case/parameter-insensitive), never reads the body, and is called before body handling in all three callers. commands.run / files.watchDir keep throwing Error, PTY keeps CubeSandboxError/ApiError with the same message prefixes.
  • streamHttpErrorDetail / isRoutingPage / looksLikeJsonErrorDocument: parity with the Go reference holds for HTML-looking bodies, 404 text/* non-JSON bodies, JSON error documents mislabelled text/html, truncated-JSON peeks, empty/whitespace bodies and non-text/* types. Status-specific hints (401/403, ≥500, otherwise), the 4 KiB peek, the 200-byte UTF-8-boundary truncation and the [truncated] marker all match.
  • Bounding/cleanup: error bodies are capped (64 KiB + 1, 10 s) and the reader is cancelled, so files.watchDir (which has no caller-side timeout) cannot hang indefinitely on an error page; the idle signal still bounds the read for commands.run/PTY.
  • Existing tests: by inspection, filesystem-watch.test.ts (WatchDir failed: HTTP 500 with no body → detail ""), pty.test.ts (503 JSON body → non-routing, unchanged) and connect-frames.test.ts (oversized/partial frames) still hold with the new logic.

Minor, non-blocking observations

  1. readErrorBody (sdk/node/src/connect.ts) — posted inline: the loop buffers whole chunks (so more than MAX_ERROR_BODY_BYTES + 1 bytes may be read before clipping), and on a read error / 10 s cancel the partial body is returned without a [truncated] marker, so a cut-off page reads as complete (Go returns "" on a read error).
  2. streamHttpErrorDetail reads up to 64 KiB before classifying, while classification only needs the 4 KiB peek (Go stops at the peek for routing pages). Functional output is identical; it only affects how long a slowly-streamed 4xx/5xx page can delay the diagnostic (still bounded by the 10 s timer).
  3. CONNECT_KNOWN_FLAGS in sdk/node/src/commands.ts is inserted directly above the following doc comment without a blank line — cosmetic only.

Scope note (out of scope, for a follow-up)

Sandbox.runCode (/execute, ndjson on the Jupyter port) is another data-plane stream that can be misrouted: a 200 HTML body currently parses to no events and returns an empty Execution rather than an error. It is pre-existing and not introduced here, but it is the same class of failure this PR addresses for Connect streams.

Overall: the implementation is faithful to the Go/Python intent and the added test matrix is thorough; only low-severity diagnostics polish is suggested.

@liciazhu

Copy link
Copy Markdown
Collaborator

Thanks for the fix @fxia0018 . Two issues must be fixed before merge.


1. Error-body reads regress from "bounded by timeout" to "unbounded, no timeout" (Major)

streamHttpErrorDetail (sdk/node/src/commands.ts:87-123):

  • The non-routing-page branch has no byte limit:
    for (;;) {                       // unbounded
      const { done, value } = await reader.read();
      if (done) break;
      chunks.push(Buffer.from(value));
    }
  • The function takes no AbortSignal, and all three call sites clear the idle timer before calling it (commands.ts:396, pty.ts:476, filesystem.ts:284) — actively removing the only timeout guard.

Measured:

Scenario Result
5 MB text/plain error body Full 5,242,880 bytes buffered, no [truncated] marker
502 response whose body never closes Still reading after 3 s, never returns

The old resp.text() was also unbounded but at least stayed under idle.signal, so this is a net regression and contradicts the PR's own "bounded" description. Go has io.LimitReader(resp.Body, maxErrorBodyBytes) (sdk/go/errors.go:130-157, 64 KiB + " [truncated]" + rune-boundary fallback).


2. isRoutingPage is missing Go's 404 + text/* branch (Major)

Go has this branch at sdk/go/connect.go:152-160; the Node version (commands.ts:55-62) does not.

Measured: 404 + text/plain + "404 page not found" (a Go net/http.NotFound gateway — one of the most common misrouting shapes) produces no hint at all in Node, while Go reports it as a routing page.

This is also a documentation/implementation mismatch: docs/guide/troubleshooting/sdk-connect-stream-routing.md explicitly states that 404 text pages carry the routing hint.


3. Additional findings

Medium

Location Issue
commands.ts:93 reader never calls releaseLock() on any of its three return paths
commands.ts:404, filesystem.ts:288 resp.body?.cancel() is called on an already-locked stream, so it always rejects and is swallowed by .catch() — dead code
commands.ts:252-257 After readConnectFrames throws, the response stream is never cancelled. Measured: the server-side connection stays open, so a misrouted request leaks a hanging connection
commands.ts:97-104 The peek loop checks length < 4096 before reading, but a single read() can return 64 KB or more, so chunks holds far more than 4096 bytes. The "bounded peek" comment does not match the implementation
commands.ts:108/122 Server-controlled content is concatenated into Error.message without filtering C0 control characters or ANSI escapes

Test coverage gaps

The tests bypass the most dangerous paths:

  • The 100 KB case goes through the truncation branch, so the unbounded-read path has zero coverage.
  • No assertion that the stream is cancelled after readConnectFrames throws.
  • No coverage of the !resp.body early return.
  • No 404 + text/plain case (the exact gap in item 2).
  • No test for <html without a doctype, for truncateDiagnostic boundaries, or for cross-chunk multi-byte UTF-8 cuts.
  • The module-level mutable reply variable (connect-routing.test.ts:36) makes the suite fragile if in-file concurrency is ever enabled.

Suggestions (non-blocking)

  • The diagnostic logic lives in commands.ts, which makes filesystem.ts and pty.ts depend on a facade module and creates a type-only dependency cycle (sandbox → commands → (type) sandbox). Consider extracting src/connect.ts; at minimum, stop adding new symbols to commands.ts.
  • The three copies of the status >= 400 / htmlResponseError / !body template (commands.ts:395-406, filesystem.ts:282-296, pty.ts:475-489) could become a shared describeConnectFailure().
  • truncateDiagnostic truncates by code point (Array.from), while Go truncates by byte with a utf8.RuneStart fallback, so the two produce different lengths for non-ASCII text.
  • HttpResponseLike omits body, so resp.body is type-erased to any and the tsc null check is bypassed. Declare it explicitly.

…les, release streams

Second review round on the misrouted Connect stream diagnostics:

- Move the diagnostics into src/connect.ts, so filesystem.ts and pty.ts
  no longer depend on commands.ts for them; ConnectResponse declares
  body.
- Read at most 64 KiB (+1 byte) of an HTTP error body, as the Go SDK's
  readErrorMessage does, mark longer bodies ' [truncated]', give up
  after 10 s (files.watchDir has no idle timeout), and keep the idle
  timers of commands.run and PTY running during the read. The reader is
  cancelled and released afterwards.
- isRoutingPage now mirrors Go's isRoutingPage: a 404 with a non-empty,
  non-JSON text/* body (Go's '404 page not found') is a routing page, a
  JSON error document needs a non-empty message/detail/error string,
  and possibly truncated JSON is left alone. The peek is exactly 4 KiB.
- Truncate routing-page text to 200 UTF-8 bytes on a character
  boundary, as Go does, and replace control characters in server text.
- readConnectFrames cancels the body when it stops early (an error or a
  consumer that stops), so a rejected stream no longer leaves the
  connection open.
- Prefix the HTTP 200 text/html error in commands.run with
  'command failed: ' like its other errors.

Signed-off-by: Jace Xia <fxia0018@student.monash.edu>
@fxia0018

Copy link
Copy Markdown
Author

Thanks for the thorough review. Pushed a follow-up commit:

1. Bounded error-body read. streamHttpErrorDetail reads at most 64 KiB + 1 byte (Go's maxErrorBodyBytes), adds [truncated] after that, and cancels and releases the reader afterwards. commands.run and the PTY now clear their idle timers after the read, so the read stays under idle.signal. files.watchDir has no idle timeout, so the read itself also gives up after 10 s. Tests: an endless 502 body is truncated and the server sees the connection closed for all four callers, and a 502 that stalls after the headers ends within the commands.run idle timeout.

2. 404 + text/*. isRoutingPage(status, type, body) now mirrors Go's: the 404 text branch, looksLikeJSONErrorDocument (valid JSON with a non-empty message/detail/error string), and the truncated-JSON guard. 404 page not found now gets the hint, and a table test checks the classification against Go's cases.

Additional findings:

  • releaseLock(): called on every path in the new reader.
  • Leaked connection after readConnectFrames throws: its finally now cancels the body before releasing the lock, which also covers consumers that stop early. A test checks that the server sees the close for a rejected non-envelope stream and for the 200 text/html path of run / watch / create.
  • The peek is now exactly 4 KiB (raw.subarray(0, 4096)).
  • Control characters / ANSI escapes in server text are replaced with U+FFFD.
  • Tests also cover the !resp.body return, <html without a doctype, truncateDiagnostic boundaries, and multi-byte cuts.

Suggestions: the diagnostics now live in src/connect.ts, so commands.ts no longer gains symbols and filesystem.ts / pty.ts import from there. ConnectResponse declares body. truncateDiagnostic cuts by UTF-8 bytes like Go. I kept the three call sites as they are, since each one has its own error type and timer handling. I can fold them into a describeConnectFailure() if you prefer.

The tests share one server with a module-level reply. That is fine because vitest runs the tests of a file one after another, and I left it as is.

npx vitest run: 363 passed, 2 skipped. typecheck and build pass.

Comment thread sdk/node/src/connect.ts
Comment on lines +131 to +159
async function readErrorBody(body: ReadableStream<Uint8Array>): Promise<Buffer | null> {
const reader = body.getReader();
const chunks: Buffer[] = [];
let length = 0;
let failed = false;
const timer = setTimeout(() => {
void reader.cancel().catch(() => undefined);
}, ERROR_BODY_READ_TIMEOUT_MS);
(timer as { unref?: () => void }).unref?.();
try {
while (length <= MAX_ERROR_BODY_BYTES) {
const { done, value } = await reader.read();
if (done) {
break;
}
chunks.push(Buffer.from(value));
length += value.length;
}
} catch {
failed = true;
} finally {
clearTimeout(timer);
await reader.cancel().catch(() => undefined);
reader.releaseLock();
}
if (failed && length === 0) {
return null;
}
return Buffer.concat(chunks).subarray(0, MAX_ERROR_BODY_BYTES + 1);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit (non-blocking): the documented bound is not enforced on the read, and partial reads are reported as complete.

  • while (length <= MAX_ERROR_BODY_BYTES) pushes whole chunks, so a single chunk larger than the remaining budget buffers more than MAX_ERROR_BODY_BYTES + 1 bytes before subarray(0, MAX + 1) clips the result. The doc comment says "Read at most MAX_ERROR_BODY_BYTES + 1 bytes"; clipping value to the remaining budget would make that true. (Harmless with undici's typical chunk sizes.)
  • When reader.read() throws, or the 10 s timer cancels the reader mid-body, the partial bytes are returned without a [truncated] marker — only the >64 KiB path appends one. A cut-off error page then looks like a complete message. Go's readErrorMessage returns "" on a read error. Consider returning null (status-only message) or marking partial reads.

Neither affects normal responses; only the diagnostic text on stalled/aborted error bodies.

@liciazhu

Copy link
Copy Markdown
Collaborator

Thanks for the fix @fxia0018 . Two items remain.

1. sanitize (connect.ts:122-124) misses C1 and mangles \r

text.replace(/[\x00-\x08\x0b-\x1f\x7f]/g, "\uFFFD")

Measured:

  • "line1\r\nline2" → line1\uFFFD\nline2 — \r sits inside \x0b-\x1f, so a CRLF body (common from gateways) renders as line1� plus a newline.
  • "\x1b[31m" (C0 ESC) → �[31m — handled correctly.
  • "\u009b31m" (C1 CSI) survives untouched (code point check: 15 chars in, 15 out) — C1 escapes are not covered, so the terminal-injection surface isn't fully closed.

Also visible end-to-end through commands.run, not just at the function level:

command failed: HTTP 500: line1�\nline2
command failed: HTTP 502: <html>�\n<body>oops</body>�\n</html>; received an HTML error page ...
command failed: HTTP 404: {"message":"a�\nb"}

Suggest normalising line endings first, then widening the class:

const normalized = text.replace(/\r\n?/g, "\n");
return normalized.replace(/[\x00-\x08\x0b-\x1f\x7f\u0080-\u009f]/g, "\uFFFD");

2. No documentation change

Node now honours the "404 text pages" promise at docs/guide/troubleshooting/sdk-connect-stream-routing.md:41, but that page is still titled "Go SDK Connect Stream Returns an HTML or Text Response" and names only Commands.Run / Files.WatchDir / Pty.Create. Node users can't find it, and three user-visible behaviours are undocumented: message shapes (command failed: …, WatchDir failed: …, pty.create failed: …), the 200-byte / 64 KiB caps, and the runCode / readFile / writeFile gaps Go already lists.

…ment Node

sanitize now turns CRLF/CR into a line break before replacing control
characters, and also replaces C1 controls such as the 8-bit CSI. The
troubleshooting page covers the Node SDK: affected calls, message
prefixes, the 200-byte / 64 KiB caps, and the runCode / files.read /
files.write gaps. The page wording follows the Python SDK PR (TencentCloud#2336).

Signed-off-by: Jace Xia <236735305+fxia0018@users.noreply.github.com>
@fxia0018
fxia0018 requested a review from fslongjin as a code owner November 20, 2026 11:03
@fxia0018

fxia0018 commented Nov 20, 2026 •

Copy link
Copy Markdown
Author

Thanks. Pushed c2dc6ca:

  1. sanitize: CRLF and CR become \n first, then C0 (except \t and \n), DEL and C1 (\x80-\x9f, including the 8-bit CSI) become U+FFFD. A test checks a body with CRLF, a bare CR, an 8-bit CSI (U+009B) and NEL (U+0085) through all four callers.
  2. Docs: the troubleshooting page now covers the Node SDK: the affected calls (commands.run, files.watchDir, pty.create, pty.connect), the message prefixes (command failed:, WatchDir failed:, Start failed: / Connect failed:), the 200-byte / 64 KiB caps, and the runCode / files.read / files.write gaps, in English and Chinese. I used the same title and wording as the Python SDK PR (fix(sdk/python): diagnose misrouted HTML responses on Connect streams #2336), which also generalizes this page, so whichever merges second needs only a small rebase there.

npx vitest run: 367 passed, 2 skipped. tsc and build pass.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants