Repository navigation
Conversation
When a data-plane request is misrouted (for example CUBE_PROXY_NODE_IP
unset or pointing at the Web UI), the server returns an HTML page with
HTTP 200. readConnectFrames read "<!DOC" as an envelope header and
commands.run, files.watchDir and pty.create failed with
Connect stream message too large: 558124867 bytes
TencentCloud#1823 fixed the same diagnosis in the Go SDK. Port its checks:
- htmlResponseError(): a 200 text/html response is rejected before the
stream is read, with the CubeProxy routing hint;
- readConnectFrames() rejects flag bits other than compressed and
end-stream before interpreting the length, so HTML, JSON or gzip
bodies without an HTML content type report "response is not a
Connect envelope" with the first five bytes;
- streamHttpErrorDetail(): a non-200 HTML page gets the Go SDK's
status-specific routing hint and is reported from a bounded,
truncated prefix of the body. A JSON error document keeps the
server's message.
Valid Connect streams are unaffected.
Signed-off-by: Jace Xia <fxia0018@student.monash.edu>
| const hint = routingPageHint(resp.status); | ||
| return text ? `${truncateDiagnostic(text)}; ${hint}` : hint; | ||
| } | ||
| for (;;) { |
There was a problem hiding this comment.
Unbounded error-body read for non-routing bodies. For a body that is not a routing page this loop reads the response to EOF and keeps every chunk in chunks, then embeds the full text in the thrown message. The Go SDK caps HTTP error diagnostics at 64 KiB (maxErrorBodyBytes / " [truncated]" in sdk/go/errors.go), and this SDK's control-plane helper caps non-JSON bodies at 500 chars (errorMessageFromResponse in sandbox.ts).
commands.run and pty.openStream already did resp.text() on this path (and both clear their idle timer before calling this helper, so it is not bounded by it either). filesystem.watchDir is the new case: it previously aborted immediately without reading the error body, has no idle/body timeout, and now controller.abort() runs only after this read. A misrouted endpoint that returns a very large — or never-ending — non-HTML error body therefore makes watchDir() buffer and block before throwing. Consider bounding the total buffered prefix (e.g. 64 KiB) and appending a truncation marker, matching the Go behaviour.
| * Whether an HTTP error body looks like a Web UI or gateway page. A JSON | ||
| * error document keeps the server's message even when labelled text/html. | ||
| */ | ||
| function isRoutingPage(type: string, body: Buffer): boolean { |
There was a problem hiding this comment.
Routing-page detection diverges from the Go isRoutingPage this PR mirrors (sdk/go/connect.go):
- Any body starting with
{is accepted as a genuine JSON error document without validating it. Go'slooksLikeJSONErrorDocumentrequires valid JSON with a non-emptymessage/detail/error; otherwise atext/htmlbody that merely starts with{counts as a routing page. Here such a response gets no hint. - Go also treats a 404 with a non-empty, non-JSON
text/*body (e.g. Go's net/http404 page not found) as a routing page. Heretext/plainnever matches, so that common gateway page still produces a bareHTTP 404: 404 page not foundwith noCUBE_PROXY_NODE_IPhint.
If parity with #1823 is intended, mirroring both (plus a 404 text/plain test) would close the gap; otherwise it is worth documenting the narrower HTML-only scope.
Automated review — #2344
|
|
Thanks for the fix @fxia0018 . Two issues must be fixed before merge. 1. Error-body reads regress from "bounded by timeout" to "unbounded, no timeout" (Major)
Measured:
The old 2.
|
| Location | Issue |
|---|---|
commands.ts:93 |
reader never calls releaseLock() on any of its three return paths |
commands.ts:404, filesystem.ts:288 |
resp.body?.cancel() is called on an already-locked stream, so it always rejects and is swallowed by .catch() — dead code |
commands.ts:252-257 |
After readConnectFrames throws, the response stream is never cancelled. Measured: the server-side connection stays open, so a misrouted request leaks a hanging connection |
commands.ts:97-104 |
The peek loop checks length < 4096 before reading, but a single read() can return 64 KB or more, so chunks holds far more than 4096 bytes. The "bounded peek" comment does not match the implementation |
commands.ts:108/122 |
Server-controlled content is concatenated into Error.message without filtering C0 control characters or ANSI escapes |
Test coverage gaps
The tests bypass the most dangerous paths:
- The 100 KB case goes through the truncation branch, so the unbounded-read path has zero coverage.
- No assertion that the stream is cancelled after
readConnectFramesthrows. - No coverage of the
!resp.bodyearly return. - No
404 + text/plaincase (the exact gap in item 2). - No test for
<htmlwithout a doctype, fortruncateDiagnosticboundaries, or for cross-chunk multi-byte UTF-8 cuts. - The module-level mutable
replyvariable (connect-routing.test.ts:36) makes the suite fragile if in-file concurrency is ever enabled.
Suggestions (non-blocking)
- The diagnostic logic lives in
commands.ts, which makesfilesystem.tsandpty.tsdepend on a facade module and creates a type-only dependency cycle (sandbox → commands → (type) sandbox). Consider extractingsrc/connect.ts; at minimum, stop adding new symbols tocommands.ts. - The three copies of the
status >= 400/htmlResponseError/!bodytemplate (commands.ts:395-406,filesystem.ts:282-296,pty.ts:475-489) could become a shareddescribeConnectFailure(). truncateDiagnostictruncates by code point (Array.from), while Go truncates by byte with autf8.RuneStartfallback, so the two produce different lengths for non-ASCII text.HttpResponseLikeomitsbody, soresp.bodyis type-erased toanyand thetscnull check is bypassed. Declare it explicitly.
…les, release streams Second review round on the misrouted Connect stream diagnostics: - Move the diagnostics into src/connect.ts, so filesystem.ts and pty.ts no longer depend on commands.ts for them; ConnectResponse declares body. - Read at most 64 KiB (+1 byte) of an HTTP error body, as the Go SDK's readErrorMessage does, mark longer bodies ' [truncated]', give up after 10 s (files.watchDir has no idle timeout), and keep the idle timers of commands.run and PTY running during the read. The reader is cancelled and released afterwards. - isRoutingPage now mirrors Go's isRoutingPage: a 404 with a non-empty, non-JSON text/* body (Go's '404 page not found') is a routing page, a JSON error document needs a non-empty message/detail/error string, and possibly truncated JSON is left alone. The peek is exactly 4 KiB. - Truncate routing-page text to 200 UTF-8 bytes on a character boundary, as Go does, and replace control characters in server text. - readConnectFrames cancels the body when it stops early (an error or a consumer that stops), so a rejected stream no longer leaves the connection open. - Prefix the HTTP 200 text/html error in commands.run with 'command failed: ' like its other errors. Signed-off-by: Jace Xia <fxia0018@student.monash.edu>
|
Thanks for the thorough review. Pushed a follow-up commit: 1. Bounded error-body read. 2. 404 + Additional findings:
Suggestions: the diagnostics now live in The tests share one server with a module-level
|
| async function readErrorBody(body: ReadableStream<Uint8Array>): Promise<Buffer | null> { | ||
| const reader = body.getReader(); | ||
| const chunks: Buffer[] = []; | ||
| let length = 0; | ||
| let failed = false; | ||
| const timer = setTimeout(() => { | ||
| void reader.cancel().catch(() => undefined); | ||
| }, ERROR_BODY_READ_TIMEOUT_MS); | ||
| (timer as { unref?: () => void }).unref?.(); | ||
| try { | ||
| while (length <= MAX_ERROR_BODY_BYTES) { | ||
| const { done, value } = await reader.read(); | ||
| if (done) { | ||
| break; | ||
| } | ||
| chunks.push(Buffer.from(value)); | ||
| length += value.length; | ||
| } | ||
| } catch { | ||
| failed = true; | ||
| } finally { | ||
| clearTimeout(timer); | ||
| await reader.cancel().catch(() => undefined); | ||
| reader.releaseLock(); | ||
| } | ||
| if (failed && length === 0) { | ||
| return null; | ||
| } | ||
| return Buffer.concat(chunks).subarray(0, MAX_ERROR_BODY_BYTES + 1); |
There was a problem hiding this comment.
nit (non-blocking): the documented bound is not enforced on the read, and partial reads are reported as complete.
while (length <= MAX_ERROR_BODY_BYTES)pushes whole chunks, so a single chunk larger than the remaining budget buffers more thanMAX_ERROR_BODY_BYTES + 1bytes beforesubarray(0, MAX + 1)clips the result. The doc comment says "Read at mostMAX_ERROR_BODY_BYTES + 1bytes"; clippingvalueto the remaining budget would make that true. (Harmless with undici's typical chunk sizes.)- When
reader.read()throws, or the 10 s timer cancels the reader mid-body, the partial bytes are returned without a[truncated]marker — only the >64 KiB path appends one. A cut-off error page then looks like a complete message. Go'sreadErrorMessagereturns""on a read error. Consider returningnull(status-only message) or marking partial reads.
Neither affects normal responses; only the diagnostic text on stalled/aborted error bodies.
|
Thanks for the fix @fxia0018 . Two items remain. 1. text.replace(/[\x00-\x08\x0b-\x1f\x7f]/g, "\uFFFD")Measured:
Also visible end-to-end through Suggest normalising line endings first, then widening the class: const normalized = text.replace(/\r\n?/g, "\n");
return normalized.replace(/[\x00-\x08\x0b-\x1f\x7f\u0080-\u009f]/g, "\uFFFD");2. No documentation change Node now honours the "404 text pages" promise at |
…ment Node sanitize now turns CRLF/CR into a line break before replacing control characters, and also replaces C1 controls such as the 8-bit CSI. The troubleshooting page covers the Node SDK: affected calls, message prefixes, the 200-byte / 64 KiB caps, and the runCode / files.read / files.write gaps. The page wording follows the Python SDK PR (TencentCloud#2336). Signed-off-by: Jace Xia <236735305+fxia0018@users.noreply.github.com>
|
Thanks. Pushed c2dc6ca:
|
Summary
Node.js counterpart of #2336 (Python) and #1823 (Go). It already includes the changes requested in the review of #2336.
When a data-plane request is misrouted (for example
CUBE_PROXY_NODE_IPis not set, or the endpoint points at the Web UI), the server returns an HTML page. With HTTP 200,readConnectFramesreads<!DOCas an envelope header (flags0x3c, length558124867), andcommands.run,files.watchDirandpty.createfail with:With a 404/502 HTML page, the error is the raw page with no hint about routing.
Change (
src/commands.ts, used bycommands.ts,filesystem.ts,pty.ts)htmlResponseError(): an HTTP 200text/htmlresponse is rejected before the stream is read, with the Go SDK's message and hint.readConnectFrames(): flag bits other than compressed (0x01) / end-stream (0x02) are rejected before the length is read, so an HTML, JSON or gzip body without an HTML content type reportsresponse is not a Connect envelope (first 5 bytes: 3c21444f43); ....streamHttpErrorDetail(): on HTTP ≥ 400, a body that istext/htmlor starts with<!doctype/<htmlis reported from a bounded (4 KiB) prefix, truncated to 200 characters, followed by the same status-specific hint as the Go SDK'svalidateConnectResponse(401/403, 5xx, other). A JSON error document keeps the server's message even when labelledtext/html; other bodies are reported as before.ROUTING_HINTandNOT_AN_ENVELOPE_HINT.Error classes are unchanged (
Errorfor commands/watch,CubeSandboxError/ApiErrorfor PTY).files.watchDirnow includes the response body in its HTTP error, like the other two callers.Tests
New
test/connect-routing.test.ts(52 cases, local HTTP server, no network):commands.run(stdout + stderr + exit code) andpty.create/pty.connectstreams, also written 3 bytes at a time so envelope headers span chunks;readConnectFramesrejects HTML, BOM, leading newline, JSON and gzip prefixes, and accepts flags0x00–0x03;text/html, and a plain-text error (no hint).On
master, 36 of the 52 fail. Full suite:npx vitest run→ 333 passed, 2 skipped.npm run typecheckandnpm run buildpass.The troubleshooting page is updated in #2336; once that lands I can add the Node.js method names there too.