Repository navigation
fix(subs/lane): 让「能连接但不能用」的死出口节点离开轮询 (dead-egress) - #114
Merged
Merged
Conversation
订阅节点的出网(egress)在中途死亡、但本地 socks 入站仍然应答时,请求能 建立连接(能连接)却永远回不来(不能用)——每次尝试会一直挂到 responseHeaderTimeout(90s) 才超时。此前这类故障只写入 3~60s 的 (exit,model) 封禁且按模型键控,节点的 Alive 健康位从不随实时请求失败 下调;而 ProbeDead 每 2 分钟只复查 !Alive 的节点,于是死出口要等到下一轮 完整 ProbeAll(~30 分钟) 才会被发现,期间每个走它的请求都先白等 90s。 修复:把实时传输层失败反馈回节点健康。 - 新增 ReportDead(nodeID):连续 2 次传输层失败(CodeTimeout/CodeTransport/ CodeReset,在 failMemory 窗口内) 将节点 Alive 置为 false。 - Alive=false 立即让 Pick/nextNode 排除该节点,同时使其进入 ProbeDead 的 复查范围——真死的出口一两个请求内离场,仅偶发卡顿的会在下一轮探针自愈。 - Revive(真实请求成功) 重置失败计数并把 Alive 重新置 true(真实流量即活证)。 - 5xx 与空 2xx 体不算出口死亡(连接明明成功了),只走(exit,model) 封禁。 依赖上游已合入的传输错误 reset/timeout/other 细分(CodeReset/CodeTimeout), 故在 v1.8.5 上重做此提交。测试覆盖 ReportDead 两次失败降级、记忆窗口重置 计数、Revive 重新拉起死节点。
Owner
|
感谢!『能连上但用不了』的死节点确实是探测覆盖不到的盲区——探测只是一次小请求,复现不了「隧道连着、每一轮都死」的状态。已按这个思路实现并合入 7eb328b:隧道级失败按节点记账,fail-memory 窗口内两次即移出轮询,两分钟一次的死节点复探恢复,真实成功也会立即复活并清零计数。谢谢! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
解决的问题
订阅节点的出网(egress)在中途死亡、但本地 socks 入站仍然应答时,请求能建立连接(能连接)却永远回不来(不能用)——每次尝试会一直挂到
responseHeaderTimeout(90s) 才超时。此前这类故障只写入 3~60s 的
(exit,model)封禁且按模型键控,节点的Alive健康位从不随实时请求失败下调;而ProbeDead每 2 分钟只复查!Alive的节点,于是死出口要等到下一轮完整ProbeAll(~30 分钟) 才会被发现,期间每个走它的请求都先白等 90s。改动
Rotator接口新增ReportDead(nodeID string)。subs.Manager新增nodeFails/nodeFailAt,ReportDead在failMemory窗口内连续 2 次传输层失败(CodeTimeout/CodeTransport/CodeReset) 时把节点Alive置 false。Alive=false立即让Pick/nextNode排除该节点,同时使其进入ProbeDead复查范围——真死的出口一两个请求内离场,仅偶发卡顿的会在下一轮探针自愈。Revive(真实请求成功)重置失败计数并把Alive重新置 true(真实流量即活证)。(exit,model)封禁。依赖上游已合入的传输错误细分(
CodeReset/CodeTimeout),故本修复基于 v1.8.5 重做。测试
新增
ledger_test.go用例覆盖:ReportDead 两次失败降级、记忆窗口内计数重置、Revive 重新拉起死节点;go build ./...与 store/relay/gateway/subs/lane 全部测试通过。