Repository navigation
feat(provider): 支持单个模型商配置多个 API Key 做负载均衡 (closes #104) - #113
Merged
Merged
Conversation
一个供应商可填写多个 Key，请求在它们之间轮询分摊，并自动 避开失效(401/403)或被限流(429/5xx)的 Key。 - store.Provider 新增 APIKeys 字段与 EffectiveKeys 合并方法 - relay 新增负载均衡器：round-robin 选 Key + 失败冷却 - 转发/探测路径接入多 Key；管理 API 与 dashboard 表单同步支持 - 向后兼容：单 Key 仍走 APIKey 字段，配置/导出均可读 Co-Authored-By: WorkBuddy <workbuddy@tencent.com>
LAGcomcom
added a commit
that referenced
this pull request
Oct 23, 2026
Lands the ideas reviewed from #113-#116 as our own implementations, with tests written against the failure shapes rather than the patches: - usage: streaming usage snapshots fold per-field max instead of summation, which compounded repeated terminal frames up to 1400x (issue #105). The relay path shares the aggregation, and OpenAI's nested prompt_tokens_details.cached_tokens is now read (flat keys still win). - hidden models (issue #103): user-unchecked ids stop consuming probe rounds and stop acting as failover/throttle-switch candidates while staying directly callable; the set is seeded on boot and re-synced on visibility toggles and imports. An all-hidden round no longer reads as a quota wall. - dead egress: a tunnel-level failure (transport/timeout/reset) now books the node, not just the (exit, model) pair — two strikes inside the fail-memory window retire it from rotation until the dead-node probe answers; a raced success still resurrects it, and a probe revival clears the strike counter. - providers accept several API keys with per-request round-robin and classed cooldowns (credential 10m, 429 retry-after/10s, 5xx 15s; issue #104). Keys are write-only from the panel, masked in state and exports, and the single-key field stays the storage shape for one-key providers.
Owner
|
感谢贡献！多 Key 负载均衡确实是刚需。已按这个思路实现并合入 7eb328b：providers 支持多 Key 按请求轮换，失败按类别冷却（凭据拒绝 10 分钟、429 按 Retry-After、5xx 15 秒），面板支持多行输入、状态与导出均做掩码处理，单 Key 的存储形态保持不变以兼容旧版本。欢迎使用并反馈！ |
ghshhf
pushed a commit
to ghshhf/zen-gate
that referenced
this pull request
Nov 3, 2026
上游把我们的 LAGcomcom#113-LAGcomcom#116、probe-verdicts 与 LAGcomcom#79 全部 revert 后按自己思路重写 (7eb328b / 0ae4c0b / 97eb4f3)。核对结果是重写版为我们需求的重写超集： usage 累加、hidden 探测、dead-egress、多 key 轮询、404 误判同样都修了， 另外还覆盖嵌套 cached_tokens、Kilo 判定过期、CodeArts 400、探测进度 (LAGcomcom#117)、渠道墙、前缀缓存、ZCode/TRAE/Ext 账号池、面板重构、macOS sing-box 资产选择、无头编译期隔离。 冲突处理：17 个冲突文件以上游为基线。依据是自研版在这些点上均被超集覆盖， 保留自研实现等于丢功能，而非"优先自己"能保住的东西——真要自主，从这次 合并之后的新功能开始（后续提交见）。 顺带清理：删除 internal/lane/hidden_test.go 与 usage_test.go，它们测的是 旧 []string 语义；上游 hidden_skip_test.go / http_usage_max_test.go 已等价 且更完整地覆盖同一行为。 验证：go build ./... 通过；go vet ./... 通过；lane/gateway/relay/subs/store 五个包测试全绿。
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
解决的问题
issue #104：当前每个模型商只能填一个 API Key，要多 key 只能重复添加同一模型商，还不能做负载均衡。本 PR 让单个模型商可填多个 Key，请求时在它们之间做负载均衡 + 失败冷却。
改动
store.Provider新增APIKeys []string，并提供EffectiveKeys()合并去重（旧字段APIKey仍兼容）。internal/relay/balancer.go：轮询选 key，按失败类型冷却——凭据错(401/403) 10 分钟、限流(429) 10 秒、5xx 15 秒；全部冷却时取最早到期者，避免彻底饿死。relay.postStreamed用选中的 key 发请求，出错按lane.ClassifyFailure的 code 冷却对应 key。providerIn支持apiKeys，保存时归一化（合并后只有 1 个 → 写回APIKey，多个 → 写APIKeys）；列表视图返回keyCount。web/index.html（已 go:embed 进二进制）：把单个密码框换成多行文本框，按行拆分多个 Key，卡片显示「N 个 Key 负载均衡」。兼容性
APIKey）完全保留；EffectiveKeys()会把旧字段并入。firstKey()，单 key 路径行为不变。测试
新增
balancer_test.go/lb_e2e_test.go(TestCompleteLoadBalancesKeys) /provider_keys_test.go，覆盖轮询、冷却、去重、归一化保存；go build ./...与 store/relay/gateway/subs/lane 全部测试通过。Closes #104