高级检索

    星汉:面向AI智能体的内核域名感知透明路由系统

    ASTRA: An In-Kernel Domain-Aware Transparent Routing System for AI Agents

    • 摘要: 随着大语言模型驱动的AI智能体技术的快速发展,智能体通过工具调用机制在任务执行过程中动态访问外部服务已成为主流范式。在多租户智能体平台中,每个智能体沙箱往往需要同时访问大量异构外部服务,如大模型推理API、网页抓取API、代码仓库API及金融数据API等。这些API往往需要经由不同的网络出口才能可达。传统方案中,隧道方案需要为每个服务分别建立通道,维护成本随服务数量急剧增长;基于IP地址的策略路由方案则因API接入点IP地址的动态变化而难以维护;用户态代理(如HAProxy,Envoy)存在上下文切换开销且需要侵入式配置。针对上述问题,提出星汉系统,一种透明、高性能的按名路由系统。星汉系统利用TLS握手中已有的SNI(server name indication)扩展字段作为服务标识,无需修改应用或基础设施即可实现与现有生态兼容的透明路由;同时将基于服务域名的路由决策完全置于Linux 内核中执行,消除用户态代理的上下文切换开销,实现高性能转发。星汉系统包含3项关键技术:1)跨层按名路由方案,以TLS服务名和TCP端口的组合作为服务标识符;2)4路TCP-TLS联合握手代理模型,在内核态透明地拦截连接并提取服务名;3)延迟内核态名字解析机制,基于定制的基数树实现高效的名字-地址转换。实验结果表明,在典型API负载大小下,星汉系统的请求速率超过HAProxy 27.16%、超过Envoy 50.74%,P50延迟低于HAProxy 17.68%、低于Envoy 43.60%。星汉系统在名字规则从1 000增长到50 000时保持几乎恒定的查找延迟,随名字规则数量的增长保持良好的可扩展性。除此之外,在CPU资源利用效率上,星汉系统以每核心支持的请求速率衡量的CPU资源效率达到HAProxy 的1.91倍、Envoy 的1.68倍。

       

      Abstract: With the rapid advancement of large language model (LLM)-driven AI agents, tool use has become the dominant paradigm through which agents dynamically interact with external services during task execution. In multi-tenant agent platforms, each agent sandbox often needs to access a wide range of heterogeneous external services concurrently, including LLM inference APIs, web scraping APIs, code space APIs, and financial data APIs. These APIs often require distinct network egress paths. Existing solutions each have significant limitations: tunnel-based approaches require establishing a separate channel per service, leading to maintenance costs that grow sharply with the number of services; IP-based policy routing struggles with the dynamically changing endpoints of API providers; and user-space proxies such as HAProxy and Envoy incur context-switching overhead and require intrusive configuration. To address these challenges, this paper proposes ASTRA, a transparent, high-performance name-based routing system. ASTRA leverages the existing TLS Server Name Indication (SNI) extension as a service identifier, enabling transparent routing compatible with the existing ecosystem without requiring modifications to applications or infrastructure. Meanwhile, it implements domain-name-based routing decisions entirely within the Linux kernel, preventing the context-switching overhead of user-space proxies to achieve high-performance forwarding. ASTRA incorporates three key techniques: (1) a cross-layer name-based routing scheme using the combination of TLS server name and TCP port as the service identifier; (2) a 4-way TCP-TLS joint handshake proxy model that transparently intercepts connections and extracts service names in kernel space; and (3) a deferred in-kernel name resolution mechanism based on a custom radix tree for efficient name-to-address translation. Experimental results show that, under typical API payload sizes, ASTRA outperforms HAProxy and Envoy in request rate by 27.16% and 50.74%, and in P50 latency by 17.68% and 43.60%, respectively. ASTRA maintains near-constant lookup latency as name rules scale from 1 000 to 50 000, showing scalability as the number of name rules grows. In terms of CPU resource efficiency measured by request rate per core, ASTRA achieves 1.91× the HAProxy baseline and 1.68× the Envoy baseline.

       

    /

    返回文章
    返回