llm-d Router
An intelligent inference router for LLM traffic with load-aware and cache-aware routing.
The Linkredibles take
Why it's worth knowing.
Optimize inference traffic using model, load, cache, and scheduling signals.
Topics
#llm#inference#routing#kubernetes#gpu#performance
Open source repository
Explore the source behind llm-d Router.
Read the source, explore the project, and see how it is being developed on GitHub.
Keep exploring
You might also like.
ai→
MiroFish
A Simple and Universal Swarm Intelligence Engine, Predicting Anything. 简洁通用的群体智能引擎，预测万物
★ 74,570Python
automation→
Activepieces
AI Agents & MCPs & AI Workflow Automation • (~400 MCP servers for AI agents) • AI Automation / AI Agent with MCPs • AI Workflows & AI Agents • MCPs for AI Agents
★ 24,726TypeScript
ai agent-skills→
Addy Osmani Agent Skills
Production-grade engineering skills for AI coding agents covering planning, implementation, testing, review, simplification, and shipping.
★ 98,956JavaScript