llm-d is a new open source project focused on providing distributed inferencing for Generative AI runtimes on any Kubernetes cluster. Its architecture is designed for high performance and scalability, aiming to reduce costs through a spectrum of hardware and software efficiency improvements. llm-d prioritizes ease of deployment and use, as well as the operational needs of running large GPU clusters, including SRE concerns and day 2 operations. At launch, its key features include prefill/decode disaggregation, KV cache distribution and management, an AI-aware router with customizable scoring, operational telemetry, Kubernetes-based deployment, and the NIXL inference optimized transfer library.
Company Details
- Founded
- -
- Address
- Hours Today (7/31) At 2pm Est. Https://t.co/x8hahyr3rl
- Industry
- Software Development
- NAICS
-
Software Publishers
- Website
- https://llm-d.ai/
Llm-D Questions
Llm-D's website is https://llm-d.ai/
Llm-D's LinkedIn profile is https://www.linkedin.com/company/llm-d
Llm-D's industry is
Software Development
Llm-D's top competitors are
Softbank Research Institute Of Advanced Technology,
Solo.io,
Monk.io,
Red Hat,
Starfish,
Fairbanks,
Decentral Plays,
Aci Worldwide,
Lmcache Lab,
Vllm.
Llm-D's categories are Software Development
Llm-D's founding year is 2025
