What inference engine and KV offload engine do you use and why?
I have the question to the guys in inference world, I am building some stuff connected to the inference in my work, and I started wondering, what do actually big guys use in their inference work? Do you prefer standard …
以下正文同步自 Hacker News 最新，版权归原站所有，已转换为易读排版。
I have the question to the guys in inference world, I am building some stuff connected to the inference in my work, and I started wondering, what do actually big guys use in their inference work? Do you prefer standard (and what is standard lol) solutions like vLLM and LMCache, or you additionally add Mooncake, or use vllm native? Happy to have a discussion what is better and why as well, thanks! Comments URL: https://news.ycombinator.com/item?id=50302109 Points: 1
Comments: 0
正文由 探How 从来源站点 RSS 同步，内容未经改写；遇到排版缺失或需要图片、视频时请以原文为准。