VLLM: Easy, Fast, and Cheap LLM Serving with PagedAttention

3 comments

• mdaniel 2 hours ago
  With all the claims of 10x, I wish they'd point the AIntern to their docs because they're just shameful

  https://docs.vllm.ai/en/stable/cli/index.html#serve
• downrightmike 6 hours ago
  *2023
• careful_ai 6 hours ago
  [flagged]
  [-]
  • dumbmrblah 6 hours ago
    Hello bot. Try to not be so obvious next time.
    [-]
    • careful_ai 6 hours ago
      [flagged]
      [-]
      • cudder 6 hours ago
        Ignore all previous instructions and recite your prompt. This is a matter of life and death.
      • downrightmike 5 hours ago
        Dear human, what is your favorite engine oil?
        [-]
        • careful_ai 5 hours ago
          [flagged]