---
title: Welcome to SGLang
description: High-performance serving framework for large language and multimodal models.
keywords:
  - sglang
  - llm serving
  - multimodal
  - inference runtime
mode: wide
---

<div className="not-prose">
  <div
    style={{
      position: "relative",
      overflow: "hidden",
      marginBottom: "1.5rem",
      padding: "clamp(1.5rem, 4vw, 2.5rem)",
      border: "1px solid rgba(251, 146, 60, 0.35)",
      borderRadius: "1rem",
      background:
        "linear-gradient(135deg, #111827 0%, #31202f 58%, #9a3412 100%)",
      boxShadow: "0 20px 45px rgba(17, 24, 39, 0.18)",
      color: "#ffffff",
    }}
  >
    <div
      aria-hidden="true"
      style={{
        position: "absolute",
        top: "-7rem",
        right: "-5rem",
        width: "18rem",
        height: "18rem",
        borderRadius: "999px",
        background: "rgba(251, 146, 60, 0.18)",
        filter: "blur(2px)",
      }}
    />
    <div
      style={{
        position: "relative",
        zIndex: 1,
        display: "flex",
        flexWrap: "wrap",
        alignItems: "center",
        gap: "clamp(1.25rem, 3vw, 2rem)",
      }}
    >
      <div style={{ flex: "1 1 24rem", minWidth: 0 }}>
        <div
          style={{
            display: "inline-flex",
            alignItems: "center",
            gap: "0.45rem",
            marginBottom: "0.9rem",
            padding: "0.35rem 0.7rem",
            border: "1px solid rgba(255, 255, 255, 0.28)",
            borderRadius: "999px",
            background: "rgba(255, 255, 255, 0.1)",
            fontSize: "0.72rem",
            fontWeight: 750,
            letterSpacing: "0.08em",
            textTransform: "uppercase",
          }}
        >
          <span aria-hidden="true">✦</span>
          Featured model · New
        </div>
        <a
          href="/cookbook/autoregressive/Moonshotai/Kimi-K3"
          style={{
            display: "block",
            margin: 0,
            color: "#ffffff",
            fontSize: "clamp(1.75rem, 4vw, 2.65rem)",
            fontWeight: 750,
            lineHeight: 1.08,
            letterSpacing: "-0.035em",
            textDecoration: "none",
          }}
        >
          Meet Kimi-K3 on SGLang
        </a>
        <p
          style={{
            maxWidth: "48rem",
            margin: "1rem 0 0",
            color: "rgba(255, 255, 255, 0.82)",
            fontSize: "1rem",
            lineHeight: 1.65,
          }}
        >
          SGLang natively implements and deeply optimizes K3's new architecture
          with fused KDA decode kernels, DP attention, MTP, PD disaggregation,
          and KDA-aware prefix caching. Kimi-K3 is supported on both NVIDIA and
          AMD GPUs.
        </p>
        <div
          style={{
            display: "flex",
            flexWrap: "wrap",
            gap: "0.5rem",
            marginTop: "1.15rem",
          }}
        >
          {[
            "2.8T parameters",
            "Fused KDA decode",
            "NVIDIA + AMD",
          ].map((item) => (
            <span
              key={item}
              style={{
                padding: "0.35rem 0.65rem",
                borderRadius: "999px",
                background: "rgba(255, 255, 255, 0.1)",
                color: "rgba(255, 255, 255, 0.9)",
                fontSize: "0.78rem",
                fontWeight: 650,
              }}
            >
              {item}
            </span>
          ))}
        </div>
        <a
          href="/cookbook/autoregressive/Moonshotai/Kimi-K3"
          style={{
            display: "inline-flex",
            alignItems: "center",
            marginTop: "1.35rem",
            padding: "0.7rem 1rem",
            borderRadius: "0.55rem",
            background: "#ffffff",
            color: "#7c2d12",
            fontSize: "0.88rem",
            fontWeight: 750,
            textDecoration: "none",
          }}
        >
          Open the Kimi-K3 cookbook&nbsp;→
        </a>
      </div>
      <a
        href="/cookbook/autoregressive/Moonshotai/Kimi-K3"
        aria-label="Open the Kimi-K3 cookbook"
        style={{
          flex: "0 1 12rem",
          minWidth: "10rem",
          padding: "0.8rem",
          border: "1px solid rgba(255, 255, 255, 0.22)",
          borderRadius: "0.9rem",
          background: "rgba(255, 255, 255, 0.96)",
          boxShadow: "0 16px 35px rgba(0, 0, 0, 0.22)",
          textDecoration: "none",
        }}
      >
        <div
          role="img"
          aria-label="Moonshot AI"
          style={{
            width: "100%",
            aspectRatio: "16 / 9",
            borderRadius: "0.45rem",
            backgroundColor: "#ffffff",
            backgroundImage: "url('/cards/logos/moonshotai.png')",
            backgroundPosition: "center",
            backgroundRepeat: "no-repeat",
            backgroundSize: "cover",
          }}
        />
        <div
          style={{
            padding: "0.65rem 0.35rem 0.2rem",
            color: "#111827",
            textAlign: "center",
            fontSize: "0.78rem",
            fontWeight: 750,
            letterSpacing: "0.06em",
            textTransform: "uppercase",
          }}
        >
          Kimi-K3 deployment guide
        </div>
      </a>
    </div>
  </div>
</div>

<a
  class="github-button"
  href="https://github.com/sgl-project/sglang"
  data-size="large"
  data-show-count="true"
  aria-label="Star sgl-project/sglang on GitHub"
>
  Star
</a>
<a
  class="github-button"
  href="https://github.com/sgl-project/sglang/fork"
  data-icon="octicon-repo-forked"
  data-size="large"
  data-show-count="true"
  aria-label="Fork sgl-project/sglang on GitHub"
>
  Fork
</a>
<script async defer src="https://buttons.github.io/buttons.js"></script>
<br></br>

<CardGroup cols={2}>
  <Card title="Performance & Runtime" icon="arrow-trend-up">
    Designed for low-latency, high-throughput inference with RadixAttention, prefix caching, and multi-GPU parallelism.
  </Card>

<Card title="Models & Ecosystem" icon="hexagon-nodes">
  Broad support for Llama, Qwen, DeepSeek, and more. Compatible with Hugging
  Face and OpenAI APIs.
</Card>

<Card title="Extensive Hardware Support" icon="microchip">
  Native support across <a href="./docs/hardware-platforms/overview">Hardware Platforms</a>
  including NVIDIA, AMD, Intel Xeon, Google TPU, and Ascend NPU accelerators.
</Card>

  <Card title="Community & Training" icon="users">
    Open-source with widespread adoption, powering 400k+ GPUs and integrated with major RL frameworks.
  </Card>
</CardGroup>

SGLang powers large-scale production deployments, generating trillions of tokens each day across more than 400,000 GPUs worldwide. It is hosted under the non-profit open-source organization [LMSYS](https://lmsys.org/about/).

---

## Get Started

SGLang is an inference framework meant for production level serving.
It is designed to deliver low-latency and high-throughput inference across a wide range of setups, from a single GPU to large distributed clusters.

<CardGroup cols={2}>
<Card title="Install SGLang" icon="angles-down" href="./docs/get-started/install">
  Install SGLang with pip, from source, or via Docker on your preferred hardware platform.
</Card>

<Card title="Quickstart" icon="zap" href="./docs/get-started/quickstart">
  Launch your first model server and send requests in minutes with OpenAI-compatible APIs.
</Card>
</CardGroup>

## News and latest blogs

{/* BEGIN_LMSYS_SGLANG_BLOG_CARDS */}
<div className="not-prose">
  <div
    style={{
      display: "grid",
      gridTemplateColumns: "repeat(auto-fit, minmax(300px, 1fr))",
      gap: "1rem",
      alignItems: "stretch",
    }}
  >
    <a
      href="https://lmsys.org/blog/2026-07-27-kimi-k3-day0-support/"
      target="_blank"
      rel="noopener noreferrer"
      style={{
        display: "block",
        border: "1px solid rgba(128, 128, 128, 0.3)",
        borderRadius: "0.75rem",
        overflow: "hidden",
        textDecoration: "none",
        color: "inherit",
        height: "100%",
      }}
    >
      <div
        style={{
          aspectRatio: "16 / 9",
          overflow: "hidden",
          background: "rgba(128, 128, 128, 0.15)",
        }}
      >
        <img
          src="https://lmsys.org/images/blog/kimi-k3-day0-support/cover-kimi-k3.png"
          alt="SGLang and Miles Add Day-0 Support for Kimi K3"
          style={{
            width: "100%",
            height: "100%",
            objectFit: "cover",
            objectPosition: "center",
            display: "block",
          }}
        />
      </div>
      <div style={{ padding: "0.9rem 1rem 1rem" }}>
        <p
          style={{
            margin: 0,
            fontWeight: 600,
            lineHeight: 1.35,
            fontSize: "0.98rem",
          }}
        >
          {"SGLang and Miles Add Day-0 Support for Kimi K3"}
        </p>
        <p
          style={{
            margin: "0.55rem 0 0",
            fontSize: "0.85rem",
            opacity: 0.75,
          }}
        >
          {"July 27, 2026"}
        </p>
      </div>
    </a>
    <a
      href="https://lmsys.org/blog/2026-07-18-opd-support-in-miles/"
      target="_blank"
      rel="noopener noreferrer"
      style={{
        display: "block",
        border: "1px solid rgba(128, 128, 128, 0.3)",
        borderRadius: "0.75rem",
        overflow: "hidden",
        textDecoration: "none",
        color: "inherit",
        height: "100%",
      }}
    >
      <div
        style={{
          aspectRatio: "16 / 9",
          overflow: "hidden",
          background: "rgba(128, 128, 128, 0.15)",
        }}
      >
        <img
          src="https://lmsys.org/images/blog/opd-support-in-miles/figure-1.png"
          alt="OPD Support in Miles"
          style={{
            width: "100%",
            height: "100%",
            objectFit: "cover",
            objectPosition: "center",
            display: "block",
          }}
        />
      </div>
      <div style={{ padding: "0.9rem 1rem 1rem" }}>
        <p
          style={{
            margin: 0,
            fontWeight: 600,
            lineHeight: 1.35,
            fontSize: "0.98rem",
          }}
        >
          {"OPD Support in Miles"}
        </p>
        <p
          style={{
            margin: "0.55rem 0 0",
            fontSize: "0.85rem",
            opacity: 0.75,
          }}
        >
          {"July 18, 2026"}
        </p>
      </div>
    </a>
    <a
      href="https://lmsys.org/blog/2026-07-15-inkling-day0-support/"
      target="_blank"
      rel="noopener noreferrer"
      style={{
        display: "block",
        border: "1px solid rgba(128, 128, 128, 0.3)",
        borderRadius: "0.75rem",
        overflow: "hidden",
        textDecoration: "none",
        color: "inherit",
        height: "100%",
      }}
    >
      <div
        style={{
          aspectRatio: "16 / 9",
          overflow: "hidden",
          background: "rgba(128, 128, 128, 0.15)",
        }}
      >
        <img
          src="https://lmsys.org/images/blog/inkling-day0-support/inkling-cover.png"
          alt="SGLang and Miles Add Day-0 Support for Inkling, a Frontier Multimodal Model"
          style={{
            width: "100%",
            height: "100%",
            objectFit: "cover",
            objectPosition: "center",
            display: "block",
          }}
        />
      </div>
      <div style={{ padding: "0.9rem 1rem 1rem" }}>
        <p
          style={{
            margin: 0,
            fontWeight: 600,
            lineHeight: 1.35,
            fontSize: "0.98rem",
          }}
        >
          {"SGLang and Miles Add Day-0 Support for Inkling, a Frontier Multimodal Model"}
        </p>
        <p
          style={{
            margin: "0.55rem 0 0",
            fontSize: "0.85rem",
            opacity: 0.75,
          }}
        >
          {"July 15, 2026"}
        </p>
      </div>
    </a>
    <a
      href="https://lmsys.org/blog/2026-07-13-glm52-optimization/"
      target="_blank"
      rel="noopener noreferrer"
      style={{
        display: "block",
        border: "1px solid rgba(128, 128, 128, 0.3)",
        borderRadius: "0.75rem",
        overflow: "hidden",
        textDecoration: "none",
        color: "inherit",
        height: "100%",
      }}
    >
      <div
        style={{
          aspectRatio: "16 / 9",
          overflow: "hidden",
          background: "rgba(128, 128, 128, 0.15)",
        }}
      >
        <img
          src="https://lmsys.org/images/blog/glm52-optimization/glm52-nvfp4-performance-pareto.png"
          alt="Serving GLM5.2 NVFP4 Agentic Workload with SGLang: Reaching 500 TPS in 2 Weeks"
          style={{
            width: "100%",
            height: "100%",
            objectFit: "cover",
            objectPosition: "center",
            display: "block",
          }}
        />
      </div>
      <div style={{ padding: "0.9rem 1rem 1rem" }}>
        <p
          style={{
            margin: 0,
            fontWeight: 600,
            lineHeight: 1.35,
            fontSize: "0.98rem",
          }}
        >
          {"Serving GLM5.2 NVFP4 Agentic Workload with SGLang: Reaching 500 TPS in 2 Weeks"}
        </p>
        <p
          style={{
            margin: "0.55rem 0 0",
            fontSize: "0.85rem",
            opacity: 0.75,
          }}
        >
          {"July 14, 2026"}
        </p>
      </div>
    </a>
    <a
      href="https://lmsys.org/blog/2026-07-10-rocm-miles-dsv4/"
      target="_blank"
      rel="noopener noreferrer"
      style={{
        display: "block",
        border: "1px solid rgba(128, 128, 128, 0.3)",
        borderRadius: "0.75rem",
        overflow: "hidden",
        textDecoration: "none",
        color: "inherit",
        height: "100%",
      }}
    >
      <div
        style={{
          aspectRatio: "16 / 9",
          overflow: "hidden",
          background: "rgba(128, 128, 128, 0.15)",
        }}
      >
        <img
          src="https://lmsys.org/images/blog/rocm-miles-dsv4/preview.png"
          alt="Bringing DeepSeek-V4 Flash RL Training to AMD Instinct MI355X GPUs with Miles"
          style={{
            width: "100%",
            height: "100%",
            objectFit: "cover",
            objectPosition: "center",
            display: "block",
          }}
        />
      </div>
      <div style={{ padding: "0.9rem 1rem 1rem" }}>
        <p
          style={{
            margin: 0,
            fontWeight: 600,
            lineHeight: 1.35,
            fontSize: "0.98rem",
          }}
        >
          {"Bringing DeepSeek-V4 Flash RL Training to AMD Instinct MI355X GPUs with Miles"}
        </p>
        <p
          style={{
            margin: "0.55rem 0 0",
            fontSize: "0.85rem",
            opacity: 0.75,
          }}
        >
          {"July 10, 2026"}
        </p>
      </div>
    </a>
    <a
      href="https://lmsys.org/blog/2026-07-06-dspark-sglang/"
      target="_blank"
      rel="noopener noreferrer"
      style={{
        display: "block",
        border: "1px solid rgba(128, 128, 128, 0.3)",
        borderRadius: "0.75rem",
        overflow: "hidden",
        textDecoration: "none",
        color: "inherit",
        height: "100%",
      }}
    >
      <div
        style={{
          aspectRatio: "16 / 9",
          overflow: "hidden",
          background: "rgba(128, 128, 128, 0.15)",
        }}
      >
        <img
          src="https://lmsys.org/images/blog/dspark-sglang/perf-compare.png"
          alt="DSpark in SGLang: Speculative Decoding with Confidence-Driven, Variable-Length Verification"
          style={{
            width: "100%",
            height: "100%",
            objectFit: "cover",
            objectPosition: "center",
            display: "block",
          }}
        />
      </div>
      <div style={{ padding: "0.9rem 1rem 1rem" }}>
        <p
          style={{
            margin: 0,
            fontWeight: 600,
            lineHeight: 1.35,
            fontSize: "0.98rem",
          }}
        >
          {"DSpark in SGLang: Speculative Decoding with Confidence-Driven, Variable-Length Verification"}
        </p>
        <p
          style={{
            margin: "0.55rem 0 0",
            fontSize: "0.85rem",
            opacity: 0.75,
          }}
        >
          {"July 6, 2026"}
        </p>
      </div>
    </a>
  </div>
</div>
{/* END_LMSYS_SGLANG_BLOG_CARDS */}

---

## Learn more and join the community

<div className="not-prose">
  <div
    style={{
      padding: "0.9rem 0",
      borderTop: "1px solid rgba(128, 128, 128, 0.24)",
      borderBottom: "1px solid rgba(128, 128, 128, 0.24)",
    }}
  >
    <p
      style={{
        margin: "0 0 0.35rem",
        fontSize: "0.82rem",
        fontWeight: 700,
        letterSpacing: "0.08em",
        textTransform: "uppercase",
        opacity: 0.72,
      }}
    >
      Stay connected
    </p>
    <div
      style={{
        display: "grid",
        gap: "0.55rem",
        fontSize: "0.97rem",
        lineHeight: 1.7,
      }}
    >
      <div>
        <span style={{ display: "inline-flex", alignItems: "center", verticalAlign: "-0.125em" }}>
          <Icon icon="map" size={14} />
        </span>{" "}
        <a href="https://roadmap.sglang.io">Development roadmap</a>
        <span style={{ opacity: 0.62 }}> to follow current priorities and upcoming work.</span>
      </div>
      <div>
        <span style={{ display: "inline-flex", alignItems: "center", verticalAlign: "-0.125em" }}>
          <Icon icon="calendar-days" size={14} />
        </span>{" "}
        <a href="https://meet.sglang.io">Weekly public development meeting</a>
        <span style={{ opacity: 0.62 }}> to hear updates and join open discussions.</span>
      </div>
      <div>
        <span style={{ display: "inline-flex", alignItems: "center", verticalAlign: "-0.125em" }}>
          <Icon icon="slack" size={14} />
        </span>{" "}
        <a href="https://slack.sglang.io/">Slack</a>
        <span style={{ opacity: 0.62 }}> for questions, feedback, and community support.</span>
      </div>
      <div>
        <a href="https://x.com/lmsysorg">X Twitter</a>
        <span style={{ opacity: 0.62 }}> and </span>
        <span style={{ display: "inline-flex", alignItems: "center", verticalAlign: "-0.125em" }}>
          <Icon icon="linkedin" size={14} />
        </span>{" "}
        <a href="https://www.linkedin.com/company/sgl-project/">LinkedIn</a>
        <span style={{ opacity: 0.62 }}> for project updates.</span>
      </div>
      <div>
        <span style={{ display: "inline-flex", alignItems: "center", verticalAlign: "-0.125em" }}>
          <Icon icon="newspaper" size={14} />
        </span>{" "}
        <a href="https://lmsys.org/blog/">LMSYS blog</a>
        <span style={{ opacity: 0.62 }}> for release notes, benchmarks, and technical deep dives.</span>
      </div>
      <div>
        <span style={{ display: "inline-flex", alignItems: "center", verticalAlign: "-0.125em" }}>
          <Icon icon="book-open" size={14} />
        </span>{" "}
        <a href="https://github.com/sgl-project/sgl-learning-materials">Learning materials</a>
        <span style={{ opacity: 0.62 }}> for blogs, slides, and videos.</span>
      </div>
    </div>
  </div>
</div>
