开源框架整理 ¶

训练 ¶

• Megatron
• DeepSpeed

SFT¶

• hiyouga/LLaMA-Factory

RL¶

• volcengine/verl
• OpenRL-Lab/openrl
• alibaba/ROLL
• allenai/open-instruct
• NovaSky-AI/SkyRL
• PrimeIntellect-ai/prime-rl
• unslothai/unsloth
• OpenRLHF/OpenRLHF
• agentica-project/verl-pipeline
• RLinf/RLinf
• NVIDIA-NeMo/RL

调度 ¶

• LLM-d
• aibrix
• NVIDIA-dynamo
• Ray
• sgl-project/rbg

Ckpt¶

• Mooncake
• NIXL

KV 管理 ¶

• Mooncake
• LM-Cache

推理 ¶

• vLLM
• SGlang
• TensorRT-LLM
• LMdeploy
• ktransformers
• ggml-org/llama.cpp
• thu-pacman/chitu
• alibaba/rtp-llm
• jd-opensource/xllm

• Nano vLLM
• mini-sglang

accelerate¶

• flashinfer-ai/flashinfer
• Dao-AILab/flash-attention
• SageAttention

PD-disaggregation¶

• LLMServe/DistServe

Speculative-Decoding¶

• SafeAILab/EAGLE
• smart-lty/nano-PEARL

算子与通信 ¶

• deepseek-ai/DeepEP

• Triton
• PyTorch Distributed

• NIXL
• RDMA
• NVLimk
• PCIE
• Ascend HIXL

可观测 ¶

• Nsight System
• Nsight Compute
• Prometheus
• AIConfigurator
• AIprofiling

云原生 ¶

• Kubernetes
• slurm
• Docker
• Hami
• fluid-cloudnative/fluid

基础设施 ¶

• Hardware
  • CPU
  • GPU NVIDIA
  • NPU
  • Ascend

• Cloud
  • Aliyun
  • huoshan
  • tencent
  • AWS