Andy Luo

Andy Luo leads AI application engineering at AMD, where he works on AMD Instinct GPU software and open-source AI inference and training ecosystems, including ROCm, vLLM, SGLang, PyTorch, LMCache, and Mooncake. His technical writing covers GPU inference optimization, serving systems, and performance techniques for large language models.

Career

Cosigned byLoading...

CosignedSign in to view

Elsewhere

Fudan University

Products & media

rocm.blogs.amd.com

Best Practices for Competitive Inference Optimization on AMD Instinct MI300X GPUs

A practical guide to configuring MI300X GPUs for LLM inference, covering hardware utilization, memory, bandwidth, and scaling.

Writing

rocm.blogs.amd.com

Efficient LLM Serving with MTP: DeepSeek V3 and SGLang on AMD Instinct GPUs

Explains how multi-token prediction and speculative decoding can accelerate DeepSeek V3 inference with SGLang on AMD GPUs.

Writing

rocm.blogs.amd.com

Unleashing AMD Instinct MI300X GPUs for LLM Serving: Disaggregating Prefill & Decode with SGLang

Describes using SGLang to separate prefill and decode workloads across GPUs to improve latency and throughput.

Writing

rocm.blogs.amd.com

vLLM V1 Meets AMD Instinct GPUs: A New Era for LLM Inference Performance

Explores vLLM V1 improvements on AMD GPUs, including serving performance and multimodal capabilities.

Writing

vllm.ai

EAGLE3 Speculative Decoding on AMD Instinct GPUs: Training and Serving with vLLM and AMD Quark

Details an EAGLE3 workflow for training draft models and accelerating inference on AMD Instinct GPUs.

Writing

blog.lmcache.ai

AMD × LMCache: AMD GPU Acceleration with LMCache

Introduces LMCache integration with vLLM and describes its impact on long-context inference workloads on AMD GPUs.

Writing

Profile name

Company name

Company name

Profile name

Company name

Company name

Profile name

Company name

Company name

Profile name

Company name

Company name

Profile name

Company name

Company name

Andy Luo

Fudan University

Products & media

Best Practices for Competitive Inference Optimization on AMD Instinct MI300X GPUs

Efficient LLM Serving with MTP: DeepSeek V3 and SGLang on AMD Instinct GPUs

Unleashing AMD Instinct MI300X GPUs for LLM Serving: Disaggregating Prefill & Decode with SGLang

vLLM V1 Meets AMD Instinct GPUs: A New Era for LLM Inference Performance

EAGLE3 Speculative Decoding on AMD Instinct GPUs: Training and Serving with vLLM and AMD Quark

AMD × LMCache: AMD GPU Acceleration with LMCache

Curated Lists

List Inclusions