Xinpeng Wei

Xinpeng Wei

First-year Master Student

Georgia Institute of Technology

About

I am Xinpeng Wei, a first year master student at Georgia Institute of Technology majoring in Computer Science. Before that, I received my B.E. degree in Software Engineering from Shanghai Jiao Tong University.

Currently my research interest lies in building robust and scalable systems for large-scale machine learning workloads like large language models (LLMs). Specifically, I’m dedicated to high-performance/parallel computing(e.g. CUDA), distributed training/inference(e.g. EP, DP, PP, TP), machine learning frameworks(e.g. PyTorch), and other fancy technologies in this emerging field.

I’m so fortunate to have had the opportunity to work at some outstanding companies, laboratories, and receive mentorship from lots of supportive and insightful advisers. Check my Work Experience and Research Experience.

Interests
  • Machine Learning Systems
  • High-Performance Computing
  • Cloud Infrastructure
  • Operating Systems
Education
  • M.S. in Computer Science, 2024 (Expected)

    Georgia Institute of Technology

  • B.Eng. in Software Engineering, 2023

    Shanghai Jiao Tong University

News

  • 2024.1.17: 🎉 I will join Google Cloud Borglet team as a software engineer intern in the summer of 2024 at Sunnyvale, CA to build next-generation ML infrastructure ~
  • 2024.1.8: 🎉 I start working as a Graduate Teaching Assistant for CS 6465/4365 Introduction to Enterprise Computing taught by Prof. Calton Pu in 2024 Spring ~
  • 2023.9.22: 🎉 I will join Bytedance Applied Machine Learning team as a ML systems research engineer intern in the fall of 2024 at Seattle, WA to explore the infinite potential of Large Language Models ~
  • 2023.8.21: 🎉 I begin my master’s study at Georgia Institute of Technology ~
  • 2023.6.20: 🎉 I graduate from Shanghai Jiao Tong University with a bachelor’s degree in Software Engineering and is honorly awared Outstanding Graduate ~

Work Experience

 
 
 
 
 
TikTok, Applied Machine Learning Team
Machine Learning System R&D Intern
May 2023 – Aug 2023 Beijing, China
  • Integrated DeepSpeed to Bpex’s benchmarking tool to use distributed training and ZeRO to support larger models.
  • Wrote high-performance CUDA kernels for GEMM that utilize OP fusion, memory hierarchy and tensor cores.
  • Profiled performances of various attention implementations, analyzed in depth the differences between xFormers’s cutlass implementation and flash-attention(v1, v2), and delivered a comprehensive report in an internal presentation.
 
 
 
 
 
Microsoft, Cloud+AI Group
Software Engineer intern
Jun 2022 – Sep 2022 Shanghai, China
  • Developed and maintained a Teams Bot utilizing Microsoft Bot Framework and .NET core, which enables users to effortlessly query databases and request internal APIs using natural language.
  • Implemented QuickAccess, an admin web application to easily manipulate and search tax records using ASP.NET core MVC and Azure Cosmos DB.
  • Built tools utilizing Azure DevOps SDK to automate routine jobs including a daemon synchronizing data between task items and pull requests and a crawler maintaining data source for the Teams Bot, saving 10 working hours/month.
 
 
 
 
 
RisingWave Labs
Database System R&D Intern
Feb 2022 – Jun 2022 Shanghai, China

Participated in developing RisingWave, the next-generation cloud native streaming database.

  • Refined system’s value encoding.
  • Implemented common table expressions for front end.
  • Implemented internal table catalog, including inference and storage.

Research Experience

 
 
 
 
 
Northeastern University, Systems Research Group
Serving DNN Models with Multi-Instance GPUs
Sep 2022 – May 2023 Boston, MA, US
  • Adviser: Prof. Cheng Tan
  • Built a benchmarking tool to measure different machine learning models’ performance on Multi-Instance GPU.
  • Implemented a “exchange-and-compact” development transition algorithm on top of Kubernetes.
  • Optimized the transition time from 30’ to 1'.
  • Built the testbed containing 4-instances on GCP and carried out a 24-hour end to end test.
 
 
 
 
 
Institute of Parallel and Distributed Systems(IPADS)
Secure Personal Data Storage and Analysis System
Sep 2021 – Sep 2022 Shanghai, China
  • Adviser: Prof. Yubin Xia and Ph.D. Mingyu Li
  • Implemented a secure personal data analysis and storage system on Hikey960 using OPTEE and AOSP.
  • Designed and developed an app to generate SJTU students’ personal annual report.
  • Participated in Shanghai “Internet+” College Student Innovation and Entrepreneurship Contest as an SJTU representative.

Honors and Awards

2023

  • Graduate Teaching Assistant Fellowship
    • Issued by School of Computer Science, Georgia Institute of Technology to promote master students with great research potential
  • Outstanding Graduate of Shanghai Jiao Tong University
    • Issued by Shanghai Jiao Tong University

2022

  • Xiaomi Scholarship
    • 1 out of 97 senior undergraduates in the School of Software
    • Issued by Xiaomi Inc.

2021

  • Wish Scholarship
    • Two undergraduate students per year in the School of Software
    • Issued by Wish Inc.

2020

  • National Scholarship
    • Top 0.2% nationwide
    • Issued by Ministry of Education of P.R. China