Compilers for Machine Learning (Spring 2027)

Draft syllabus. Items marked TBD must be finalized before the course begins.

A hands-on, one-semester course where you build your own compiler from scratch, starting with elementwise programs and ending with training modern LLMs on GPUs. Each week builds aggressively on the previous week. Keep your code simple: mistakes and unnecessary complexity compound.

This course covers the design and implementation of a modern machine learning compiler, and the interaction between IR design, hardware capabilities, and the structure of machine learning programs. Topics include term rewriting, code generation, movement operators, kernel fusion, memory hierarchies, GPU architecture, automatic differentiation, and flash attention.

Format: In-person lectures and quizzes, a semester-long compiler project, and a final project. Work individually or in teams of two. Choose your own implementation language. There is no starter code; the specification uses a simple interchange format for the compiler IR.

Preparation: No formal prerequisites. You should be deeply familiar with 15-213 or equivalent material, highly skilled in your chosen implementation language, comfortable with C and some GPU programming, and able to manage a programming project that lasts a full semester.

By the end of the course, you should be able to:

All percentages below are percentages of the final course grade.

Weekly deadlines: Tests are due Monday in weeks 1–10; compilers are due Wednesday following weeks 1–11. All are due at 10 AM, on the dates in the schedule. No submissions are due during spring break. All course times and deadlines are Pittsburgh local time (America/New_York).

No midterm or final exam. Letter grades: A ≥ 90%; B ≥ 80% and < 90%; C ≥ 70% and < 80%; F < 70%.

Grading environment: Weeks 1–7 focus on correctness; GPU work begins in week 8. Performance grading in weeks 8–11 will use a fixed benchmark suite on identical GPUs, tentatively NVIDIA GeForce RTX 3080s. Final hardware and access details are TBD. Performance scores are 0 for the lowest-performing submission, 100 for the highest, with scores interpolated linearly between them.

Autograder: A custom autograder will pull code from each student or team’s private GitHub repository at a designated branch/tag and run the tests. Repository setup, branch/tag naming, submission mechanics, test visibility, and resubmission rules are TBD.

Final project: The planned deadlines are Monday, April 12 at 10 AM for a short proposal, and Friday, April 30 at 10 AM for code, reproducible results, and a short report. Projects start on April 14; the detailed rubric will be published before proposals are due.

TBD: Detailed style rubric, reference hardware, performance benchmark aggregation and tie handling, and rounding policy.

Tentative course plan, based on CMU’s official Spring 2027 calendar. Course weeks run Wednesday–Friday–Monday, with the compiler due the following Wednesday. All dates below are in 2027; all deadlines are 10 AM. See the technical syllabus for operation definitions.

First class: January 20. Last class: April 30. No class January 18 (MLK Day, before the semester starts), March 8–12 (spring break), or April 16 (Carnival). The course has 40 class meetings and no final exam.

Religious observance conflicts, including the April 30 project deadline on Orthodox Holy Friday, will be accommodated. Contact the instructor in advance; see CMU’s religious observances guide.

Attendance. Attend class; lectures are not posted online. Quizzes occur at the start of class. No separate attendance or participation grade.

Quizzes. Quizzes are individual and closed-book, with no devices except approved accommodations. The lowest two quiz scores are dropped, including zeros for missed quizzes. Contact the instructor about additional excused absences so that make-up or alternative arrangements can be made.

Partners. Work alone or with the same partner throughout the semester; partner changes require instructor approval. Both partners must understand all submitted code. Project grades are normally shared, but may be adjusted individually for substantially unequal contributions. Raise partner problems early.

Academic integrity. CMU’s academic integrity policy applies. Discuss concepts outside your team, not implementation code. Keep repositories private; share tests only through course submissions. Study existing compilers, but do not copy their implementations. Cite sources and ask before using other assistance.

Libraries and tools. General-purpose libraries, including graph and DAG utilities, are allowed, as are Clang, LLVM, and CUDA APIs or thin wrappers such as cuda-python. Libraries providing prewritten numerical kernels, such as BLAS or cuBLAS, are not allowed. ML frameworks and tensor/kernel compiler frameworks, such as MLIR, PyTorch, tinygrad, Triton, TileLang, cuTile, etc., are also not allowed.

AI tools. AI may help explain concepts, debug code you wrote, and write tests or tooling. It may not write or rewrite your compiler implementation. Disclose its use and understand everything you submit. Academic integrity rules still apply. No AI during quizzes.

This course aggressively builds on the previous week, and is an exercise in slop management. If you let any slop in early, it will compound and you will not finish the class.

Late work. Each student has three 24-hour late days, for weekly compiler submissions only, at most one per submission. Both partners must have and spend a late day. No late tests. Late work without an available late day or approved extension receives no credit. Contact the instructor promptly about illness or other extension requests; approved accommodations are handled separately.

Regrading. Submit a written request to the course staff explaining the suspected grading error within seven business days of the grade being released.

Devices and recording. Devices are allowed unless disruptive, except during quizzes. Recording classroom activities requires instructor permission or an approved accommodation.

Accommodations. Discuss approved disability accommodations with the instructor early so we can arrange appropriate support. To request accommodations, contact Disability Resources. Let us know in advance about conflicts with religious observances so we can arrange appropriate accommodations.

Respect. Treat everyone with respect. This course holds all students to high standards and evaluates the quality of their work, not their background. We aim to give students of all backgrounds the opportunity to meet these standards.

Well-being. Take care of yourself and ask for help early. Contact the instructor or your advisor with concerns. For confidential mental health support, CaPS is available 24/7 at 412-268-2922.

Mandatory reporting. The instructor, and TAs acting in their TA roles, must report information about discrimination or sexual misconduct to CMU’s Office for Institutional Equity and Title IX. The office offers support; responding is your choice. Confidential resources are available from people who are not required to report.

No required textbook. The technical syllabus and assignment specifications are required course materials. Materials on GitHub; discussion on EdStem (links TBD). Additional readings will be marked required or supplementary when assigned.

Required tools: Git, a toolchain for your implementation language, a C compiler, and access to CUDA-capable GPUs for the GPU assignments. Supported environments, installation instructions, and GPU access details are TBD.