IBM 新课程讲解 AI 模型如何在 GPU、TPU、Cerebras、Groq、Spyre 等加速器上运行

IBM Research：AI·2026-10-19 20:00· 1 小时前AI 评分36

New course teaches how AI models run on modern accelerators

AI 导读

IBM Research 研究员 Kaoutar El Maghraoui 在 Coursera 推出六门课专项课程《Dataflow Architectures for AI: From Principles to Practice》。

正文 · 原文

Most engineers building AI models today work on GPUs. They write models in PyTorch, run them, and measure latency and utilization. What happens between the framework and the silicon often remains hidden. That was manageable when one architecture dominated. It is much harder now, as AI hardware has diversified across GPUs, Google TPUs, the Cerebras Wafer-Scale Engine (WSE), the Groq Language Processing Unit (LPU), and IBM's Spyre accelerator.

A new Coursera specialization, Dataflow Architectures for AI: From Principles to Practice, is built for engineers who need to work across that range. Taught by Kaoutar El Maghraoui, a principal research scientist at IBM Research and adjunct professor of computer science at Columbia University, the six-course program covers execution models, compilation, memory systems, debugging, and performance modeling through hands-on labs.

"Sometimes the model itself isn't the difficult part," said El Maghraoui. "The difficult part is understanding how that model maps onto a particular architecture and why something that works very well on one system may behave completely differently on another."

One PyTorch model and five execution models

The specialization starts from a single PyTorch model. At the framework level it is the same nn.Module on every target. Once a compiler maps it onto hardware, the execution looks very different, and each stack supports its own set of operators, layouts, and graph transformations. A GPU runs the model as kernels across many parallel threads, with hardware-managed caches and specialized matrix units. A TPU maps the work onto systolic arrays, with the XLA compiler directing tiling and data movement. Cerebras spreads compute and more than 40 GB of on-chip SRAM across a full wafer. Groq relies on its compiler to schedule when each operation runs and when data moves. Spyre compiles the model into a dataflow graph, and its compiler decides layout and scratchpad placement while the runtime orchestrates transfers and launches. The details vary by product generation.

Those choices change which optimizations matter. Fusion removes intermediate memory traffic, but how much it saves depends on whether the compiler already keeps intermediates on chip, how tensors are tiled, and whether the fused kernel adds memory pressure. A tensor dimension that maps cleanly on one target can require padding or data rearrangement on another. The specialization treats dataflow execution as an architectural class with shared principles. Each concept is introduced in general terms before any specific system is discussed. Spyre is the main worked example because El Maghraoui's team works with it directly, but it is one of five systems compared throughout.

Two implementations of the same operation can also run at very different speeds for reasons that have little to do with arithmetic. For matrix multiplication and attention, tiling, data layout, synchronization, and movement between on-chip and off-chip memory often set the achievable performance.

Figure 1: The Accelerator Landscape Map. Students carry this table through all six courses and add the reasoning behind each cell as they go.

"There's no single architecture that wins every row," El Maghraoui said. "GPUs prioritize flexibility and broad programmability. Other systems give up some of that flexibility in order to gain efficiency, determinism, scale, or tighter control over execution."

The compiler gets particular attention. On specialized accelerators it is responsible for tiling, fusion, scheduling, placement, and memory management. That is where much of the performance is won or lost during model enablement. "You can have an accelerator that's extremely efficient for the operations it supports," she said, "but then one unsupported operator, one tensor shape that maps badly, or one memory-placement problem can have a large impact on the full model."

Figure 2: A simplified view of PyTorch compilation, covered in Course 3. Which transformations run depends on the compilation mode and the backend. With torch.compile, Inductor does the lowering by default, and torch-spyre extends it for Spyre. TPUs lower through XLA instead. Dataflow backends make most of their fusion, tiling, and placement decisions during lowering.

What the specialization covers

The specialization is a sequence of six courses. Each course covers one layer of the stack, and each ends in a lab. Together they build up the accelerator landscape map one row at a time. Completing all six earns the professional certificate in AI Systems Performance Engineering.

• Course 1 — The accelerator landscape: Domain-specific architectures, control-flow and dataflow execution, memory hierarchies, and programming models across all five systems.
• Course 2 — Dataflow execution models: Nodes, edges, firing rules, and static and dynamic scheduling. A transformer attention block traced through Spyre and TPU.
• Course 3 — Compilation for dataflow: TorchDynamo, AOTAutograd, and backend code generation. Fusion, tiling, and scheduling in XLA and the Spyre compiler.
• Course 4 — Memory systems: Caches, scratchpads, and SRAM-only designs. How each memory hierarchy changes KV cache management for LLM inference.
• Course 5 — Debugging across the dataflow stack: An eight-layer taxonomy of failures, including graph capture errors and scheduling deadlocks, DMA timeouts, and numerical issues.
• Course 6 — Performance modeling and optimization: Roofline analysis adapted for software-managed memory, with scheduling measured as a third bottleneck alongside compute and memory.

Debugging and performance across the stack

On a dataflow accelerator, a wrong result or a slowdown often shows up far from its cause. A NaN at the model output can come from an operator decomposition, a fusion pass, a tensor layout mismatch, or a DMA error. Course 5 organizes these failures into an eight-layer taxonomy, from graph capture errors in the frontend to scheduling conflicts, deadlocks, DMA timeouts, and numerical issues in the backend.

For correctness, students shrink the failure to a small reproducer, compare each layer's output against a CPU or GPU reference, and find the first point where results diverge. For performance, Course 6 bounds the timed region, splits kernel time from transfer time, compares the result against the compiler's ideal, and attributes the remainder to bytes moved at each memory level. Each operation then goes on a Roofline chart.

In one bug reported in the open-source torch-spyre project, cloning a 65-element fp16 tensor returned a wrong last value. The transfer log was correct, but the compiled copy covered only the first 64 elements, and comparing each layer's output against a reference found the error at that step.

Figure 3: The debugging and performance methods taught in Courses 5 and 6, drawn from the open-source torch-spyre debugging and profiling guides. Both walk the same compiler stack. Correctness work looks for the first layer whose output diverges from a reference. Performance work attributes measured time to the layer and memory level that caused it.

Figure 4: Roofline analysis as taught in Course 6. A roofline shows how far an operation sits below its bound. It does not show why. On a dataflow target the cause can be scheduling: data shuffling, graph traversal, or pipeline stalls. Profilers localize time to kernels, data transfers, and communication. How finely they attribute time inside a kernel depends on the tool.

Labs on Google Colab and Spyre

Every course culminates in a core lab that runs on Google Colab with standard PyTorch. Each lab also has an optional Spyre practice portion that runs on Spyre hardware. The labs are developed by Rashed Bhatti, PhD, a senior technical staff member at IBM Research and a contributing lab instructor for the specialization. Each takes 90 minutes to two hours, and students write and run their own code. In the first lab they build a benchmarking harness from scratch with torch.utils.benchmark and explain the CPU to GPU gap in terms of SIMT parallelism and HBM bandwidth. Later labs trace a transformer block with torch.fx, compare eager mode with torch.compile, read the PyTorch Profiler memory timeline during LLM inference, diagnose three planted bugs, and build a Roofline chart for an LLM workload.

No proprietary tools are required for the core labs. In the optional Spyre practice portion, called the Dataflow Deep Dive, students in partnered programs run the same labs on Spyre hardware. There they work directly with the dataflow concepts from each course: They read the dataflow graph the Spyre compiler produces and catalog the fusions and data-movement edges it inserts, follow how tensors are tiled into sticks and placed in scratchpad memory, measure traffic between HBM and the scratchpads with hardware counters, and build a Roofline from those counters for a software-managed memory hierarchy. The deep dive is not required to finish the specialization or earn the certificate.

Who should take it

The specialization is designed for intermediate to advanced learners, including graduate students in computer science or electrical engineering, compiler engineers, hardware architects, and ML practitioners who optimize workloads across hardware targets. Participants should be comfortable with intermediate Python and C/C++, familiar with CNN and transformer architectures, and grounded in basic computer architecture.

Learners who complete the program receive a professional certificate in AI Systems Performance Engineering.

"By the end of the course, when you see a new AI accelerator, I want you to know what questions to ask about it," El Maghraoui said. "What's the execution model? Where is the memory? Who schedules the operations? What does the compiler control?"

The specialization is open for enrollment on Coursera.

来源：IBM Research：AI · research.ibm.com