Course Outline
Introduction to GPU-Accelerated Computing
- Heterogeneous computing and CPU-GPU architecture.
- CUDA execution and memory spaces.
- Compiling CUDA C++ with nvcc and CMake.
- Verifying the GPU development environment.
Parallel Algorithms with Thrust and CUB
- GPU-accelerated sorting, reduction, and transformation.
- Refactoring STL algorithms for GPU execution.
- Thrust device vectors and execution policies.
- CUB device-wide primitives for custom pipelines.
GPU Memory Architecture and Management
- Global, constant, and texture memory types.
- Explicit device memory allocation and transfers.
- Unified Memory for simplified data access.
- Memory coalescing and access pattern optimization.
Asynchronous Execution with CUDA Streams
- Creating and managing CUDA streams.
- Overlapping kernel execution with data transfers.
- Using CUDA events for dependency management.
- Stream priorities and concurrency tuning.
Writing Custom CUDA Kernels
- The SIMT programming model and warp execution.
- Kernel launch configuration and grid-stride loops.
- Thread indexing and multidimensional grids.
- Error handling and CUDA runtime API checks.
Thread Hierarchy and Execution Model
- Grids, blocks, and threads in device code.
- Warp-level primitives and ballot operations.
- Block-level synchronization and barriers.
- Occupancy and resource utilization analysis.
Cooperative Groups for Flexible Parallelism
- The cooperative_groups API and group types.
- Thread-block tiles and tiled_partition.
- Grid-level cooperative launches.
- Multi-grid synchronization patterns.
Shared Memory Optimization Techniques
- Shared memory banks and avoiding bank conflicts.
- Tiling strategies for matrix operations.
- Using shared memory as user-managed cache.
- cuda::shared_memory_mdspan for multidimensional views.
Kernel Fusion and Advanced Parallel Patterns
- Fusing multiple kernels to reduce launch overhead.
- Scan, reduce-by-key, and segmented algorithms.
- Atomic operations and lock-free data structures.
- Warp-aggregated atomics for improved throughput.
Profiling and Optimization with Nsight Systems
- Timeline analysis of CPU and GPU activity.
- Identifying memory transfer bottlenecks.
- Kernel performance and occupancy profiling.
- Iterative optimization with Nsight Compute.
Modern C++ Features in CUDA Device Code
- Lambdas, constexpr, and auto usage in kernels.
- C++17 parallel algorithms and execution policies.
- C++20 concepts and ranges on device.
- C++23 support in nvcc and CCCL 3.x.
CUDA Graphs and Advanced Asynchrony
- Defining and launching CUDA graphs.
- Capturing graphs from stream execution.
- Graph updates and conditional execution nodes.
- Reducing launch latency for iterative workloads.
Integration Patterns for Existing Applications
- Wrapping GPU code behind C++ interfaces.
- Managing multi-GPU and NUMA systems.
- Build system integration with CMake and CUDA.
- Debugging device code with cuda-gdb.
Summary and Best Practices
- Choosing between Thrust, CUB, and custom kernels.
- Ensuring performance portability across GPU architectures.
- Code organization and RAII for CUDA resources.
- Next steps and advanced CUDA learning paths.
Requirements
- Fundamental C++ proficiency, including lambda expressions, templates, and the STL.
- Understanding of standard algorithms, containers, and iterators.
- Comfort with loops, conditionals, and functions.
- No prior experience with CUDA or GPU programming is required.
Audience
- C++ developers looking to accelerate compute-intensive applications with GPUs.
- Software engineers transitioning from CPU-only environments to heterogeneous parallel programming.
- Performance engineers and quantitative developers handling large datasets.
Testimonials (3)
Initially, the trainer's pace seemed a bit too fast for me, but after offering the feedback during the training, he acknowledged it and slowed down the pace without sacrificing any of the lectures. Good rapport with the audience, very friendly and open for discussions.
Alexandru Ostafi - Siemens
Course - Advanced C++ : Practical workshop
Detailed explanation, re-iteration of points in a quite subtle way that really drove the knowledge home very well. Rod's willingness to double-check the odd obscure question that we rasied, to be sure that his answers were 100% right. Also, his interest in discussing the pros & cons of alternate coding styles so that we learnt not only how to use C++ in our intended way, but why it should be done that way.
Nick Dillon - cellxica Ltd
Course - Using C++ in Embedded Systems - Applying C++11/C++14
Experience sharing, it's teacher's know-how and valuable.
