GPU programming with HIP

Course material for the CSC course "GPU programming with HIP". The course is part of the EuroCC training activities at CSC.

Agenda

Day 1

Time	Topic
09:00–09:35	Welcome & Introduction to GPU programming
09:35–09:45	Break
09:45-10:30	HIP and GPU kernels
10:30–10:45	Break & Snacks
10:45-11:30	Exercises
11:30-12:15	Streams, events, and synchronization
12:15-13:00	Lunch
13:00-13:30	Exercises
13:30-14:15	Memory allocations, access and unified memory
14:15-15:45	Exercises & Break
15:45-16:00	Day summary

Day 2

Time	Topic
09:00–09:35	Fortran and HIP
09:35–09:45	Break
09:45-10:30	Multi-GPU programming, HIP+MPI
10:30–10:45	Break & Snacks
10:45-11:30	Exercises
11:30-12:15	Kernel optimizations
12:15-13:00	Lunch
13:00-13:30	Exercises
13:30-14:15	Code design, conditional compilation, lambdas, hipify
14:15-15:45	Exercises & Break
15:45-16:00	Close-up

Link to slides

Exercises

General instructions

Introduction and GPU kernels

• Hello world
• Error checking
• Kernel saxpy
• Kernel copy2d

Streams, events, and synchronization

• Understanding asynchronity using events
• Investigating streams and events

Memory allocations, access, and unified memory

• Memory management strategies
• The stream-ordered memory allocator and memory pools
• Unified memory and structs

Fortran and HIP

• Hipfort exercise

Optimization

• Matrix Transpose

Multi-GPU programming and HIP+MPI

• Peer to peer device access
• Vector sum on two GPUs without MPI
• Ping-pong with multiple GPUs and MPI

Code design, conditional compilation, lambdas, hipify

• Host-device lambda functions and general kernels
• Reductions with host-device lambdas and hipCUB
• Monte Carlo simulation with hipRAND library

Bonus

• Heat equation with HIP