Set maximum Grid size and threads Per Block for GEMM CUDA kernel

[nnaron]

Hi,

I need to change the Grid size & threads Per Block for the GEMM kernel together. By changing the configuration I am not able to do it.

For example, setting the Grid size to 8 and threads Per Block to 32. Then I think the GEMM should decompose the whole operation in many small GEMMs.

[hwu36]

what do you mean? are you setting the launch bound? threads per threadblock is determined by warp tile size and threadblock tile size. grid size is determined by problem size and threadblock tile size. launch bound is a hint to the compiler, it has nothing to do with the tiling.

[nnaron]

I want to make some SMs idle, for that reason I need to decrease the grid size to a value lower than the total number of SMs.

[miniskar]

Does cutlas gemm provides configurable grid size and threadblock tile size?
It is dependent on the problem size. However, we want to have this control for some experiments.

[hwu36]

threadblock tile size is configurable during compilation time.

As to the grid size, in general if your kernel is persistent such as hopper kernels, then you can control the grid size. If it is not persistent such as the most of ampere kernels, it depends on the problem size and your chosen threadblock size.