How to modify threadblock shape when A/B is transposed in GEMM?

[imoneoi]

I wrote a custom kernel for grouped GEMM using cutlass. Should I transpose the threadblock, warp and instruction shapes here accordingly when A/B is transposed?

Besides, how to tune these shapes for optimal performance?

[jackkosaian]

Do you mean when there is an "internal transposition" due to the output being column major? If so, you should not need to change those template parameters.

To tune for performance, it's suggested to try out many valid combinations of threadblock shape, warp shape, and stage count. You may find it useful to follow some of the combinations listed by the CUTLASS kernel generator. For example, these lines show a subset of vali combinations threadblock shape, stage count, and warp count for an SM80 FP16 GEMM.

[imoneoi]

Thank you for your response! What I mean is when the input matrix A or B is transposed (column major) while C is always row major. Also, if any input becomes column major, does the best performing threadblock/warp shape/stage count change?

[jackkosaian]

Thanks for clarifying. A column-major input is unlikely to alter the best-performing combination of these parameters.