add a heuristic to set kWidth for attention decode#921

scxiao wants to merge 4 commits intorelease/internal/3.5.xROCm/triton:release/internal/3.5.xfrom

scxiao/kWidth_heuristicROCm/triton:scxiao/kWidth_heuristicCopy head branch name to clipboard