Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR removes the check that
IndexType
isInt32
when using TMA.When TMA is enabled we have a coordinate array specifying the data pointer of the global memory array and the box coordinates. The global array might be larger enough to require an
int64_t
to linearly index its last element, in which case we would useint64_t
indexing. However, since it is possibly a multidimensional array, it is still possible that the multidimensional coords of all TMA boxes can still be expressed withint32_t
. These are the cases we want to cover for matmul since the input is 2D and we don't commonly expect each individual dimension to be larger than 2^31.Currently I am just static casting the box dims to
int32_t
inmemory.cu
. We should additionally verify that none of these coords will be larger than the capacity ofint32_t
. We could handle that more broadly actually by writing a more sophisticated analysis that takes every indexing expression (including TMA box coords) in the Fusion and proves bounds for it. This is trickier than what we currently do which is just look at allTensorView
s and determine the position of the furthest strided element within that tensor, but it might reveal that we useint64_t
in many cases where it's unnecessary (such as matmuls).Note that it's still possible to have a box coordinate that cannot be expressed with a 32-bit index, in which case we should not allow TMA.
TODO: we should check the above condition in the core heuristic where we have access to problem sizes.
Fixes #3595