gemm_operation_profiler.cu

[scm-later]

What is your question?
I added split_k serial of cutlass 2.x to cutlass 3.x, slice_k as a parameter of problem_size. Now I want to use cutlass_profiler to test whether I should add a parameter to problem_size in gemm_operation_profiler, or use the existing split_k_slices parameter , but split_k_slices is not a parameter of problem_size

/***************************************************************************************************

• Copyright (c) 2017 - 2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
• SPDX-License-Identifier: BSD-3-Clause
• Redistribution and use in source and binary forms, with or without
• modification, are permitted provided that the following conditions are met:
• 1. Redistributions of source code must retain the above copyright notice, this
• list of conditions and the following disclaimer.
• 2. Redistributions in binary form must reproduce the above copyright notice,
• this list of conditions and the following disclaimer in the documentation
• and/or other materials provided with the distribution.
• 3. Neither the name of the copyright holder nor the names of its
• contributors may be used to endorse or promote products derived from
• this software without specific prior written permission.
• THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
• AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
• IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
• DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
• FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
• DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
• SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
• CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
• OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
• OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

**************************************************************************************************/
/*! \file
\brief Tests for device-wide GEMM interface
*/

#include

#include "cutlass/cutlass.h"
#include "cute/tensor.hpp"
#include "cute/atom/mma_atom.hpp"

#include "cutlass/numeric_types.h"

#include "cutlass/gemm/device/gemm_universal_adapter.h"
#include "default_gemm_configuration.hpp"

#include "common/gemm_test_3x.hpp"

using namespace cute;

/////////////////////////////////////////////////////////////////////////////////////////////////
int main(int argc, char* argv[]){
int m = 256;
int n = 256;
int k = 32;
int slice_k = 1;
if(argc >= 2){
m = atoi(argv[1]);
}
if(argc >= 3){
n = atoi(argv[2]);
}
if(argc >= 4){
k = atoi(argv[3]);
}
if(argc >= 5){
slice_k = atoi(argv[4]);
}

using Config = cutlass::gemm::device::DefaultGemmConfigurationToCutlass3Types<
cutlass::arch::OpClassTensorOp, cutlass::arch::Sm75,
half_t, cutlass::layout::ColumnMajor,
half_t, cutlass::layout::RowMajor,
half_t, cutlass::layout::ColumnMajor,
float>;

using GemmKernel = cutlass::gemm::kernel::GemmUniversal<
Shape<int,int,int,int>,
Config::CollectiveMainloop,
Config::CollectiveEpilogue

;

using Gemm = cutlass::gemm::device::GemmUniversalAdapter;
using ElementScalar = typename Gemm::EpilogueOutputOp::ElementScalar;
using ProblemShapeType = typename Gemm::GemmKernel::ProblemShape;
using Testbed = test::gemm::device::Testbed3x<Gemm, cutlass::epilogue::thread::Identity>;

Testbed testbed = {};
ProblemShapeType problem_size;
double alpha,beta;

problem_size = ProblemShapeType{m, n, k, slice_k};
alpha = 1.0;
beta = 0.0;

bool passed = testbed.run(
problem_size,
cutlass::from_real(alpha),
cutlass::from_real(beta)
);

int iterations = 100;
char perf = 'N';
if(perf == 'Y')
test::gemm::device::TestGemm3x(iterations);
}

/////////////////////////////////////////////////////////////////////////////////////////////////

[thakkarV]

but split_k_slices is not a parameter of problem_size

The "right" way to do split K in 3.x is to write a custom tile scheduler similar to streamK. Then the arguments of the scheduler would contain the num splits argument.

Aside: our StreamK scheduler supports a pure split K mode as well. Curious why you are implementing your own split K only scheduler when we support it OOTB?