BaseTester: support plugin EPs with compiled nodes and registered kernels #27176
Conversation
adrianlizarraga
changed the title
| const auto& provider_type = ep.Type(); | ||
|
|
||
| constexpr std::array kEpsThatDoNotUseKernelRegistry{ | ||
| constexpr std::array kEpsThatDoNotUseKernelRegistryForAllNodes{ |
There was a problem hiding this comment.
maybe call them "EPs that compile nodes" instead of "EPs that do not use kernel registry for all nodes"
| constexpr std::array kEpsThatDoNotUseKernelRegistryForAllNodes{ | ||
| kOpenVINOExecutionProvider, | ||
| kTensorrtExecutionProvider, | ||
| kTensorrtExecutionProvider, // uses kernel registry for Memcpy* nodes, but compiles all others. |
There was a problem hiding this comment.
just to confirm, the plugin EP name will be different, right? that assumption is made here:
onnxruntime/onnxruntime/test/unittest_util/base_tester.cc
Lines 689 to 691 in f83d4d0
There was a problem hiding this comment.
Good question. @chilo-ms do you know if the plugin EP name is different?
There was a problem hiding this comment.
I assume so, but let me confirm with Nvidia for their plugin TRT RTX EP.
Also, do we want to include provider-bridge TRT RTX EP, kNvTensorRTRTXExecutionProvider, in this list as well?
There was a problem hiding this comment.
we should only add kNvTensorRTRTXExecutionProvider to this list if its name differs from the plugin EP version, right?
There was a problem hiding this comment.
we should only add kNvTensorRTRTXExecutionProvider to this list if its name differs from the plugin EP version, right?
Yes, waiting for Nvidia's reply for the name of their plugin TRT RTX EP, will update.
Description
Updates the
BaseTesterclass used by theonnxruntime_provider_testtool to support plugin EPs that use a kernel registry but compile other nodes. For example, TRT EP only uses registered kernels for Memcpy* nodes, but compiles every other node.Without this change, plugin EPs that use a mix of compiled nodes and registered kernels cannot be tested with
onnxruntime_provider_test.Motivation and Context