How does TransformerEngine interact with TensorRT/Onnx? #238
Unanswered
tylerweitzman
asked this question in
Q&A
Replies: 1 comment
-
|
Hi @tylerweitzman, We are evaluating support for exporting TE models to ONNX and import from TensorRT. Support for FP8 GPT export will be rolled out first. |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
Hi,
We are often converting PyTorch models to other engines in order to optimize inference speed. Is it possible to convert a PyTorch model that is using TransformerEngine into TensorRT for faster inference? Or is it a choice either between
nn.Linearwith TensorRT right now?Beta Was this translation helpful? Give feedback.
All reactions