Skip to content

TorchServe v0.4.1 Release Notes
Compare
Choose a tag to compare
@lxning lxning released this 22 Jul 17:38
v0.4.1
c435485

This is the release of TorchServe v0.4.1.

New Features

  • PyTorch 1.9.0 support - TorchServe is now certified working with torch 1.9.0 torchvision 0.10.0, torchtext 0.10.0 and torchaudio 0.9.0
  • Model configuration support - Added support for model performance tuning on SageMaker via model configuration in config.properties.
  • Serialize config snapshots to DynamoDB - Added support for serializing config snapshots to DDB.
  • Prometheus metrics plugin support - Added support for Prometheus metrics plugin.
  • Kubeflow Pipelines support - Added support for Kubeflow pipelines and Google Vertex AI Manages pipelines, see examples here
  • KFServing docker support - Added production docker for KFServing.
  • Python 3.9 support - TorchServe is now certified working with Python 3.9.

Improvements

  • HF BERT models multiple GPU support - Added multi-gpu support for HuggingFace BERT models.
  • Error log for customer python package installation - Added support to log error of customer python package installation.
  • Workflow documentation optimization - Optimized workflow documentation.

Tooling improvements

  • Mar file automation integration - Integrated mar file generation automation into pytest and postman test.
  • Benchmark automation for AWS neuron support - Added support for AWS neuron benchmark automation.
  • Staging binary build support - Added support for staging binary build.

Platform Support

Ubuntu 16.04, Ubuntu 18.04, MacOS 10.14+, Windows 10 Pro, Windows Server 2019, Windows subsystem for Linux (Windows Server 2019, WSLv1, Ubuntu 18.0.4)

GPU Support

Torch 1.9.0 + Cuda 10.2, 11.1
Torch 1.8.1 + Cuda 9.2, 10.1

Assets 41
Loading