File tree Expand file tree Collapse file tree 1 file changed +2
-2
lines changed
Expand file tree Collapse file tree 1 file changed +2
-2
lines changed Original file line number Diff line number Diff line change @@ -138,10 +138,10 @@ We support external backends for faster inference with fused kernels. You can en
138138``` Python
139139from hqq.utils.patching import prepare_for_inference
140140
141- # Pytorch backend that makes the model compatible with fullgrah torch.compile: works with any settings
141+ # Pytorch backend that makes the model compatible with fullgraph torch.compile: works with any settings
142142# prepare_for_inference(model)
143143
144- # Torchao's tiny_gemm backned (fastest): nbits=4, compute_dtype=bfloat16, axis=1
144+ # Torchao's tiny_gemm backend (fastest): nbits=4, compute_dtype=bfloat16, axis=1
145145prepare_for_inference(model, backend = " torchao_int4" )
146146
147147# Gemlite backend: nbits=4/2/1, compute_dtype=float16, axis=1
You can’t perform that action at this time.
0 commit comments