Parler TTS Large model problems on Linux

[vuaieo]

hi something is wrong with this Parler TTS Large model , here the error when tryuing to generate

Model: parler-tts/parler-tts-large-v1
Loading checkpoint shards: 100%|████████████████████████████████████████| 2/2 [00:00<00:00, 3.34it/s]
Traceback (most recent call last):
File "/home/vuaieo/.config/upbge/4.3/scripts/addons/Pallaidium-main/init.py", line 3126, in execute
pipe = ParlerTTSForConditionalGeneration.from_pretrained("parler-tts/parler-tts-large-v1").to(gfx_device)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/vuaieo/Desktop/UPBGE_29_8_2024/4.3/python/lib/python3.11/site-packages/transformers/modeling_utils.py", line 2848, in to
return super().to(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/vuaieo/Desktop/UPBGE_29_8_2024/4.3/python/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1174, in to
return self._apply(convert)
^^^^^^^^^^^^^^^^^^^^
File "/home/vuaieo/Desktop/UPBGE_29_8_2024/4.3/python/lib/python3.11/site-packages/torch/nn/modules/module.py", line 780, in _apply
module._apply(fn)
File "/home/vuaieo/Desktop/UPBGE_29_8_2024/4.3/python/lib/python3.11/site-packages/torch/nn/modules/module.py", line 780, in _apply
module._apply(fn)
File "/home/vuaieo/Desktop/UPBGE_29_8_2024/4.3/python/lib/python3.11/site-packages/torch/nn/modules/module.py", line 780, in _apply
module._apply(fn)
[Previous line repeated 5 more times]
File "/home/vuaieo/Desktop/UPBGE_29_8_2024/4.3/python/lib/python3.11/site-packages/torch/nn/modules/module.py", line 805, in _apply
param_applied = fn(param)
^^^^^^^^^
File "/home/vuaieo/Desktop/UPBGE_29_8_2024/4.3/python/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1160, in convert
return t.to(
^^^^^
torch.OutOfMemoryError: CUDA out of memory. Tried to allocate 40.00 MiB. GPU 0 has a total capacity of 11.87 GiB of which 71.19 MiB is free. Including non-PyTorch memory, this process has 10.23 GiB memory in use. Of the allocated memory 9.62 GiB is allocated by PyTorch, and 217.09 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True to avoid fragmentation. See documentation for Memory Management (https://pytorch.org/docs/stable/notes/cuda.html#environment-variables)
Error: Python: Traceback (most recent call last):
File "/home/vuaieo/Desktop/UPBGE_29_8_2024/4.3/python/lib/python3.11/site-packages/torch/nn/modules/module.py", line 805, in _apply
param_applied = fn(param)
^^^^^^^^^
File "/home/vuaieo/Desktop/UPBGE_29_8_2024/4.3/python/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1160, in convert
return t.to(
^^^^^
torch.OutOfMemoryError: CUDA out of memory. Tried to allocate 40.00 MiB. GPU 0 has a total capacity of 11.87 GiB of which 71.19 MiB is free. Including non-PyTorch memory, this process has 10.23 GiB memory in use. Of the allocated memory 9.62 GiB is allocated by PyTorch, and 217.09 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True to avoid fragmentation. See documentation for Memory Management (https://pytorch.org/docs/stable/notes/cuda.html#environment-variables)

[tin2tin]

huggingface/parler-tts#120 (comment)

[tin2tin]

@vuaieo Parler now works on Windows(with my latest update). Could you check it on Linux?

[vuaieo]

yep i checked and sadly i still getting error:

Flash attention 2 is not installed
Model: parler-tts/parler-tts-large-v1
/home/vuaieo/Desktop/UPBGE_29_8_2024/4.3/python/lib/python3.11/site-packages/torch/nn/utils/weight_norm.py:134: FutureWarning: torch.nn.utils.weight_norm is deprecated in favor of torch.nn.utils.parametrizations.weight_norm.
WeightNorm.apply(module, name, dim)
Loading checkpoint shards: 100%|██████████████████████████████| 2/2 [00:03<00:00, 1.65s/it]
Seed: -1194386060
Seed: 202547
The attention mask is not set and cannot be inferred from input because pad token is same as eos token.As a consequence, you may observe unexpected behavior. Please pass your input's attention_mask to obtain reliable results.
Traceback (most recent call last):
File "/home/vuaieo/.config/upbge/4.3/scripts/addons/Pallaidium-main/init.py", line 3424, in execute
generation = pipe.generate(input_ids=input_ids, prompt_input_ids=prompt_input_ids)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/vuaieo/Desktop/UPBGE_29_8_2024/4.3/python/lib/python3.11/site-packages/torch/utils/_contextlib.py", line 116, in decorate_context
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/home/vuaieo/Desktop/UPBGE_29_8_2024/4.3/python/lib/python3.11/site-packages/parler_tts/modeling_parler_tts.py", line 3461, in generate
outputs = self._sample(
^^^^^^^^^^^^^
File "/home/vuaieo/Desktop/UPBGE_29_8_2024/4.3/python/lib/python3.11/site-packages/transformers/generation/utils.py", line 2932, in _sample
outputs = self(**model_inputs, return_dict=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/vuaieo/Desktop/UPBGE_29_8_2024/4.3/python/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/vuaieo/Desktop/UPBGE_29_8_2024/4.3/python/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/vuaieo/Desktop/UPBGE_29_8_2024/4.3/python/lib/python3.11/site-packages/parler_tts/modeling_parler_tts.py", line 2728, in forward
decoder_outputs = self.decoder(
^^^^^^^^^^^^^
File "/home/vuaieo/Desktop/UPBGE_29_8_2024/4.3/python/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/vuaieo/Desktop/UPBGE_29_8_2024/4.3/python/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/vuaieo/Desktop/UPBGE_29_8_2024/4.3/python/lib/python3.11/site-packages/parler_tts/modeling_parler_tts.py", line 1773, in forward
outputs = self.model(
^^^^^^^^^^^
File "/home/vuaieo/Desktop/UPBGE_29_8_2024/4.3/python/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/vuaieo/Desktop/UPBGE_29_8_2024/4.3/python/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/vuaieo/Desktop/UPBGE_29_8_2024/4.3/python/lib/python3.11/site-packages/parler_tts/modeling_parler_tts.py", line 1674, in forward
decoder_outputs = self.decoder(
^^^^^^^^^^^^^
File "/home/vuaieo/Desktop/UPBGE_29_8_2024/4.3/python/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/vuaieo/Desktop/UPBGE_29_8_2024/4.3/python/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/vuaieo/Desktop/UPBGE_29_8_2024/4.3/python/lib/python3.11/site-packages/parler_tts/modeling_parler_tts.py", line 1494, in forward
layer_outputs = decoder_layer(
^^^^^^^^^^^^^^
File "/home/vuaieo/Desktop/UPBGE_29_8_2024/4.3/python/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/vuaieo/Desktop/UPBGE_29_8_2024/4.3/python/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/vuaieo/Desktop/UPBGE_29_8_2024/4.3/python/lib/python3.11/site-packages/parler_tts/modeling_parler_tts.py", line 909, in forward
hidden_states, self_attn_weights, present_key_value = self.self_attn(
^^^^^^^^^^^^^^^
File "/home/vuaieo/Desktop/UPBGE_29_8_2024/4.3/python/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/vuaieo/Desktop/UPBGE_29_8_2024/4.3/python/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/vuaieo/Desktop/UPBGE_29_8_2024/4.3/python/lib/python3.11/site-packages/parler_tts/modeling_parler_tts.py", line 773, in forward
key_states, value_states = past_key_value.update(
^^^^^^^^^^^^^^^^^^^^^^
File "/home/vuaieo/Desktop/UPBGE_29_8_2024/4.3/python/lib/python3.11/site-packages/transformers/cache_utils.py", line 364, in update
self.value_cache[layer_idx] = torch.cat([self.value_cache[layer_idx], value_states], dim=-2)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
torch.OutOfMemoryError: CUDA out of memory. Tried to allocate 20.00 MiB. GPU 0 has a total capacity of 11.87 GiB of which 18.69 MiB is free. Including non-PyTorch memory, this process has 10.70 GiB memory in use. Of the allocated memory 8.96 GiB is allocated by PyTorch, and 285.11 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True to avoid fragmentation. See documentation for Memory Management (https://pytorch.org/docs/stable/notes/cuda.html#environment-variables)
Error: Python: Traceback (most recent call last):
File "/home/vuaieo/Desktop/UPBGE_29_8_2024/4.3/python/lib/python3.11/site-packages/torch/utils/_contextlib.py", line 116, in decorate_context
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/home/vuaieo/Desktop/UPBGE_29_8_2024/4.3/python/lib/python3.11/site-packages/parler_tts/modeling_parler_tts.py", line 3461, in generate
outputs = self._sample(
^^^^^^^^^^^^^
File "/home/vuaieo/Desktop/UPBGE_29_8_2024/4.3/python/lib/python3.11/site-packages/transformers/generation/utils.py", line 2932, in _sample
outputs = self(**model_inputs, return_dict=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/vuaieo/Desktop/UPBGE_29_8_2024/4.3/python/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/vuaieo/Desktop/UPBGE_29_8_2024/4.3/python/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/vuaieo/Desktop/UPBGE_29_8_2024/4.3/python/lib/python3.11/site-packages/parler_tts/modeling_parler_tts.py", line 2728, in forward
decoder_outputs = self.decoder(
^^^^^^^^^^^^^
File "/home/vuaieo/Desktop/UPBGE_29_8_2024/4.3/python/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/vuaieo/Desktop/UPBGE_29_8_2024/4.3/python/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/vuaieo/Desktop/UPBGE_29_8_2024/4.3/python/lib/python3.11/site-packages/parler_tts/modeling_parler_tts.py", line 1773, in forward
outputs = self.model(
^^^^^^^^^^^
File "/home/vuaieo/Desktop/UPBGE_29_8_2024/4.3/python/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/vuaieo/Desktop/UPBGE_29_8_2024/4.3/python/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/vuaieo/Desktop/UPBGE_29_8_2024/4.3/python/lib/python3.11/site-packages/parler_tts/modeling_parler_tts.py", line 1674, in forward
decoder_outputs = self.decoder(
^^^^^^^^^^^^^
File "/home/vuaieo/Desktop/UPBGE_29_8_2024/4.3/python/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/vuaieo/Desktop/UPBGE_29_8_2024/4.3/python/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/vuaieo/Desktop/UPBGE_29_8_2024/4.3/python/lib/python3.11/site-packages/parler_tts/modeling_parler_tts.py", line 1494, in forward
layer_outputs = decoder_layer(
^^^^^^^^^^^^^^
File "/home/vuaieo/Desktop/UPBGE_29_8_2024/4.3/python/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/vuaieo/Desktop/UPBGE_29_8_2024/4.3/python/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/vuaieo/Desktop/UPBGE_29_8_2024/4.3/python/lib/python3.11/site-packages/parler_tts/modeling_parler_tts.py", line 909, in forward
hidden_states, self_attn_weights, present_key_value = self.self_attn(
^^^^^^^^^^^^^^^
File "/home/vuaieo/Desktop/UPBGE_29_8_2024/4.3/python/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/vuaieo/Desktop/UPBGE_29_8_2024/4.3/python/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/vuaieo/Desktop/UPBGE_29_8_2024/4.3/python/lib/python3.11/site-packages/parler_tts/modeling_parler_tts.py", line 773, in forward
key_states, value_states = past_key_value.update(
^^^^^^^^^^^^^^^^^^^^^^
File "/home/vuaieo/Desktop/UPBGE_29_8_2024/4.3/python/lib/python3.11/site-packages/transformers/cache_utils.py", line 364, in update
self.value_cache[layer_idx] = torch.cat([self.value_cache[layer_idx], value_states], dim=-2)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
torch.OutOfMemoryError: CUDA out of memory. Tried to allocate 20.00 MiB. GPU 0 has a total capacity of 11.87 GiB of which 18.69 MiB is free. Including non-PyTorch memory, this process has 10.70 GiB memory in use. Of the allocated memory 8.96 GiB is allocated by PyTorch, and 285.11 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True to avoid fragmentation. See documentation for Memory Management (https://pytorch.org/docs/stable/notes/cuda.html#environment-variables)

[vuaieo]

oh ok sorry it looks like it works now afetr restarting blender, the first time i got the error was because first i used other model and then parler tss so i think the model before wasnt deleted from VRam thats why the out of memory error? super cool : ) thankss suepr much : )