Error Integrating Trained LoRA Model into ComfyUI (ValueError & KeyError: 'lora_up.weight')

[norbob]

Hello everyone,

I successfully trained a LoRA model following the provided instructions. The base model I used is CogVideoX-Fun-V1.1-5b-Control, and my trained LoRA model consists of two .safetensors files.

I am attempting to integrate this LoRA model into ComfyUI for further use, but the integration fails. I have tried the following methods:

Loading the first part of the LoRA model individually.
Merging the two .safetensors files into a single merged.safetensors file and loading this merged file.

In both cases, I encounter an error.

Error Message from ComfyUI Log:

ComfyUI Error Report

Error Details

• Node ID: 449
• Node Type: DownloadAndLoadCogVideoModel
• Exception Type: ValueError
• Exception Message: Can't recognize LoRA /workspace/ComfyUI/models/CogVideo/loras/S1DI/checkpoint-50/transformer/merged_model.safetensors

Stack Trace

File "/workspace/ComfyUI/execution.py", line 323, in execute
output_data, output_ui, has_subgraph = get_output_data(obj, input_data_all, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb)

File "/workspace/ComfyUI/execution.py", line 198, in get_output_data
return_values = _map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb)

File "/workspace/ComfyUI/execution.py", line 169, in _map_node_over_list
process_inputs(input_dict, i)

File "/workspace/ComfyUI/execution.py", line 158, in process_inputs
results.append(getattr(obj, func)(**inputs))

File "/workspace/ComfyUI/custom_nodes/ComfyUI-CogVideoXWrapper/model_loading.py", line 335, in loadmodel
raise ValueError(f"Can't recognize LoRA {l['path']}")

KeyError: 'lora_up.weight'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/workspace/ComfyUI/execution.py", line 323, in execute
output_data, output_ui, has_subgraph = get_output_data(obj, input_data_all, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb)

File "/workspace/ComfyUI/execution.py", line 198, in get_output_data
return_values = _map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb)

File "/workspace/ComfyUI/execution.py", line 169, in _map_node_over_list
process_inputs(input_dict, i)

File "/workspace/ComfyUI/execution.py", line 158, in process_inputs
results.append(getattr(obj, func)(**inputs))

File "/workspace/ComfyUI/custom_nodes/ComfyUI-CogVideoXWrapper/model_loading.py", line 335, in loadmodel
raise ValueError(f"Can't recognize LoRA {l['path']}")
ValueError: Can't recognize LoRA /workspace/ComfyUI/models/CogVideo/loras/S1DI/checkpoint-50/transformer/merged_model.safetensors

System Information:

ComfyUI Version: v0.3.7-32-g1b3a650
OS: Linux
Python Version: 3.10.12
PyTorch Version: 2.5.1+cu121
GPU: NVIDIA H100 PCIe

Steps to Reproduce:

Training the LoRA Model:
    Use CogVideoX-Fun-V1.1-5b-Control as the base model.
    Train the LoRA model following the official instructions.
    The training results in two .safetensors files.

Integrating the LoRA Model into ComfyUI:
    Option 1: Load the first part of the LoRA model individually.
    Option 2: Merge the two .safetensors files into a single merged.safetensors file and load this merged file.

Execute the Workflow in ComfyUI.

Observe the error message in the log.

Expected Behavior:

The LoRA model should be successfully loaded into ComfyUI and function without errors.

Actual Behavior:

Loading the LoRA model fails with the error message:

ValueError: Can't recognize LoRA /workspace/ComfyUI/models/CogVideo/loras/S1DI/checkpoint-50/transformer/merged_model.safetensors

Additionally, a KeyError occurs:

KeyError: 'lora_up.weight'

This suggests that the model may not contain the expected keys or that the structure of the .safetensors files does not align with ComfyUI's expectations.

Question:

Does anyone have an idea of what might be causing this issue or how to resolve it? Are there specific requirements when loading LoRA models into ComfyUI that I might have overlooked? Specifically, what could be the cause of the KeyError: 'lora_up.weight'?

Thank you in advance for your assistance!

Attached:

Full log and screenshot of workflow in comfyui

log.txt

[norbob]

My dataset consists of images. Is that a problem? Can I even use images for training CogVideo Loras? I assumed so, since JPGs are listed in the README:

📦 project/
├── 📂 datasets/
│ ├── 📂 internal_datasets/
│ ├── 📂 train/
│ │ ├── 📄 00000001.mp4
│ │ ├── 📄 00000002.jpg
│ │ └── 📄 .....
│ └── 📄 json_of_internal_datasets.json