Skip to content

Conversation

@shanto12
Copy link

Summary

This PR resolves FP8Linear signature compatibility issues between different versions of the library that were causing TypeError: FP8Linear.forward() got an unexpected keyword argument errors during inference.

Root Cause

The issue stems from mismatched function signatures across different FP8Linear implementations. Attention processors and other modules were calling FP8Linear layers with arguments that aren't always supported across all versions, leading to signature mismatches.

Changes Made

Core Fix

  • Added _linear_call() compatibility wrapper in q8_kernels/integration/diffusers.py that:
    • Uses inspect.signature() to dynamically determine supported parameters
    • Filters out unsupported keyword arguments before calling the layer
    • Preserves dtype information when supported
    • Provides unified calling interface for all FP8Linear variants

Updated Call Sites

Refactored all FP8Linear call sites to use the compatibility wrapper:

  • self.to_q() calls in attention processors
  • self.to_k() calls in attention processors
  • self.to_v() calls in attention processors
  • self.to_out[0]() calls for output projection
  • self.proj() calls in GELU forward functions
  • self.net[2]() calls in feedforward functions

Testing

  • Added comprehensive test suite in tests/test_fp8_signature.py:
    • Mock FP8Linear classes with different signatures
    • Tests for argument filtering behavior
    • Verification that processor factory functions use the wrapper
    • Call site compatibility tests

Affected Lines

  • Lines 189, 196, 201, 237 in q8_kernels/integration/diffusers.py: Updated to use _linear_call()
  • Lines 294, 305 in processor functions: Updated GELU and feedforward calls

Compatibility Notes

This change maintains backward compatibility with existing FP8Linear implementations while providing forward compatibility with future signature changes. The wrapper automatically adapts to whatever signature the underlying layer supports.

Closes #231

…placeholder

Added compatibility wrapper for FP8Linear layers to unify calling patterns and filter unsupported arguments. Updated attention processor and forward methods to utilize the new wrapper for linear calls.
This commit adds a comprehensive test suite for the FP8Linear compatibility wrapper and processor factory functions, ensuring correct handling of various argument signatures and filtering of unsupported arguments.
@shanto12
Copy link
Author

📢 Call for Testers from Issue #231

Hi everyone tracking the FP8Linear signature compatibility issues in #231! This PR implements a solution that should resolve the TypeError: FP8Linear.forward() got an unexpected keyword argument errors. We'd appreciate your help testing this fix.

🔧 Technical Approach

This PR introduces a compatibility wrapper (_linear_call()) that:

  • Dynamically inspects FP8Linear signatures using inspect.signature()
  • Automatically filters unsupported keyword arguments before calling the layer
  • Preserves dtype information when the implementation supports it
  • Provides a unified interface across all FP8Linear variants

✅ Backward Compatibility

The wrapper maintains full backward compatibility with existing FP8Linear implementations while providing forward compatibility with future signature changes. It adapts automatically to whatever signature the underlying layer supports—no breaking changes to existing code.

🧪 Quick Testing Steps

To test this branch:

# Clone and checkout this PR branch
git clone https://github.com/shanto12/LTX-Video-Q8-Kernels.git
cd LTX-Video-Q8-Kernels
git checkout fix/fp8linear-arg-mismatch

# Install in development mode
pip install -e .

# Run your inference workflow that previously failed
# The wrapper should handle signature mismatches automatically

📊 Test Coverage

The PR includes comprehensive tests in tests/test_fp8_signature.py covering:

  • Argument filtering behavior
  • Multiple FP8Linear signature variants
  • All affected call sites (attention processors, GELU, feedforward)

Please report any issues or successful tests here. Your feedback is invaluable for getting this merged! 🙏

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant