Fix/fp8linear arg mismatch #20
Open
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
This PR resolves FP8Linear signature compatibility issues between different versions of the library that were causing
TypeError: FP8Linear.forward() got an unexpected keyword argumenterrors during inference.Root Cause
The issue stems from mismatched function signatures across different FP8Linear implementations. Attention processors and other modules were calling FP8Linear layers with arguments that aren't always supported across all versions, leading to signature mismatches.
Changes Made
Core Fix
_linear_call()compatibility wrapper inq8_kernels/integration/diffusers.pythat:inspect.signature()to dynamically determine supported parametersUpdated Call Sites
Refactored all FP8Linear call sites to use the compatibility wrapper:
self.to_q()calls in attention processorsself.to_k()calls in attention processorsself.to_v()calls in attention processorsself.to_out[0]()calls for output projectionself.proj()calls in GELU forward functionsself.net[2]()calls in feedforward functionsTesting
tests/test_fp8_signature.py:Affected Lines
q8_kernels/integration/diffusers.py: Updated to use_linear_call()Compatibility Notes
This change maintains backward compatibility with existing FP8Linear implementations while providing forward compatibility with future signature changes. The wrapper automatically adapts to whatever signature the underlying layer supports.
Closes #231