Skip to content

Add naive MultiHeadFFN#43

Open
AntonOresten wants to merge 1 commit intomainfrom
naive-mhf
Open

Add naive MultiHeadFFN#43
AntonOresten wants to merge 1 commit intomainfrom
naive-mhf

Conversation

@AntonOresten
Copy link
Member

I was able to get a forward pass working that was ~20% faster, with no intermediate activations, but we already have the chunked StarGLU, and I would rather wait for something like cuTile in Julia to get actual good performance.

@codecov
Copy link

codecov bot commented Dec 18, 2025

Codecov Report

❌ Patch coverage is 0% with 13 lines in your changes missing coverage. Please review.
✅ Project coverage is 31.20%. Comparing base (2279585) to head (df5d4d6).

Files with missing lines Patch % Lines
src/transformers/MultiHeadFFN.jl 0.00% 13 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main      #43      +/-   ##
==========================================
- Coverage   31.72%   31.20%   -0.53%     
==========================================
  Files          36       37       +1     
  Lines         769      782      +13     
==========================================
  Hits          244      244              
- Misses        525      538      +13     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

Comments