[Feature] Log-linear Sparse Attention

[SingleZombie]

Motivation

We present Log-linear Sparse Attention (LLSA), an efficient and effective sparse attention. We want to contribute our code to this repo. Could the maintainers give us some guides (any similar PRs, minimum files to add)? Thank you.

Related resources

GitHub: https://github.com/SingleZombie/LLSA

[SolitaryThinker]

Hi, thanks for your interest! Please also join our slack channel and directly ask any questions!

here are some example PRs adding new attention backends to fastvideo:
vmoba: #778
sageattn3: #815

We recently switched to compiling all kernels under fastvideo-kernel package so please add your kernels there. We will help you update the README with your docs as well as citation. We'll soon be adding a more detailed attention page in fastvideo's documentation as well.

[SolitaryThinker]

Hi @SingleZombie, we've finished updating our kernels, here's a new doc detailing instructions on adding a new attention backend. https://hao-ai-lab.github.io/FastVideo/attention/developer/

Please let me know if you have any questions