Skip to content

Commit

Permalink
piano samples
Browse files Browse the repository at this point in the history
  • Loading branch information
lucidrains authored Jun 18, 2022
1 parent ced2b84 commit bfa7526
Showing 1 changed file with 2 additions and 0 deletions.
2 changes: 2 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,8 @@

Implementation of <a href="https://arxiv.org/abs/2202.07765">Perceiver AR</a>, Deepmind's new long-context attention network based on Perceiver architecture, in Pytorch.

<a href="https://storage.googleapis.com/perceiver-ar/index.html">Generated piano samples</a>

I am building this out of popular demand, not because I believe in the architecture. As someone else puts it succinctly, this is equivalent to an encoder / decoder transformer architecture where the encoder has 0 layers (and the decoder cross attention is restricted to 1 layer)

However, the experimental results they provided are still worthwhile and I'll build it out so students and researchers alike can explore along this avenue.
Expand Down

0 comments on commit bfa7526

Please sign in to comment.