Skip to content

Releases: feifeibear/long-context-attention

v0.3 released on 27th August 2024!

27 Aug 08:13
@feifeibear feifeibear
v0.3
4bfd585
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Learn about vigilant mode.
Compare
Choose a tag to compare
v0.3 released on 27th August 2024!

upgrade flash_attn >= 2.6.0

Assets 2
Loading

v0.2 released on 24th June 2024!

25 Jun 03:07
@feifeibear feifeibear
v0.2
13f5af8
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Learn about vigilant mode.
Compare
Choose a tag to compare
v0.2 released on 24th June 2024!
  1. Ulysses supports T4 and V100.
  2. Updates some directory structures.
Assets 2
Loading

v0.1

11 Apr 08:32
@feifeibear feifeibear
v0.1
80c8b46
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Learn about vigilant mode.
Compare
Choose a tag to compare
v0.1

Sequence parallel attention adopting a hybrid ulysses and ring attention approach.
Support GQA
Support QKV packed.

Assets 2
Loading