Skip to content

GitLab

Explore

Sign in

Primary navigation

Project

Shape Transformer RWKV
- Activity
- Members
- Labels
- Terraform modules
- Incidents
- Contributor analytics
- Model experiments

Snippets Groups Projects

dc1565de

Commit dc1565de authored 3 years ago by Moritz Ibing

Downloads
- Patches
- Plain Diff

updated heads to allow different embedding dimension and variable number of layers

parent 1194c4a4

No related branches found

No related tags found

No related merge requests found

Changes 12

Hide whitespace changes

Inline Side-by-side

Showing

modules/generative_head/composite_head_A.py 33 additions, 29 deletions

modules/generative_head/composite_head_A.py
modules/generative_head/composite_head_B.py 15 additions, 13 deletions

modules/generative_head/composite_head_B.py
modules/generative_head/composite_head_C.py 14 additions, 12 deletions

modules/generative_head/composite_head_C.py
modules/generative_head/convolution_head_A.py 36 additions, 9 deletions

modules/generative_head/convolution_head_A.py
modules/generative_head/double_substitution_head.py 73 additions, 29 deletions

modules/generative_head/double_substitution_head.py
modules/generative_head/head_factory.py 17 additions, 13 deletions

modules/generative_head/head_factory.py
modules/generative_head/linear_head.py 12 additions, 2 deletions

modules/generative_head/linear_head.py
modules/generative_head/substitution_head.py 53 additions, 17 deletions

modules/generative_head/substitution_head.py
modules/shape_transformer.py 4 additions, 1 deletion

modules/shape_transformer.py
modules/token_embedding/embedding_factory.py 3 additions, 3 deletions

modules/token_embedding/embedding_factory.py
modules/utils/embedding.py 7 additions, 10 deletions

modules/utils/embedding.py
modules/utils/linear.py 1 addition, 1 deletion

modules/utils/linear.py

with 268 additions and 139 deletions

Loading

0% Loading or .

You are about to add 0 people to the discussion. Proceed with caution.

Finish editing this message first!

Please register or sign in to comment