Skip to content

Bnb/transformer #73

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
bnb32 wants to merge 74 commits into
base: main
Choose a base branch
from
Open

Bnb/transformer #73

Show file tree
Hide file tree
Changes from 71 commits
Commits
Show all changes
74 commits
Select commit Hold shift + click to select a range
b0aa4b9
update Python version requirement and TensorFlow dependency in pyproj…
bnb32 Apr 19, 2026
67b7907
fix: update Python versions in GitHub Actions workflow
bnb32 Apr 19, 2026
0968996
update code for compliance with keras 3.
bnb32 Apr 19, 2026
6846aaa
just ruff linting
bnb32 Apr 19, 2026
34f24b3
feat: add custom layer serialization tests and enhance layer configur…
bnb32 Apr 19, 2026
cff12c5
Update pyproject.toml
bnb32 Apr 19, 2026
0891fb0
refactor: remove unused Attention and AxialAttentionBlock layers.
bnb32 Apr 19, 2026
2d466bd
refactor: remove unused Attention and SparseAttention layers from tests
bnb32 Apr 19, 2026
eaf3d84
refactor: remove unused test for S3A layer
bnb32 Apr 19, 2026
6233a45
feat: enhance model serialization and deserialization with Keras supp…
bnb32 Apr 20, 2026
4b66f22
Update phygnn/base.py
bnb32 Apr 20, 2026
f889192
removed all the input_shape nonsense. simplified skip connection seri…
bnb32 Apr 20, 2026
573b723
refactor: streamline Keras model loading and remove legacy support
bnb32 Apr 21, 2026
be104cd
massive simplifcation of save / load
bnb32 Apr 22, 2026
36b3d2e
refactor: revert tf_model changes
bnb32 Apr 22, 2026
c6b804b
refactor: simplify save/load methods in CustomNetwork class
bnb32 Apr 22, 2026
abb4c4d
refactor: remove unused from_layers method and relink_skip_connection…
bnb32 Apr 22, 2026
1ab1c5f
refactor: remove unused regex import from handlers.py
bnb32 Apr 22, 2026
4127ebf
minimal keras 3 changes for tf model save / load
bnb32 Apr 22, 2026
7ce0732
chore: update Python version support in pyproject.toml and CI workflow
bnb32 Apr 22, 2026
21612a7
fix: update Python version constraint to 3.14 max
bnb32 Apr 22, 2026
065380f
fix: update tensorflow version constraint to <2.21
bnb32 Apr 22, 2026
faf82ee
attention block for obs assimilation
bnb32 Mar 5, 2026
72667cc
linting
bnb32 Mar 5, 2026
dfee63a
test fix
bnb32 Mar 5, 2026
9005eb4
split out base layer with tokenization and encoding and renamed atten…
bnb32 Mar 6, 2026
f280326
project field deprecated -> workspace
bnb32 Mar 7, 2026
c8d9de5
refactor Sup3rCrossAttention: rename mlp_head to final_proj and updat…
bnb32 Mar 9, 2026
5fd0a0b
chore: remove deprecated linter configurations and update linter work…
bnb32 Mar 9, 2026
5c83143
linting
bnb32 Mar 9, 2026
418977f
fix pos encoding dimensionality
bnb32 Mar 9, 2026
8bd9481
vectorized positional encoding computation
bnb32 Mar 10, 2026
c872ed1
moved pooling before encoding
bnb32 Mar 10, 2026
5d0c83e
linting
bnb32 Mar 10, 2026
c53b9f5
attn and encoding tests for patch_size > 1
bnb32 Mar 10, 2026
58e2dce
Enhance Sup3rCrossAttention and TokenizeEncodeBase layers with patch …
bnb32 Mar 16, 2026
247f034
Refactor TokenizeEncodeBase layer for improved NaN handling and dynam…
bnb32 Mar 19, 2026
77e430d
Fix NaN handling in TokenizeEncodeBase and update Sup3rCrossAttention…
bnb32 Mar 20, 2026
81ed3f5
reverting llm lies
bnb32 Mar 20, 2026
5c0f5b6
Refactor test_double_skip to use set for index checks
bnb32 Mar 20, 2026
40f592e
`use_bias=False` for dense layer tokenization
bnb32 Mar 21, 2026
b10f6d9
Fix mask shape in test_cross_attn_2d for correct dimensionality
bnb32 Mar 21, 2026
2f14d78
Add lat / lon / time based encoding.
bnb32 Mar 22, 2026
6af4838
Update version in pixi.lock and refactor test layers for TokenizeEnco…
bnb32 Mar 23, 2026
1e17621
- Split TokenizeEncode class into Tokenizer and PositionEncoder classes.
bnb32 Mar 29, 2026
a643c56
Enhance MultiHeadAttention to support optional positional encodings f…
bnb32 Mar 29, 2026
89c2425
Refactor MultiHeadAttention and Sup3rCrossAttention to remove dropout…
bnb32 Mar 30, 2026
36f9e90
Refactor PositionEncoder frequency encoding parameters; simplify Sup3…
bnb32 Mar 30, 2026
305f0fa
Add clarification on positional encodings in Sup3rCrossAttention; ref…
bnb32 Mar 31, 2026
436cace
Removed index based positional encoding. Base attention layer uses su…
bnb32 Apr 2, 2026
5167670
Refactor PositionEncoder: remove generic positional encoding method a…
bnb32 Apr 3, 2026
c446fac
Refactor Tokenizer to Embedder; Moved layer norm and mlp from attenti…
bnb32 Apr 18, 2026
aae32c0
Refactor test_pos_encoding_patch_size_gt1_3d: Simplify latitude and l…
bnb32 Apr 18, 2026
d36276e
Refactor MultiHeadAttention: Introduce fused dot-product attention an…
bnb32 Apr 18, 2026
789ff54
Refactor MultiHeadAttention: Enhance Keras mask handling for compatib…
bnb32 Apr 19, 2026
bdc0497
Refactor: Remove FNO layer and associated tests for 2D and 3D data
bnb32 Apr 19, 2026
11974f2
Refactor: Simplify get_config methods and ensure proper layer buildin…
bnb32 Apr 21, 2026
1df3909
Refactor Sup3rTransformerBlock: make sure each layer builds correctly…
bnb32 Apr 21, 2026
1edeb8e
Refactor PatchLayer and Embedder: Enhance initialization with additio…
bnb32 Apr 22, 2026
04a9616
Refactor: massive simplification of save / load
bnb32 Apr 22, 2026
e7632f1
Refactor: Update layer build methods to ensure proper initialization …
bnb32 Apr 22, 2026
eccd0b4
add tf.function decorators to call methods, head_slopes as non traina…
bnb32 Apr 22, 2026
7904fac
Refactor Sup3rTransformerLayerAlibi: Replace sigma with alpha paramet…
bnb32 Apr 25, 2026
de74f0d
Add SwiGLU activation and update TransformerLayer to use RMSNormaliza…
bnb32 Apr 29, 2026
c8fb8c3
removed patch > 1 tests for transformer layers - just doing pixel wis…
bnb32 Apr 29, 2026
595f5e6
Refactor: Ensure proper layer initialization by calling super().build…
bnb32 Apr 29, 2026
fa891ad
Remove unused imports: Embedder and PatchLayer from test_layers.py
bnb32 Apr 30, 2026
11c06c4
Remove unused import of importlib in test_transformer_kwargs.py
bnb32 Apr 30, 2026
23c2de8
Update Python version in GitHub Actions workflow to 3.10
bnb32 May 1, 2026
16d4ae7
Optional layer norms in transformer layers - default off for alibi, s…
bnb32 May 2, 2026
c78c774
optional layer norm for inputs only.
bnb32 May 2, 2026
ecfb656
Refactor PatchLayer and Sup3rTransformerLayer to handle NaN values an…
bnb32 May 6, 2026
ca5c0be
Fix position encoding tests to correctly calculate token counts for p…
bnb32 May 6, 2026
84672e2
Refactor TransformerLayer to use LayerNormalization instead of RMSNor…
bnb32 May 6, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 0 additions & 4 deletions .github/linters/.flake8
View file
Open in desktop

This file was deleted.

Loading
Loading