Skip to content

Commit f55a9a1

Browse files
authored
Merge pull request #1785 from Shashank-Tripathi-07/fix/tinytorch-modules-10-20-audit
Fix GPT causal mask convention in module 13 to match module 12's _apply_mask expectation (1-mask convention).
2 parents 73f847e + b9ce5b2 commit f55a9a1

1 file changed

Lines changed: 3 additions & 2 deletions

File tree

tinytorch/src/13_transformers/13_transformers.py

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1354,8 +1354,9 @@ def __call__(self, tokens):
13541354
def _create_causal_mask(self, seq_len):
13551355
"""Create causal mask to prevent attending to future positions."""
13561356
### BEGIN SOLUTION
1357-
# Upper triangular matrix filled with -inf
1358-
mask = np.triu(np.ones((seq_len, seq_len)) * -np.inf, k=1)
1357+
# Lower triangular binary mask: 1=allow (past/present), 0=block (future)
1358+
# _apply_mask in module 12 expects this convention: adder = (1-mask)*MASK_VALUE
1359+
mask = np.tril(np.ones((seq_len, seq_len)))
13591360
return Tensor(mask)
13601361
### END SOLUTION
13611362

0 commit comments

Comments
 (0)