Skip to content

Commit 9f7b43b

Browse files
committed
Update comments in the short_context_recovery method
1 parent 5f6c465 commit 9f7b43b

File tree

1 file changed

+2
-1
lines changed

1 file changed

+2
-1
lines changed

src/main.py

+2-1
Original file line numberDiff line numberDiff line change
@@ -725,7 +725,8 @@ def progressive_extension(
725725
def short_context_recovery(model, data, base_length, lambda_factors_base, n_hat_base):
726726
"""
727727
This function ensures that the model maintains good performance on shorter contexts (4k and 8k)
728-
even after being extended to very long contexts.
728+
even after being extended to very long contexts. It's a crucial step in the LongRoPE approach
729+
to prevent performance degradation on shorter sequences.
729730
730731
Args:
731732
model (nn.Module): Extended LongRoPE model.

0 commit comments

Comments
 (0)