[Proposal] Logit Lens Analysis for Gemma 2B (Ref #154) #1090
Description
Description of the feature request:
I would like to contribute a notebook implementing the Logit Lens analysis requested in Issue #154.
Plan:
Tools: Python and TransformerLens (as suggested in the request).
Content:
Load Gemma 2B.
Decode hidden states at intermediate layers.
Visualize probability distributions to illustrate information flow.
This addresses the first part of the request. Please let me know if this approach aligns with the cookbook's standards so I can begin drafting.
What problem are you trying to solve with this feature?
I would like to contribute a notebook implementing the Logit Lens analysis requested in Issue #154.
Plan:
Tools: Python and TransformerLens (as suggested in the request).
Content:
Load Gemma 2B.
Decode hidden states at intermediate layers.
Visualize probability distributions to illustrate information flow.
This addresses the first part of the request. Please let me know if this approach aligns with the cookbook's standards so I can begin drafting.
Any other information you'd like to share?
No response