Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Possible changes for biobert config changes #655

Draft
wants to merge 5 commits into
base: main
Choose a base branch
from
Draft

Possible changes for biobert config changes #655

wants to merge 5 commits into from

Conversation

pstjohn
Copy link
Collaborator

@pstjohn pstjohn commented Jan 24, 2025

No description provided.

pstjohn
pstjohn commented Jan 24, 2025
View reviewed changes
sub-packages/bionemo-llm/src/bionemo/llm/model/biobert/model.py
Comment on lines -155 to -157
transformer_layer_spec: ModuleSpec,
vocab_size: int,
max_sequence_length: int,
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nearly all these arguments that we're passing around are also in the config object, which we've mis-typed as a vanilla TransformerConfig

@pstjohn
add back transformer engine install
9db848a 
Signed-off-by: Peter St. John <[email protected]>
@pstjohn
fix docstring
bcc8328 
Signed-off-by: Peter St. John <[email protected]>
@pstjohn
start to remove extraneous kwargs
aba09fa 
Signed-off-by: Peter St. John <[email protected]>
@pstjohn
try moving to heirarchical config
6131a81 
Signed-off-by: Peter St. John <[email protected]>
@pstjohn
add bionemo-llm test
fad7408 
Signed-off-by: Peter St. John <[email protected]>
@pstjohn pstjohn force-pushed the pstjohn/clean-up-biobert branch from 206f124 to fad7408 Compare January 24, 2025 23:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@jstjohn jstjohn Awaiting requested review from jstjohn jstjohn will be requested when the pull request is marked ready for review jstjohn is a code owner

@skothenhill-nv skothenhill-nv Awaiting requested review from skothenhill-nv skothenhill-nv will be requested when the pull request is marked ready for review skothenhill-nv is a code owner

@jomitchellnv jomitchellnv Awaiting requested review from jomitchellnv jomitchellnv will be requested when the pull request is marked ready for review jomitchellnv is a code owner

@farhadrgh farhadrgh Awaiting requested review from farhadrgh farhadrgh will be requested when the pull request is marked ready for review farhadrgh is a code owner

@sichu2023 sichu2023 Awaiting requested review from sichu2023 sichu2023 will be requested when the pull request is marked ready for review sichu2023 is a code owner

@dorotat-nv dorotat-nv Awaiting requested review from dorotat-nv dorotat-nv will be requested when the pull request is marked ready for review dorotat-nv is a code owner

@malcolmgreaves malcolmgreaves Awaiting requested review from malcolmgreaves malcolmgreaves will be requested when the pull request is marked ready for review malcolmgreaves is a code owner

@ohadmo ohadmo Awaiting requested review from ohadmo ohadmo will be requested when the pull request is marked ready for review ohadmo is a code owner

@trvachov trvachov Awaiting requested review from trvachov trvachov will be requested when the pull request is marked ready for review trvachov is a code owner

At least 2 approving reviews are required to merge this pull request.

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@pstjohn