Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Mamba fix #123

Merged
merged 33 commits into from
Dec 18, 2024
Merged

Mamba fix #123

merged 33 commits into from
Dec 18, 2024

Conversation

divya-kumari32
Copy link
Collaborator

Cleaned up main_training.py file

daviswer
daviswer approved these changes Dec 17, 2024
View reviewed changes
Copy link
Collaborator

@daviswer daviswer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good - mamba training in separate files, newcluster updates to dataloader (mostly adding AutoHandler). MyPy doesn't like imports but we can force push this if needed and fix later.

divya-kumari32 and others added 20 commits December 17, 2024 18:40
@divya-kumari32 @daviswer
Cosine 0.01 decay
db679b2 
Signed-off-by: Davis Wertheimer <[email protected]>
@daviswer
Add AutoHandler
c60500d 
Signed-off-by: Davis Wertheimer <[email protected]>
@daviswer
Add Auto cfg option for AutoHAndler
40245af 
Signed-off-by: Davis Wertheimer <[email protected]>
@daviswer
Len gets called before open
2cfda44 
Signed-off-by: Davis Wertheimer <[email protected]>
@daviswer
path/filepath typo fix
fb462de 
Signed-off-by: Davis Wertheimer <[email protected]>
@daviswer
Partitioning fix from mup-search
ad5a566 
Signed-off-by: Davis Wertheimer <[email protected]>
@divya-kumari32 @daviswer
Warmup interval change
5aaf1eb 
Signed-off-by: Davis Wertheimer <[email protected]>
@divya-kumari32 @daviswer
Schedule change
496f1b6 
Signed-off-by: Davis Wertheimer <[email protected]>
@divya-kumari32 @daviswer
Constant schedule
715b04c 
Signed-off-by: Davis Wertheimer <[email protected]>
@divya-kumari32 @daviswer
LR schedule change (cool down and constant lr)
2fc0d73 
Signed-off-by: Davis Wertheimer <[email protected]>
@divya-kumari32 @daviswer
Update dataset_utils.py
ab6e3b2 
Added a check for length of doc

Signed-off-by: Davis Wertheimer <[email protected]>
@divya-kumari32 @daviswer
LR schedule change (Warmup + constant)
e3eaa80 
Signed-off-by: Davis Wertheimer <[email protected]>
@divya-kumari32 @daviswer
Update main_training.py
824fda1 
cleanup for main_training.py

Signed-off-by: Davis Wertheimer <[email protected]>
@daviswer
Mirror doc len check into AHandler, fix mypy in autoHandler
70bc786 
Signed-off-by: Davis Wertheimer <[email protected]>
@daviswer
Linting
d48dc5f 
Signed-off-by: Davis Wertheimer <[email protected]>
@daviswer
Further linting
aeb8e61 
Signed-off-by: Davis Wertheimer <[email protected]>
@daviswer
More mypy type fix
bd0f1d3 
Signed-off-by: Davis Wertheimer <[email protected]>
@divya-kumari32 @daviswer
Rename main_training.py to main_training_mamba.py
6ab883b 
Signed-off-by: Davis Wertheimer <[email protected]>
@divya-kumari32 @daviswer
Added main_training_llama.py file
9e6b7f9 
Signed-off-by: Davis Wertheimer <[email protected]>
@divya-kumari32 @daviswer
Rename fms_to_hf.py to fms_to_hf_mamba.py
d8af68b 
Signed-off-by: Davis Wertheimer <[email protected]>
divya-kumari32 and others added 7 commits December 17, 2024 18:40
@divya-kumari32 @daviswer
Added fms_to_hf_llama.py file
6a39d7c 
Signed-off-by: Davis Wertheimer <[email protected]>
@divya-kumari32 @daviswer
Delete fms_fsdp/utils/config_utils.py
ccb1132 
Signed-off-by: Davis Wertheimer <[email protected]>
@divya-kumari32 @daviswer
Added mamba variant 9.8b
dc9e68b 
Signed-off-by: Davis Wertheimer <[email protected]>
@daviswer
Incremental mypy fix
507f5be 
Signed-off-by: Davis Wertheimer <[email protected]>
@daviswer
Fix imports (mypy)
e0b6123 
Signed-off-by: Davis Wertheimer <[email protected]>
@ani300 @daviswer
Rename adapters to work correctly
dacd8f2 
Signed-off-by: Antoni Viros i Martin <[email protected]>
Signed-off-by: Davis Wertheimer <[email protected]>
@ani300 @daviswer
linting
1b60b61 
Signed-off-by: Antoni Viros i Martin <[email protected]>
Signed-off-by: Davis Wertheimer <[email protected]>
@daviswer
Copy link
Collaborator

Tried to add my sign-off to the relevant commits, but looks like Antoni's force push breaks the ability to rebase earlier than that. So... if we end up force pushing make sure to squash all these commits

@lchu6 lchu6 merged commit 0e47e93 into main Dec 18, 2024
2 of 4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@daviswer daviswer daviswer approved these changes

@lchu6 lchu6 Awaiting requested review from lchu6

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@divya-kumari32 @daviswer @lchu6 @ani300