Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Infrastructure for training large models #27

Open
jack89roberts opened this issue May 22, 2024 · 1 comment · Fixed by #34
Open

Infrastructure for training large models #27

jack89roberts opened this issue May 22, 2024 · 1 comment · Fixed by #34
Assignees
@msmoore
Milestone
Milestone 2

Comments

@jack89roberts
Copy link
Contributor

jack89roberts commented May 22, 2024 •
edited
Loading

e.g. with reference to deep speed code in TOFU codebase

What type of training? (full fine-tuning vs. PEFT etc.)
@jack89roberts jack89roberts added this to the Milestone 2 milestone May 22, 2024
@jack89roberts jack89roberts changed the title Large models Infrastructure for training large models May 22, 2024
@jack89roberts
Copy link
Contributor Author

Cache directories (models, datasets, WandB, ...)

@msmoore msmoore self-assigned this Jul 26, 2024
@msmoore msmoore linked a pull request Jul 26, 2024 that will close this issue
Add model cache_dir config to Baskerville scripts and related config #34
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@msmoore msmoore

Labels
None yet
Projects
None yet
Milestone
Milestone 2
2 participants
@msmoore @jack89roberts