Compute/memory requirements scripts
Estimations for memory and compute requirements for training large language models.
Merge request reports
Activity
Filter activity
requested review from @vlad_andrei.badoiu1
assigned to @agherghescu2411
added 1 commit
- 2527de22 - Add more information on memory usage for training
added 1 commit
- 345f9a3f - Add more information on memory usage for training
added 1 commit
- f4abadf8 - Add note about splitting a model's layers on multiple GPU's
added 1 commit
- 7b09899e - Fix memory requiremenst, add fp16 activations
added 1 commit
- 62233327 - Fix memory requiremenst, add fp16 activations
added 1 commit
- 178a8a15 - Move optimizer state and saved percentage of fp16 activations to globals
added 1 commit
- c4f7f774 - Move optimizer state and saved percentage of fp16 activations to globals
added 1 commit
- 536de844 - Move optimizer state and saved percentage of fp16 activations to globals
Please register or sign in to reply