Skip to content
Snippets Groups Projects

Draft: Add overview of E2E training

Open Draft: Add overview of E2E training
vladb/e2e_overview into main
Open Vlad-Andrei BĂDOIU (78692) requested to merge vladb/e2e_overview into main
Compare
and
Show latest version
1 file
+ 1
− 0
Compare changes
  • Side-by-side
  • Inline
doc/overview.md
+ 1
− 0
@@ -35,6 +35,7 @@ necessary from two to one per layer
* gradient accumulation
* z-loss - improves stability
* weight decay (AdamW)
* Flash attention
Optimizations:
* Alternative training objectives