Draft: Basis for unit testing

Pull Request Title

Description

Wants to merge: vladb/unit_tests into main

Type of change

• Bug fix
• New feature
• Enhancement
• Documentation update
• Other (specify right below)

Merge request commits

• Basis for unit testing

Related Issues

Screenshots or GIFs

Checklist

• I have tested the code with the changes manually.
• My code follows the project's style guidelines.
• I have documented my code for others to understand.
• I have updated documentation as needed (including README.md, code comments and doc strings).

Reviewer Guidelines

Additional Notes

@mentions

tests/benchmark/test_benchmark.py

@@ -18,7 +18,7 @@ class BenchmarkTest(unittest.TestCase):
         lr_max: float = 1e-4
         grad_clip_norm: float = 1.0
         epochs: int = 1
-        tokenizer_path: str = 'optimus.model'
+        tokenizer_path: str = 'optimus16K-wikitext103.model'
         checkpoints_path: str = 'best_model.pth'
         dim: int = 512
         n_layers: int = 6