Awesome LLM's 🤩

Here goes any work/research/information related to LLM's that might be deemed useful at some point.

This repo

There are a number of useful markdown files in this repository, which hold information for both newcomers as well as any other interesting new research:

• Intro to AI/ML: useful for anyone not already familiar with AI/ML, or more specifically deep learning, NLP, and LLM's (large language models).
• Research intro: a list of papers which withstood the test of time. Here goes any big/relevant paper from the past, useful for newcomers to read in order to get accustomed with the relevant research of yesterday.
• LLM: related to models' architecture, and a lot of other things (including quantization, prompt engineering, models comparison, benchmarks etc.). This contains stuff related to models that is not code.
• Sytems: related to the lower-level stack of an LLM (architecture implementation, communication between CPU's, GPU's, parallelization, profiling etc.). Contains stuff that is related to code.
• Formal verification: somewhat orthogonal to LLM's, refers to how to formally verify (and specify) the code output of language models.

The rest of the repositories

The rest of the repositories are, in no particular order:

• Awesome LLM: this repo.
• LLaMA: ported source code from the official Meta Research implementation of LLaMA 2, with any modifications added.
• LLaMA Docker images: Dockerfiles for building containers for LLaMA 2; also hosts pre-built images.
• LLaMA fine-tuning demo: code for fine-tuning LLaMA 2.
• Optimus Prime: the home-grown Transformer model, used for training from scratch and learning how it works. Recent features from research are added to it as research progresses.
• Word2Vec: demo of how the Word2Vec embedding algorithm works.
• BenchZoo: bunch of benchmarks to measure communication and computation of our GPU setup