Skip to content
Snippets Groups Projects
Select Git revision
  • main default protected
  • feature/docs
  • feature/models
  • feature/scripts
  • vladb/e2e_overview
5 results
Compare
Name Last commit Last update
doc
scripts/memory_compute_estimations
README.md
README.md

Awesome LLM's 🤩

Here goes any work/research/information related to LLM's that might be deemed useful at some point.

This repo

There are a number of useful markdown files in this repository, which hold information for both newcomers as well as any other interesting new research:

  • Intro to AI/ML: useful for anyone not already familiar with AI/ML, or more specifically deep learning, NLP, and LLM's (large language models).
  • Research intro: a list of papers which withstood the test of time. Here goes any big/relevant paper from the past, useful for newcomers to read in order to get accustomed with the relevant research of yesterday.
  • LLM: related to models' architecture, and a lot of other things (including quantization, prompt engineering, models comparison, benchmarks etc.). This contains stuff related to models that is not code.
  • Sytems: related to the lower-level stack of an LLM (architecture implementation, communication between CPU's, GPU's, parallelization, profiling etc.). Contains stuff that is related to code.
  • Formal verification: somewhat orthogonal to LLM's, refers to how to formally verify (and specify) the code output of language models.
  • Datasets for training: Available online datasets used to train (open-source and other) language models

There are also a number of scripts to calculate various things, or otherwise interact in some way with the knowledge described in the docs:

  • Memory and compute estimations: Estimates and requirements for memory and compute for training large language models, given a number of tunable hyperparameters

The rest of the repositories

The rest of the repositories are, in no particular order:

  • Awesome LLM: this repo.
  • LLaMA: ported source code from the official Meta Research implementation of LLaMA 2, with any modifications added.
  • LLaMA Docker images: Dockerfiles for building containers for LLaMA 2; also hosts pre-built images.
  • LLaMA fine-tuning demo: code for fine-tuning LLaMA 2.
  • Optimus Prime: the home-grown Transformer model, used for training from scratch and learning how it works. Recent features from research are added to it as research progresses.
  • Word2Vec: demo of how the Word2Vec embedding algorithm works.
  • BenchZoo: bunch of benchmarks to measure communication and computation of our GPU setup