Since only the first process communicates with the user, the keyboard
interrupt will not reach the other processes, hence it will still crash,
whatever we catch or not. A more complex mechanism would be needed to
transmit the user intention to close the app.

Since the tokenizer model is just a 500KB file, we can easily add it
here and not deal with it later.

Previously, the script just crashed, without informing the user about
it.

Fix key-value caching for seqlen != 1 (Issue #899)

Update and add comments about the shape of the key and value
matrices in the attention component. E.g., the second dimension is
of length seqlen + cache_len not seqlen as previously stated.

Update names for consistency with code

Co-authored-by: ruanslv <ruanslv@gmail.com>

Update README.md

This commit fixes a bug in the key-value caching. Currently,
a square attention mask is misapplied to the scores matrix
despite not matching the shape of the scores matrix. This
results in a runtime error. In a correct implementation, the
decoder mask needs to describe how the new seq_len tokens
interact with all the cached tokens. That is, the attention
mask needs to be of shape (seq_len, total_len), indicating how
the token at row i (representing token i + cached_len in the
transformer model) attends to token j. Accordingly, the matrix
needs to mask entries where j > cached_len + i. This patch
horizontally appends (seq_len, cached_len) zeros to an
upper-triangular mask of size (seq_len, seq_len) to form the
(seq_len, total_len) mask.

Correct "bug," typo to "bug", in README.md

Delete FAQ.md

Update README.md

Update issue templates

[closes #858] change "Content Length" to "Context Length  MODEL_CARD.md

Faq updates

made some small fixes and added some context.

Add "--continue" flag to wget for model binary in order to resume dl

Update README.md

Updated the Meta AI mention to just Meta.

Update MODEL_CARD.md

Update FAQ.md