Llama 3.1 8B

Llama 3.1 8B is an open-weight language model developed by Meta, released on 23 July 2024 as part of the Llama 3.1 family. It is an auto-regressive, decoder-only Transformer model with 8 billion parameters. The model was trained on roughly 15 trillion tokens of publicly available data, with a knowledge cutoff of December 2023.

The architecture uses Grouped-Query Attention (GQA) for more efficient inference: 32 layers, a hidden size of 4096, 32 attention heads and 8 key-value heads (head_dim 128). The 14336-wide FFN uses a SwiGLU activation, normalization is RMSNorm, and positions are encoded with RoPE (theta 500000) using llama3-type scaling. The vocabulary contains 128256 tokens and the context length is 128K (131072 positions). The model is multilingual, officially supporting English, German, French, Italian, Portuguese, Hindi, Spanish and Thai.

The weights are distributed under the Llama 3.1 Community License, which permits commercial use with specific restrictions. The pretrained base model can be downloaded and run locally, in the cloud or on device, and can be fine-tuned. In BF16 it requires about 16 GB of VRAM, while 4-bit quantization lets it run on consumer GPUs and high-end laptops.