Weights

Weights, often used interchangeably with parameters, are the numbers inside a neural network that determine how strongly each unit influences the next. Training consists of adjusting them, and once training is done, the weights are the model: a file of numbers that, run by software, produces its behavior.

Counting parameters

The number of weights became the most quoted measure of a model’s size. OpenAI’s GPT-2 had 1.5 billion parameters in 2019; GPT-3 had 175 billion in 2020. Meta’s largest Llama 3.1 model, released in 2024, has 405 billion. Some models use a mixture-of-experts design that activates only part of their weights for each token: DeepSeek-V3 has 671 billion parameters in total, of which about 37 billion are used for each token. OpenAI, Google and Anthropic do not officially disclose the parameter counts of their leading models; OpenAI stopped with GPT-4 in 2023, citing competition and safety. Size is not the same as capability: the 2022 Chinchilla study showed that training data matters as much as size – a 70-billion-parameter model trained on four times more data beat larger models trained with the same compute (see Scaling Laws).

What a weights file is

A model’s weights are stored as numbers with a fixed precision, commonly 16 bits each, so a 405-billion-parameter model takes up roughly 800 gigabytes. Compressing them to fewer bits, a technique called quantization, lets large models run on smaller hardware at some cost in quality. The weights contain no readable rules, and training documents are not stored in them as files; what a model knows is spread across all of them, which is why interpretability research is needed to understand it. Models can nonetheless memorize and reproduce some training text, and courts disagree on whether that makes the weights a copy: a Munich court said yes for song lyrics in 2025, while a London court found no copies in an image model.

Open, closed and stolen

Because the weights are the model, whoever has them can run, copy and modify it. Companies that publish them release open-weights models; others keep them private and offer access only through apps and interfaces. Keeping frontier weights secure has become a policy issue: a 2024 RAND report described attackers ranging from opportunistic criminals to highly resourced nation-state operations, and the security levels needed to defend against them – which, against the most capable states, will take years of further investment. The EU AI Act requires adequate cybersecurity for the most capable general-purpose models, California’s SB 53 requires large developers to publish and follow their practices for securing unreleased weights, and several companies’ safety policies set security levels for them.

Why it matters

For frontier models, weights are where the cost of AI turns into an asset: months of computing and up to hundreds of millions of dollars condensed into a file. That file can be shared to spread capability widely or guarded to control it, and the choice between the two is one of the central policy questions of AI. Knowing that a model is, at bottom, its weights helps make sense of debates about open models, model theft, compute governance and what it would even mean to “shut down” an AI.

Written with Claude, made by Anthropic, which does not publish its models’ weights and has taken positions in the policy debate over open-weights models.

Related

Neural Network · Open Weights · Scaling Laws · Pre-training · Inference · Compute Governance

Sources: OpenAI: GPT-2: 1.5B release (November 5, 2019) · Tom B. Brown et al.: Language Models are Few-Shot Learners (arXiv) (2020) · OpenAI: GPT-4 Technical Report (arXiv) (March 2023) · Meta: Introducing Llama 3.1: Our most capable models to date (July 23, 2024) · DeepSeek-AI: DeepSeek-V3 Technical Report (arXiv) (December 2024) · Jordan Hoffmann et al.: Training Compute-Optimal Large Language Models (arXiv) (March 2022) · Taylor Wessing: GEMA v OpenAI (November 2025) · Latham & Watkins: Getty Images v Stability AI (November 2025) · Sella Nevo et al.: Securing AI Model Weights: Preventing Theft and Misuse of Frontier Models (RAND) (May 30, 2024) · EU AI Act, Article 55 · Mayer Brown: California enacts SB 53 (October 2025) · Anthropic: Activating ASL-3 protections (May 22, 2025)

Leave a Reply

Your email address will not be published. Required fields are marked *