Codex
No estimate
No hardware requirements for this model
The weights for this model have not been published, so it cannot be downloaded or run on your own hardware at any size. It is reachable only through its provider, and no graphics card changes that.
On record
Full specification
Everything on record for this model. Most of it describes how it was trained rather than how it runs — useful context for judging how much work went into it, and how it compares with models built at a different scale.
Origin
Who built this model, where, and when it was published.
- Organisation
- OpenAI
- Organisation type
- Industry
- Country
- United States of America
- Published
- 7 July 2021
- Authors
- Mark Chen , Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, Alex Ray, Raul Puri, Gretchen Krueger, Michael Petrov, Heidy Khlaaf, Girish Sastry, Pamela Mishkin, Brooke Chan, Scott Gray, Nick Ryder, Mikhail Pavlov, Alethea Power, Lukasz Kaiser, Clemens Winter, Philippe Tillet, Felipe Petroski Such, Dav…
What it does
The problem areas the model was built for. A model can carry several of each.
- Domain
- Language
- Task
- Code autocompletion
- Approach
- Self-supervised learning
- Base model
- GPT-3 13B
Size
How large the model is and how much data it was trained on. Parameters are the figure that decides whether it fits on a given graphics card.
- Parameters
- 12B
- Training data
- 52,788,000,000 tokens
"With just a single sample, a 12B parameter Codex solves 28.8% of these problems, and a 300M parameter Codex solves 13.2% of these problems"
"Our training dataset was collected in May 2020 from 54 million public software repositories hosted on GitHub, containing 179 GB of unique Python files under 1 MB. We filtered out files which were likely auto-generated, had average line length greater than 100, had maximum line length greater than 1000, or contained a small percentage of alphanumeric characters. After filtering, our final dataset totaled 159 GB." 1 GB ~ 200M words
Training compute
The arithmetic performed to train the model, measured in floating-point operations. It is a measure of what the training run cost, not of how fast the finished model answers you.
- Training compute
- 7.3 × 10²² FLOP
"The original training of GPT-3-12B consumed hundreds of petaflop/sdays of compute, while fine-tuning it to create Codex-12B consumed a similar amount of compute." 1 PFLOP/s-day = 8.64e19 FLOPs. "Hundreds" is likely between 200 and 900, geometric mean = 425. 2 * 425 * 8.64e19 = 7.344e22
Availability
Whether you can obtain the model and run it on your own hardware, which is what decides if any of the graphics-card figures on this page apply.
- Weights
- Closed — provider access only
- Model access
- API access
- Training code
- Unreleased
Codex was available via the OpenAI API from announcement: https://openai.com/index/openai-codex/ It is still available via the Research Access Program. https://x.com/OfficialLoganK/status/1638336152800206858
How it is classified
Labels the source dataset applies when tracking notable models, and how confident it is in the entry.
- Foundation model
- Yes
- Why it is tracked
- Significant use,Discretionary
- Record confidence
- Likely
- Citations
- 9,577
Sources
Where this record came from and when it was last checked.
- Reference
- Evaluating Large Language Models Trained on Code
- Last updated
- 25 May 2026
What the numbers mean
Background
Codex was published by OpenAI, in United States of America, in July 2021. The organisation is categorised as industry.
It works in Language, and is recorded as doing code autocompletion.
It builds on GPT-3 13B, which is why it shares that model's general shape and size.
Its weights were never published, so it can only be reached through its provider. No graphics card changes that.
What went into building it
The training run consumed about 7.3 × 10²² FLOP. That figure describes the cost of creating it and has no bearing on how quickly it generates text.
The training set ran to roughly 52,788,000,000 tokens.
Its inclusion criterion is significant use,Discretionary.
Answers
Codex — common questions
How many parameters does Codex have?
Codex has 12B parameters. "With just a single sample, a 12B parameter Codex solves 28.8% of these problems, and a 300M parameter Codex solves 13.2% of these problems". That figure is the total, and it is what decides how much memory the model needs — roughly half a gigabyte per billion at the compression most people use.
Who created Codex?
Codex was published by OpenAI, based in United States of America, categorised as industry.
When was Codex released?
Codex was published in July 2021. Capability per parameter has improved considerably since, so a newer model of the same size is often the better use of the same hardware.
What is Codex used for?
Codex works in Language, and is recorded as handling code autocompletion. These are the areas it was designed around; they describe intent rather than a hard boundary.
How much compute was used to train Codex?
Around 7.3 × 10²² FLOP. That measures what producing the model cost and says nothing about how quickly it answers once trained — inference speed comes from memory bandwidth, not from the training budget.
What GPU do I need to run Codex?
None. Codex is a closed model — its weights were never published, so it cannot be downloaded or run on your own hardware at any price. It is reachable only through its provider.
Is Codex open source?
No. Codex has not had its weights published, so it exists only as a service controlled by its owner.
The other direction
Looking at it from the other side?
This page starts from the model. If you already own a card and want to know everything it will run, start from the hardware instead.