MedBERT
No estimate
No hardware requirements for this model
The weights for this model have not been published, so it cannot be downloaded or run on your own hardware at any size. It is reachable only through its provider, and no graphics card changes that.
On record
Full specification
Everything on record for this model. Most of it describes how it was trained rather than how it runs — useful context for judging how much work went into it, and how it compares with models built at a different scale.
Origin
Who built this model, where, and when it was published.
- Organisation
- Peng Cheng Laboratory,University of Texas at Houston
- Organisation type
- Academia,Academia
- Country
- China, United States of America
- Published
- 20 May 2021
- Authors
- Laila Rasmy, Yang Xiang, Ziqian Xie, Cui Tao, Degui Zhi
What it does
The problem areas the model was built for. A model can carry several of each.
- Domain
- Medicine
- Task
- Medical diagnosis, Text classification, Prediction of hospital stay duration, Prediction of diabetic heart failure (DHF), Prediction of onset of pancreatic cancer (PaCa)
- Numerical format
- FP32
Size
How large the model is and how much data it was trained on. Parameters are the figure that decides whether it fits on a given graphics card.
- Parameters
- 17M
- Training data
- 14,587,212,800 tokens
- Epochs
- 50.5
17M from "This is possibly due to the fact that the untrained Med-BERT is an over-parameterized model (around 17 million parameters) with a huge number of configurations, so it might overfit to the training data"
data about 28M patients "Our pretraining cohort for Med-BERT is consisting of 28 million patients extracted from Cerner"
Training compute
The arithmetic performed to train the model, measured in floating-point operations. It is a measure of what the training run cost, not of how fast the finished model answers you.
- Training compute
- 9.5 × 10¹⁸ FLOP
- How it was established
- Hardware
flops = (1) * (3.13e13) * (24*7 * 3600) * (0.5) = 9.47e18 (num gpu) * (peak flops) * (time in seconds) * (assumed utilization rate) I assume higher utilization rate, because only 1 GPU is used. Citation from the text: "We used a single Nvidia Tesla V100GPU of 32 GB graphics memory capacity, and we trained the model for a week for more than 45 million steps, for which each step consists of 32 patients (batch size)." - page 11 Note that public code appears not to make use of the tensor core speed…
The training run
What it physically took to train: which chips, how many, for how long, and what that drew from the wall.
- Training hardware
- NVIDIA Tesla V100 DGXS 32 GB
- Chips used
- 1
- Chip-hours
- 168
- Wall-clock time
- 168 hours (7 days)
- Power draw
- 278 W
- Compute cost
- $62
"We used a single Nvidia Tesla V100GPU of 32 GB graphics memory capacity, and we trained the model for a week for more than 45 million steps, for which each step consists of 32 patients (batch size)." - page 11
Availability
Whether you can obtain the model and run it on your own hardware, which is what decides if any of the graphics-card figures on this page apply.
- Weights
- Closed — provider access only
- Model access
- Unreleased
- Training code
- Open source
Apache 2 https://github.com/ZhiGroup/Med-BERT training guide: https://github.com/ZhiGroup/Med-BERT/tree/master/Pretraining%20Code "Initially we really hoped to share our models but unfortunately, the pre-trained models are no longer sharable. According to SBMI Data Service Office: "Under the terms of our contracts with data vendors, we are not permitted to share any of the data utilized in our publications, as well as large models derived from those data."
How it is classified
Labels the source dataset applies when tracking notable models, and how confident it is in the entry.
- Why it is tracked
- SOTA improvement
- Record confidence
- Likely
- Citations
- 835
"This work is the first demonstration of significantly boosted performance over state-of-the-art methods on multiple clinical tasks with phenotyped cohorts." Table 4
Sources
Where this record came from and when it was last checked.
- Reference
- Med-BERT: pretrained contextualized embeddings on large-scale structured electronic health records for disease prediction
- Last updated
- 1 January 2026
What the numbers mean
About this model
MedBERT was published by Peng Cheng Laboratory,University of Texas at Houston, in China, in May 2021. academia,Academia is the category the publisher falls under.
It works in Medicine, and is recorded as doing medical diagnosis, Text classification, Prediction of hospital stay duration, Prediction of diabetic heart failure (DHF), Prediction of onset of pancreatic cancer (PaCa).
This is a closed model: the trained values stayed with whoever produced them, and there is no local version to run.
How it was trained
Training it took roughly 9.5 × 10¹⁸ FLOP of computation, on NVIDIA Tesla V100 DGXS 32 GB — a measure of what producing the model cost, not of how fast it answers.
Around 14,587,212,800 tokens went into training it.
The reason it appears in this catalogue at all is sOTA improvement.
Answers
MedBERT — common questions
Is MedBERT open source?
No. MedBERT has not had its weights published, so it exists only as a service controlled by its owner.
How many parameters does MedBERT have?
MedBERT has 17M parameters. 17M from "This is possibly due to the fact that the untrained Med-BERT is an over-parameterized model (around 17 million parameters) with a huge number of configurations, so it might overfit to the training data". That figure is the total, and it is what decides how much memory the model needs — roughly half a gigabyte per billion at the compression most people use.
Who created MedBERT?
MedBERT was published by Peng Cheng Laboratory,University of Texas at Houston, based in China, categorised as academia,Academia.
When was MedBERT released?
MedBERT was published in May 2021. Capability per parameter has improved considerably since, so a newer model of the same size is often the better use of the same hardware.
What is MedBERT used for?
MedBERT works in Medicine, and is recorded as handling medical diagnosis, Text classification, Prediction of hospital stay duration, Prediction of diabetic heart failure (DHF), Prediction of onset of pancreatic cancer (PaCa). A model can carry several of each, so these are the areas it was built for rather than a limit on what it will attempt.
How much compute was used to train MedBERT?
Around 9.5 × 10¹⁸ FLOP, on NVIDIA Tesla V100 DGXS 32 GB. That measures what producing the model cost and says nothing about how quickly it answers once trained — inference speed comes from memory bandwidth, not from the training budget.
What GPU do I need to run MedBERT?
None. MedBERT is a closed model — its weights were never published, so it cannot be downloaded or run on your own hardware at any price. It is reachable only through its provider.
The other direction
Looking at it from the other side?
This page starts from the model. If you already own a card and want to know everything it will run, start from the hardware instead.