RNN 500/10 + RT09 LM (NIST RT05)

It has a parameter count of 19.3M. "The acoustic HMMs are based on cross-word tied-states triphones trained discriminatively using MPE criteria. Feature extraction use 13 Mel-PLP’s features with deltas, double and triple deltas reduced by HLDA to 39-dimension feature vector" RNN 500/10 indicates that the model has a 500 units in the hidden layer and merges words that occur less than 10 times in the training set. Section 4.2 in this paper appears to indicate that NIST RT05 has a vocabulary of 50k words: https://www.fit.vut.cz/person/imikolov/public/rnnlm/char.pdf Using this dataset of english word frequencies ... https://www.kaggle.com/datasets/rtatman/english-word-frequency ... we can estimate that the number of words in the final vocabulary would have been around 19k when merging with threshold 5: https://colab.research.google.com/drive/1K5qH0EqXtFwTLESNtp4oelCM28GpGXt6?usp=sharing Thus the model in question would have: (500 + 19k) * 500 + 500 * 19k = 19250000 parameters The RT09 LM they interpolate with appears to be a 4-gram model. In one sense, n-gram models have parameters roughly equal to the number of unique n-grams in the training data. For comparison, the Google Web 1T 5-gram corpus has 1.3B unique 4-grams: https://catalog.ldc.upenn.edu/LDC2006T13 However, these aren't parameters in the same sense as neural networks as they are accessed as a lookup table, so on each forward pass only one parameter is active. I choose not to include these parameters in our count. That figure is the total, and it is what decides how much memory the model needs — roughly half a gigabyte per billion at the compression most people use.

It was published by Brno University of Technology,Johns Hopkins University, based in Czechia, an organisation categorised as academia,Academia.

It was published in September 2010. Capability per parameter has improved considerably since, so a newer model of the same size is often the better use of the same hardware.

It works in the domain of Speech, and is recorded as handling the task of transcription. A model can carry several of each, so these are the areas it was built for rather than a limit on what it will attempt.

Training consumed around 1.3 × 10¹⁶ FLOP. That measures what producing the model cost and says nothing about how quickly it answers once trained — inference speed comes from memory bandwidth, not from the training budget.

None. This is a closed model — its weights were never published, so it cannot be downloaded or run on your own hardware at any price. It is reachable only through its provider.

The licensing was never recorded in our source data. We treat unstated licensing as closed, because an unrecorded licence is not one to rely on.