MV-RNN

MV-RNN was published by Stanford University, based in United States of America, categorised as academia.

MV-RNN was published in July 2012. Capability per parameter has improved considerably since, so a newer model of the same size is often the better use of the same hardware.

MV-RNN works in Language, and is recorded as handling text classification. These are the areas it was designed around; they describe intent rather than a hard boundary.

None. MV-RNN is a closed model — its weights were never published, so it cannot be downloaded or run on your own hardware at any price. It is reachable only through its provider.

The licensing for MV-RNN was never recorded in our source data. We treat unstated licensing as closed, because an unrecorded licence is not one to rely on.

MV-RNN has 3.5M parameters. "We represent a word as both a continuous vector and a matrix of parameters. We initialize all word vectors x ∈ Rn with pre-trained 50-dimensional word vectors from the unsupervised model of Collobert and Weston (2008). [...] Every word is also associated with a matrix X. [...] If the vectors have dimensionality n, then each word’s matrix has dimensionality X ∈ Rn×n." "We propose the following combination function which is input dependent: p = fA,B(a, b) = f(Ba, Ab) = g(W x (Ba Ab)) ,(2) where A, B are matrices for single words, the global W ∈ Rn×2n is a matrix that maps both transformed words back into the same n-dimensional space." "For computing nonterminal phrase matrices, we define the function P = fM(A, B) = WMA, B, (3) where WM ∈ Rn×2n, so P ∈ Rn×n just like each input matrix." "If every word is represented by an n-dimensional vector and additionally by an n × n matrix, the dimensionality of the whole model may become too large with commonly used vector sizes of n = 100. In order to reduce the number of parameters, we represent word matrices by the following low-rank plus diagonal approximation: A = UV + diag(a), (5) where U ∈ Rn×r, V ∈ Rr×n, a ∈ Rnand we set the rank for all experiments to r = 3." "We train these representations by adding on top of each parent node a simple softmax classifier to predict a class distribution over, e.g., sentiment or relationship classes: d(p) = softmax(Wlabelp). If there are K labels, then d ∈ RK is a K-dimensional multinomial distribution" In total there are V*(n+n*r + r*n) + n*2n + n*2n + (n+1)*k parameters, where n is the vector dimension, r is the low-rank decomposition dimension, V is the vocabulary size and k is the number of classes. In the experiments we have that n=50, r=3, k=? and V=?. I'm guesstimating k=5 and V=10k. That figure is the total, and it is what decides how much memory the model needs — roughly half a gigabyte per billion at the compression most people use.