TODO how does the hidden layer get extracted to perform document vectoring

Network

• There is an input, single hidden and output layer.
• Input works as a "one hot" of the words going in to the bag of words.
• More than one word may be in the input layer to add "context."
• Output is a "one hot" of the next predicted word.

The output layer becomes a list of probabilities for each target word.

One-hot

• One neuron for each word the network knows how to handle.
• All are set to zero while the correct word is set to one.

References

• One-hot encoding, which is the original way to represent words for CBOW.
• Towards Data Science article