Markov State Machines

A markov model consists of one or more states.

Each state consists of a list of valid transitions from that state to another state.

Each transition has a probability of taking that transition.

When taking a particular transition or entering a particular state it is possible to record that to tape and act on it. This allows simple robots to know what words are most likely to occur next after a given word. This is sometimes used to generate random nonsense sentences as well.

Markov models do not have any memory other than the current state so they are unable to understand context deeply.