Natural languages are redundant. Redundancy constitutes a basic quality of human language communication, making for the predictability of the incoming message. There are two sources of redundancy in human discourse: the iteration of certain linguistic features, and the interdependency of the linguistic components of the message, the fact that message units are formed according to rules, which makes them interdependent. The author does not propose a pure Markovian model to the process of speech perception, he rather suggests a mechanism in the general spirit of Osgood’s proposal, in which traditional probabilities interconnect not the units of one level, but the whole hierarchy of structures in language. The author thus proposes a model framework providing for a concurrent dynamics of probability prediction at a hierarchy of levels and an interdependence of levels.