This paper reports about study on action understanding in the brain. Recent results of the neurophysiology of the mirror system in the monkey are reviewed. Based on these observations, a model of this brain system is proposed which is responsible for action recognition. The link between object affordances and action understanding is considered. To support this hypothesis, two experiments are described where some aspects of the model have been implemented. In the first experiment an action recognition system is trained by using data recorded from human movements. In the second experiment, the model is partially implemented on a humanoid robot which learns to mimic simple actions performed by a human subject on different objects. These experiments show that motor information can have a significant role in action interpretation and that a mirror-like representation can be developed autonomously as a result of the interaction between an individual and the environment.