Text comprehension involves linking the current sentence or utterance to the previous discourse context. This process of coherence building might be aided by linguistic markers, such as cohesive ties and direct anaphora. However, when the explicitly mentioned information does not provide unambiguous and consistent cues to coherence, as for example in the case of indirect anaphora, the comprehender draws on general background knowledge for inferring plausible links. This paper summarizes neuroimaging studies that shed light on the interplay between cognitive and linguistic components of text comprehension. The results indicate that the anterior temporal lobes and the left fronto-medial cortex are particularly important for coherence building.