Psycholinguistic evidence shows that spatial domains are automatically activated when processing temporal expressions. Speakers conceptualize time as a straight line deployed along different axes (mostly sagittal, though also vertical). The use of the lateral axis, which cannot be lexicalized in any language, has nonetheless been attested in temporal tasks in laboratories using a variety of experiments. This leads to the question of what axes are actually at work when conceptualizating time in oral communication. The present study examines a great number of temporal expressions, taken from television shows, noting their associated co-speech gestures. Our results show that (1) speakers overwhelmingly use the lateral axis; (2) they are not performing simple space-to-time mappings, but are using instead a “timeline”, a material anchor which is a far more complex construct and that can explain some of the intricacies and contextual variations shown in the pattern of results.