In recent years, consensus has been growing that certain co-speech gestures must be incorporated in grammatical theory. Contrary to traditional beliefs, gestural expression has been found to be systematic in various ways, to exhibit some degree of compositional structure and to be to a certain extent language-specific. Most current work on the gesture-grammar interface is, nonetheless, rather programmatic in character or lacks an appropriate theoretical framework. This paper shows that Functional Discourse Grammar (FDG) provides a promising model for understanding the grammar-gesture relationship in a more comprehensive and detailed way than has thus far been achieved. It first evaluates how a FDG perspective on language structure aligns with the ways gestures have been studied. Then it demonstrates that various types of co-speech manual gestures can be classified (perhaps even exhaustively) according to the pragmatic and semantic layers recognized in FDG. On the basis of this survey, a first sketch of a multi-channel (speech-gesture) version of FDG’s architecture is presented. Finally, some challenges are put forward that arise when taking the multimodal grammar enterprise seriously. Altogether, the content of this paper suggests that a multimodal perspective on grammar can be mutually beneficial for functionally oriented linguists and gesture researchers.