Challenges in designing digital interfaces for the study of multimodal phenomena
The paper discusses the challenges faced by researchers in developing effective digital interfaces for analyzing the meaning-making processes of multimodal phenomena. The authors propose a social semiotic approach as the underlying theoretical foundation, because interactive digital technology is the embodiment of multimodal social semiotic communication. The paper outlines the complex issues with which researchers are confronted in designing digital interface frameworks for modeling, analyzing, and retrieving meaning from multimodal data, giving due consideration to the multiplicity of theoretical frameworks and theories which have been developed for the study of multimodal text within social semiotics, and their impact on the development of a computer-based tool for the exploration, annotation, and analysis of multimodal data.
Keywords: digital interface design, digital technology, multimodal, social semiotics
Published online: 13 August 2010
Cited by 17 other publications
No author info given
Lee, Serene Qing Yun & Caroline Ho
Lim Fei, Victor, Kay L. O’Halloran, Sabine Tan & Marissa K. L. E
O'Halloran, Kay L., Sabine Tan, Bradley A. Smith & Alexey Podlasov
Paliwoda-Matiolanska, Adriana, Emilia Smolak-Lozano & Atsuho Nakayama
Pena, Zósimo López
Prieto Velasco, Juan Antonio
Smith, Bradley A., Sabine Tan, Alexey Podlasov & Kay L. O'Halloran
Vasconcelos, Luis A., Felipe Breyer, Bernardo Reis, Aline Silveira, Daniela Falcone, Judith Kelner & Ubiratan Carmo
Zagidullina, Marina V.
This list is based on CrossRef data as of 28 august 2021. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers. Any errors therein should be reported to them.