Full-duplex acoustic interaction system for cognitive experiments with cetaceans

Rychen, Jörg; Semoroz, Julie; Eckerle, Alexander; Hahnloser, Richard HR; Kleinberger, Rébecca

doi:10.1075/is.22039.ryc

Article published In:

Vocal Interactivity in-and-between Humans, Animals and Robots
Edited by Mohamed Chetouani, Elodie F. Briefer, Angela Dassow, Ricard Marxer, Roger K. Moore, Nicolas Obin and Dan Stowell
[Interaction Studies 24:1] 2023
► pp. 66–86

Full-duplex acoustic interaction system for cognitive experiments with cetaceans

Jörg Rychen | University of Zurich | ETH Zurich

Julie Semoroz | Association Motonomy

Alexander Eckerle | Ludwig-Maximilians-University of Munich

Richard HR Hahnloser | University of Zurich | ETH Zurich

Rébecca Kleinberger | MIT Media Lab | Northeastern University

Cetaceans show high cognitive abilities and strong social bonds. Their primary sensory modality to communicate and sense the environment is acoustics. Research on their echolocation and social vocalizations typically uses visual and tactile systems adapted from research on primates or birds. Such research would benefit from a purely acoustic communication system to better match their natural capabilities. We argue that a full duplex system, in which signals can flow in both directions simultaneously is essential for communication research. We designed and implemented a full duplex system to acoustically interact with cetaceans in the wild, featuring digital echo-suppression. We pilot tested the system in Arctic Norway and achieved an echo suppression of 18 dB. We discuss the limiting factors and how to improve the echo suppression further. The system enabled vocal interaction with the underwater acoustic scene by allowing experimenters to listen while producing sounds. We describe our motivations, then present our pilot deployment and give examples of initial explorative attempts to vocally interact with wild orcas and humpback whales.

Keywords: human–dolphin communication, self-interference cancellation, echo cancellation, killer-whale, orca, humpback whale, inter-species communication, animal communication, bioacoustics, underwater communication, animal cognition

Article outline

Introduction
Interactive playback experiments (Figure 2A)
Phantom echo (Figure 2B)
Animal telecommunication (Figure 2C)
Human–Animal interaction (Figure 2D)
Pilot experiments
System
Field test
Results and discussion
- Echo suppression performance
- Human experience of underwater soundscape
- Vocal interactions with orcas and humpbacks
Conclusion
Ethical statement
Acknowledgments
Author contributions
Conflict of interests
References

Available under the Creative Commons Attribution (CC BY) 4.0 license.

For any use beyond this license, please contact the publisher at [email protected].

Published online: 28 August 2023

https://doi.org/10.1075/is.22039.ryc

References

Amundin, M., Starkhammar, J., Evander, M., Almqvist, M., Lindström, K., & Persson, H. W.

(2008) “An echolocation visualization and interface system for dolphin research,” J Acoust Soc Am, vol. 123, no. 2, pp. 1188–1194.

Antsaklis, P. J.

(2021) “Linear Systems: Discrete-Time Impulse Response Descriptions,” in Encyclopedia of Systems and Control, Cham: Springer International Publishing, pp. 1144–1147.

Aubauer, R., & Au, W. W. L.

(1998) “Phantom echo generation: A new technique for investigating dolphin echolocation,” J Acoust Soc Am, vol. 104, no. 3, pp. 1165–1170.

Branstetter, B. K., et al.

(2020) “Spectral cues and temporal integration during cylinder echo discrimination by bottlenose dolphins (Tursiops truncatus),” J Acoust Soc Am, vol. 148, no. 2, pp. 614–626.

Curé, C., et al.

(2019) “Evidence for discrimination between feeding sounds of familiar fish and unfamiliar mammal-eating killer whale ecotypes by long-finned pilot whales,” Anim Cogn, vol. 22, no. 5, pp. 863–882.

Deecke, V. B.

(2007) “Studying Marine Mammal Cognition in the Wild: A Review of Four Decades of Playback Experiments,” Aquat Mamm, vol. 32, no. 4, pp. 461–482.

Delikaris-Manias, S., McCormack, L., Huhtakallio, I., & Pulkki, V.

(2018) “Real-time underwater spatial audio: a feasibility study,” 144th Audio Engineering Society Convention 2018, no. May.

Filatova, O. A., Fedutin, I. D., Burdin, A. M., & Hoyt, E.

(2011) “Responses of Kamchatkan fish-eating killer whales to playbacks of conspecific calls,” Mar Mamm Sci, vol. 27, no. 2, pp. 26–42.

Finneran, J. J., et al.

(2020) “Dolphin echo-delay resolution measured with a jittered-echo paradigm,” J Acoust Soc Am, vol. 148, no. 1, pp. 374–388.

Handel, S., Todd, S. K., & Zoidis, A. M.

(2012) “Hierarchical and rhythmic organization in the songs of humpback whales (Megaptera novaeangliae),” Bioacoustics, vol. 21, no. 2, pp. 141–156.

Haykin, S., & Chen, Z.

(2005) “The cocktail party problem,” Neural Computation, vol. 17, no. 9. MIT Press, pp. 1875–1902, Sep. 01.

Herman, L. M., Richards, D. G., & Wolz, J. P.

(1984) “Comprehension of sentences by bottlenosed dolphins,” Cognition, vol. 16, no. 2, pp. 129–219.

Herzing, D.

(2016) “Interfaces and Keyboards For Human–Dolphin Communication: What Have We Learned?,” Anim Behav Cogn, vol. 3, no. 4, pp. 243–254.

Herzing, D. L., & Johnson, C. M.

(2015) Dolphin communication and cognition : past, present, and future / edited by Denise L. Herzing and Christine M. Johnson. The MIT Press.

Holt, M. M., Noren, D. P., & Emmons, C. K.

(2011) “Effects of noise levels and call types on the source levels of killer whale calls,” J Acoust Soc Am, vol. 130, no. 5, pp. 3100–3106.

Jourdain, E., et al.

(2019) “North Atlantic killer whale Orcinus orca populations: a review of current knowledge and threats to conservation,” Mamm Rev, vol. 49, no. 4, pp. 384–400.

(2021) “Natural Entrapments of Killer Whales (Orcinus orca): A Review of Cases and Assessment of Intervention Techniques,” Frontiers in Conservation Science, vol. 21, no. August, pp. 1–13.

King, S. L.

(2015) “You talkin’ to me? Interactive playback is a powerful yet underused tool in animal communication research,” Biol Lett, vol. 11, no. 7,

King, S. L., Guarino, E., Donegan, K., McMullen, C., & Jaakkola, K.

(2021) “Evidence that bottlenose dolphins can communicate with vocal signals to solve a cooperative task,” R Soc Open Sci, vol. 8, no. 3,

Kohlsdorf, D., Gilliland, S., Presti, P., Starner, T., & Herzing, D.

(2013) “An underwater wearable computer for two way human–dolphin communication experimentation,” in ISWC 2013 – Proceedings of the 2013 ACM International Symposium on Wearable Computers, pp. 147–148.

Lang, T. G., & Smith, H. A. P.

(1965) “Communication between dolphins in separate tanks by way of an electronic acoustic link,” Science Science, vol. 150, no. 3705, pp. 1839–1844.

Lilly, J. C.

(1978) “Communication between man and dolphin: The possibilities of talking with other species.,” Crown Publishers, Inc., New York. 269pp. 1978. Crown, New York, p. 269.

Ling, F.

(2014) “Achievable performance and limiting factors of echo cancellation in wireless communications,” in 2014 Information Theory and Applications Workshop, ITA 2014 – Conference Proceedings.

Magnúsdóttir, E. E., & Lim, R.

(2019) “Subarctic singers: Humpback whale (Megaptera novaeangliae) song structure and progression from an Icelandic feeding ground during winter,” PLoS One, vol. 14, no. 1,

Malinka, C.

(2021) “Biosonar of narrow-band high-frequency toothed whales: Sampling a dynamic, multi-target world,” Aarhus University.

Marino, L.

(2002) “Convergence of complex cognitive abilities in cetaceans and primates,” in Brain, Behavior and Evolution, vol. 59, no. 1–2, pp. 21–32.

Marulanda, J. Lopez, et al.

(Apr. 2021) “Acoustic behaviour of bottlenose dolphins under human care while performing synchronous aerial jumps,” Behavioural Processes, vol. 1851, p. 104357,

Muller, M. W., Au, W. W. L., Nachtigall, P. E., Allen, J. S., & Breese, M.

(2007) “Phantom echo highlight amplitude and temporal difference resolutions of an echolocating dolphin, Tursiops truncatus,” J Acoust Soc Am, vol. 122, no. 4, pp. 2255–2262.

Narula, G., Herbst, J. A., Rychen, J., & Hahnloser, R. H. R.

(2018) “Learning auditory discriminations from observation is efficient but less robust than learning from experience,” Nat Commun, vol. 9, no. 1, p. 3218,

Pika, S., Wilkinson, R., Kendrick, K. H., & Vernes, S. C.

2018 “Taking turns: Bridging the gap between human and animal communication,” Proceedings of the Royal Society B: Biological Sciences, vol. 285, no. 1880,

Richards, D. G., Wolz, J. P., & Herman, L. M.

(1984) “Vocal mimicry of computer-generated sounds and vocal labeling of objects by a bottlenosed dolphin, Tursiops truncatus.,” J Comp Psychol, vol. 98, no. 1, pp. 10–28.

Ridgway, S., Carder, D., Jeffries, M., & Todd, M.

(2012) “Spontaneous human speech mimicry by a cetacean,” Current Biology, vol. 22, no. 20. Elsevier, pp. R860–R861.

Rossa, A., & Kimb, S. J.

(Oct. 2021) “Behavioral experiments for gathering labeled animal vocalization data,” in Proceedings of the 3rd International Workshop on Vocal Interactivity in-and-between Humans, Animals and Robots, p. 35. Accessed: Oct. 21, 2022. [Online]. Available: [URL]

Rothenberg, D.

(2008) “Whale music: Anatomy of an interspecies duet,” Leonardo Music Journal, vol. 181, pp. 47–53.

Rychen, J., Rodrigues, D. I., Tomka, T., Rüttimann, L., Yamahachi, H., & Hahnloser, R. H. R.

(2021) “A system for controlling vocal communication networks,” Sci Rep, vol. 11, no. 1, pp. 1–15.

Shapiro, A. D.

(2008) “Orchestration : the movement and vocal behavior of free-ranging Norwegian killer whales (Orcinus orca),” Massachusetts Institute of Technology.

Shen, L., Henson, B., Zakharov, Y., & Mitchell, P.

(2020) “Digital Self-Interference Cancellation for Full-Duplex Underwater Acoustic Systems,” IEEE Transactions on Circuits and Systems II: Express Briefs, vol. 67, no. 1, pp. 192–196.

Shen, L., Zakharov, Y., Henson, B., Morozs, N., & Mitchell, P. D.

(2020) “Adaptive filtering for full-duplex UWA systems with time-varying self-interference channel,” IEEE Access, vol. 81, pp. 187590–187604.

Sondhi, M. M.

(1967) “An Adaptive Echo Canceller,” Bell System Technical Journal, vol. 46, no. 3, pp. 497–511.