Metrics

  • citations in SCIndeks: 0
  • citations in CrossRef:0
  • citations in Google Scholar:[]
  • visits in previous 30 days:34
  • full-text downloads in 30 days:14

Contents

article: 10 from 20  
Back back to result list
2020, iss. 8, pp. 137-153
Gestural similarity, mathematics, psychology: Hints from a first experiment and some applications between pedagogy and research
aUniversity of Palermo, Department of Mathematics and Informatics, Italy
bAristotle University of Thessaloniki, Greece

emailmariacaterina.mannone@unipa.it, pdimitri@mus.auth.gr
Keywords: musical articulation; movement; gesture; drawing; perception
Abstract
Can music and drawings be thought of as the results of physical gestures, and thus be compared? In this paper we summarize the conjecture of "gestural similarity" developed in the framework of the mathematical theory of musical gestures. Then, we outline the history of an experiment involving mathematics, music, drawing, and psychology, aiming to evaluate the cognitive relevance of the conjecture. A simple visual form and a short homophonic musicalsequence can be considered "similar" if they can be thought of as produced by the same movements. Participants in an experiment were asked to assess the degree of similarity between given music examples and simple visuals (three visuals for each sound stimulus). Results were analyzed and confirmed the theoretical expectations. In addition, we describe some creative applications of this conjecture, including pedagogical and creative developments. In particular, we describe the music derived from a natural form, the essential structure of an ammonite, and the illusion of a "mathematical ocean" with sounds and images. We discuss challenges of these techniques and the characteristics of spectrograms in relation with gestural similarity.
References
Arias, J.S. (2018) Spaces of gestures are function spaces. Journal of Mathematics and Music, 12 (2), 89-105
Arias-Valero, J.S., Lluis-Puebla, E. (2020) Some remarks on hypergestural homology of spaces and its relation to classical homology. Journal of Mathematics and Music, 1-21, https://www.tandfonline.com/doi/full/10.1080/17459737.2020.1722269
Borgo, D. (2011) Embodied Music Cognition and Mediation Technology. Literary and Linguistic Computing, 27(1): 97-99
Daniel, L., Helen, P.M. (2017) Music and Shape. in: Studies in Musical Performance as Creative Practice, New York: Oxford University Press
Eitan, Z., Granot, R. (2006) How music moves: Musical parameters and listeners' images of motion. in: Perceptual characterization of motion evoked by sounds for synthesis control purposes, Music Perception, 23 (3), 221-247
Eitan, Z., Ornoy, E., Granot, R.Y. (2012) Listening in the dark: Congenital and early blindness and cross-domain mappings in music. Psychomusicology: Music, Mind, and Brain, 22(1): 33-45
Engeln, L., Groh, R. (2019) CoHEARence: A qualitative User-(Pre-)Test on Resynthesized Shapes for coherent visual Sound Design. in: AM'19: The 14th International Audio Mostly Conference: A Journey in Sound, September, Proceedings of, 98-102
Engeln, L., Groh, R. (2020) CoHEARence of audible shapes: A qualitative user study for coherent visual audio design with resynthesized shapes. Personal and Ubiquitous Computing
Fortuna, S. (2017) Embodiment, sound and visualization: A multimodal perspective in music education. Zbornik radova Akademije umetnosti, br. 5, str. 120-131
Godøy, R.I., Leman, M., eds. (2010) Musical Gestures: Sound, Movement, and Meaning. New York: Routledge
Godøy, R.I., Haga, E., Jensenius, A.R. (2006) Playing 'Air Instruments': Mimicry of Sound-Producing Gestures by Novices and Experts. in: Gibet S., Courty N., Kamp J.F. [ed.] Gesture in Human-Computer Interaction and Simulation: GW 2005 (Lecture Notes in Computer Science), Berlin, Heidelberg: Springer, 3881: 256-267
Jedrzejewski, F. (2019) Hétérotopies musicales. Paris: Hermann
Jensenius, A.R., Wanderley, M.M., Godøy, R.I., Leman, M. Musical Gestures: Concepts and methods in research. in: Godøy R.I., Leman M. [ed.] Musical gestures: Sound, movement, and meaning, New York: Routledge, 12-35
Kelkar, T., Jensenius, A.R. (2018) Analyzing Free-Hand Sound-Tracings of Melodic Phrases. Applied Sciences, 8 (135), 1-21
Kozak, M. (2020) Enacting Musical Time: The Bodily Experience of New Music. New York: Oxford University Press, Chapter 3
Kubovy, M., Schutz, M. (2010) Audio-Visual Objects. Review of Philosophy and Psychology, 1(1): 41-61
Leman, M. (2008) Embodied music cognition and mediation technology. Cambridge, Massachusetts: MIT Press
Leman, M. (2016) The Expressive Moment: How Interaction (with Music) Shapes Human Empowerment. Cambridge, Massachusetts: MIT Press
Mac, L.S. (1978) Categories for the Working Mathematician. New York: Springer
Mannone, M., Papageorgiou, D., Collins, T. (2018/2020) Psychological validation of the Mathematical Theory of Musical Gestures. Submitted
Mannone, M. (2019) Mathematics, Nature, Art. Palermo: Palermo University Press
Mannone, M. (2011) Dalla Musica allImmagine, dallImmagine alla Musica: Relazioni matematiche fra composizione musicale e arte figurativa / from Music to Images, from Images to Music: Mathematical relations between musical composition and figurative art. Palermo: Edizioni Compostampa
Mannone, M., Favali, F., di Donato, B., Turchet, L. (2020) Quantum GestART: Identifying and applying correlations between mathematics, art, and perceptual organization. Journal of Mathematics and Music, 1-33
Mannone, M. (2018) Introduction to gestural similarity in music: An application of category theory to the orchestra. Journal of Mathematics and Music, 12(2): 63-87
Martolini, C., Cappagli, G., Campus, C., Gori, M. (2019) Shape Recognition with Sounds: Improvement in Sighted Individuals After Audio-Motor Training. Multisensory Research, 33(4-5): 417-431
Mazzola, G., Andreatta, M. (2007) Diagrams, gestures and formulae in music. Journal of Mathematics and Music, 1(1): 23-46
Niewiadomski, R., Mancini, M., Cera, A., Piana, S., Canepa, C., Camurri, A. (2018) Does embodied training improve the recognition of mid-level expressive movement qualities sonification?. Journal on Multimodal User Interfaces, 13(3): 191-203
Niinisalo, J. (2017) The Aphex Face. (blogpost). http://www.bastwood.com/?page_id=10
Repp, B. (1992) Music as Motion: A Synopsis of Alexander Truslit's (1938). in: Gestaltung und Bewegung in der Musik: Haskins Laboratories Status Report on Speech Research, SR-111/112, pp. 265-278
Rosenblum, L.D., Dias, J.W., Dorsi, J. (2017) The supramodal brain: Implications for auditory perception. Journal of Cognitive Psychology, 29(1): 65-87
Spence, C. (2011) Crossmodal correspondences: A tutorial review. Attention, Perception, & Psychophysics, 73(4): 971-995
Timothy, C.L. (2020) On the topological characterization of gestures in a convenient category of spaces. Journal of Mathematics and Music, 1-25, https://www.tandfonline.com/doi/full/10.1080/17459737.2020.1716403
Xenakis, I. (2001) Formalized Music. New York: Pendragon
Zbikowski, L. (2017) Music, Analogy, and Metaphor. in: Ashley R., Timmers R. [ed.] The Routledge Companion to Music Cognition, New York: Routledge, 501-512
 

About

article language: English
document type: Professional Paper
DOI: 10.5937/ZbAkU2008137M
published in SCIndeks: 26/09/2020
Creative Commons License 4.0

Related records