- citations in SCIndeks: 0
- citations in CrossRef:0
- citations in Google Scholar:[
 ]
- visits in previous 30 days:5
- full-text downloads in 30 days:3
|
|
|
2020, iss. 8, pp. 137-153
|
|
Gestural similarity, mathematics, psychology: Hints from a first experiment and some applications between pedagogy and research
Gestovna sličnost, matematika, psihologija - nagoveštaji nakon prvog eksperimenta i neke primene između pedagogije i istraživanja
Abstract
Can music and drawings be thought of as the results of physical gestures, and thus be compared? In this paper we summarize the conjecture of "gestural similarity" developed in the framework of the mathematical theory of musical gestures. Then, we outline the history of an experiment involving mathematics, music, drawing, and psychology, aiming to evaluate the cognitive relevance of the conjecture. A simple visual form and a short homophonic musicalsequence can be considered "similar" if they can be thought of as produced by the same movements. Participants in an experiment were asked to assess the degree of similarity between given music examples and simple visuals (three visuals for each sound stimulus). Results were analyzed and confirmed the theoretical expectations. In addition, we describe some creative applications of this conjecture, including pedagogical and creative developments. In particular, we describe the music derived from a natural form, the essential structure of an ammonite, and the illusion of a "mathematical ocean" with sounds and images. We discuss challenges of these techniques and the characteristics of spectrograms in relation with gestural similarity.
Sažetak
Da li se muzika i crteži mogu smatrati rezultatima fizičkih gestova, te se na taj način mogu uporediti? U ovom radu će se sumirati hipoteza o "gestovnoj sličnosti" koja je razvijena u okviru matematičke teorije o muzičkim gestovima. Zatim će se izložiti istorija eksperimenta koji uključuje matematiku, muziku, crtanje i psihologiju, sa ciljem da se proceni kognitivna relevantnost hipoteze. Jednostavna vizuelna forma i kratki homofoni muzički niz mogu se smatrati "sličnim" ako se mogu zamisliti kao da su proizašli iz istih pokreta. Učesnici u eksperimentu su zamoljeni da procene stepen sličnosti između datih muzičkih primera i jednostavnih vizuelnih prikaza (tri prikaza za svaki muzički stimulus). Analizom rezultata potvrđena su teorijska očekivanja. Pored toga, rad opisuje neke kreativne primene ove hipoteze, uključujući pedagoški i kreativni razvoj. Konkretno, opisuje se muzika koja je izvedena iz prirodnog oblika, suštinske strukture amonita i iluzije "matematičkog okeana" sa zvukovima i slikama. Razmatraju se izazovi ovih tehnika i karakteristike spektrograma u odnosu na gestovnu sličnost.
|
|
|
|
References
|
|
 
|
Arias, J.S. (2018) Spaces of gestures are function spaces. Journal of Mathematics and Music, 12 (2), 89-105
|
|
|
Arias-Valero, J.S., Lluis-Puebla, E. (2020) Some remarks on hypergestural homology of spaces and its relation to classical homology. Journal of Mathematics and Music, 1-21, https://www.tandfonline.com/doi/full/10.1080/17459737.2020.1722269
|
|
|
Borgo, D. (2011) Embodied Music Cognition and Mediation Technology. Literary and Linguistic Computing, 27(1): 97-99
|
|
|
Daniel, L., Helen, P.M. (2017) Music and Shape. in: Studies in Musical Performance as Creative Practice, New York: Oxford University Press
|
|
|
Eitan, Z., Granot, R. (2006) How music moves: Musical parameters and listeners' images of motion. in: Perceptual characterization of motion evoked by sounds for synthesis control purposes, Music Perception, 23 (3), 221-247
|
|
|
Eitan, Z., Ornoy, E., Granot, R.Y. (2012) Listening in the dark: Congenital and early blindness and cross-domain mappings in music. Psychomusicology: Music, Mind, and Brain, 22(1): 33-45
|
|
|
Engeln, L., Groh, R. (2019) CoHEARence: A qualitative User-(Pre-)Test on Resynthesized Shapes for coherent visual Sound Design. in: AM'19: The 14th International Audio Mostly Conference: A Journey in Sound, September, Proceedings of, 98-102
|
|
|
Engeln, L., Groh, R. (2020) CoHEARence of audible shapes: A qualitative user study for coherent visual audio design with resynthesized shapes. Personal and Ubiquitous Computing
|
|
|
Fortuna, S. (2017) Embodiment, sound and visualization: A multimodal perspective in music education. Zbornik radova Akademije umetnosti, br. 5, str. 120-131
|
|
|
Godøy, R.I., Leman, M., eds. (2010) Musical Gestures: Sound, Movement, and Meaning. New York: Routledge
|
|
|
Godøy, R.I., Haga, E., Jensenius, A.R. (2006) Playing 'Air Instruments': Mimicry of Sound-Producing Gestures by Novices and Experts. in: Gibet S., Courty N., Kamp J.F. [ed.] Gesture in Human-Computer Interaction and Simulation: GW 2005 (Lecture Notes in Computer Science), Berlin, Heidelberg: Springer, 3881: 256-267
|
|
|
Jedrzejewski, F. (2019) Hétérotopies musicales. Paris: Hermann
|
|
|
Jensenius, A.R., Wanderley, M.M., Godøy, R.I., Leman, M. Musical Gestures: Concepts and methods in research. in: Godøy R.I., Leman M. [ed.] Musical gestures: Sound, movement, and meaning, New York: Routledge, 12-35
|
|
|
Kelkar, T., Jensenius, A.R. (2018) Analyzing Free-Hand Sound-Tracings of Melodic Phrases. Applied Sciences, 8 (135), 1-21
|
|
|
Kozak, M. (2020) Enacting Musical Time: The Bodily Experience of New Music. New York: Oxford University Press, Chapter 3
|
|
|
Kubovy, M., Schutz, M. (2010) Audio-Visual Objects. Review of Philosophy and Psychology, 1(1): 41-61
|
|
1
|
Leman, M. (2008) Embodied music cognition and mediation technology. Cambridge, Massachusetts: MIT Press
|
|
|
Leman, M. (2016) The Expressive Moment: How Interaction (with Music) Shapes Human Empowerment. Cambridge, Massachusetts: MIT Press
|
|
|
Mac, L.S. (1978) Categories for the Working Mathematician. New York: Springer
|
|
|
Mannone, M., Papageorgiou, D., Collins, T. (2018/2020) Psychological validation of the Mathematical Theory of Musical Gestures. Submitted
|
|
|
Mannone, M. (2019) Mathematics, Nature, Art. Palermo: Palermo University Press
|
|
|
Mannone, M. (2011) Dalla Musica allImmagine, dallImmagine alla Musica: Relazioni matematiche fra composizione musicale e arte figurativa / from Music to Images, from Images to Music: Mathematical relations between musical composition and figurative art. Palermo: Edizioni Compostampa
|
|
|
Mannone, M., Favali, F., di Donato, B., Turchet, L. (2020) Quantum GestART: Identifying and applying correlations between mathematics, art, and perceptual organization. Journal of Mathematics and Music, 1-33
|
|
|
Mannone, M. (2018) Introduction to gestural similarity in music: An application of category theory to the orchestra. Journal of Mathematics and Music, 12(2): 63-87
|
|
|
Martolini, C., Cappagli, G., Campus, C., Gori, M. (2019) Shape Recognition with Sounds: Improvement in Sighted Individuals After Audio-Motor Training. Multisensory Research, 33(4-5): 417-431
|
|
1
|
Mazzola, G., Andreatta, M. (2007) Diagrams, gestures and formulae in music. Journal of Mathematics and Music, 1(1): 23-46
|
|
|
Niewiadomski, R., Mancini, M., Cera, A., Piana, S., Canepa, C., Camurri, A. (2018) Does embodied training improve the recognition of mid-level expressive movement qualities sonification?. Journal on Multimodal User Interfaces, 13(3): 191-203
|
|
|
Niinisalo, J. (2017) The Aphex Face. (blogpost). http://www.bastwood.com/?page_id=10
|
|
|
Repp, B. (1992) Music as Motion: A Synopsis of Alexander Truslit's (1938). in: Gestaltung und Bewegung in der Musik: Haskins Laboratories Status Report on Speech Research, SR-111/112, pp. 265-278
|
|
|
Rosenblum, L.D., Dias, J.W., Dorsi, J. (2017) The supramodal brain: Implications for auditory perception. Journal of Cognitive Psychology, 29(1): 65-87
|
|
3
|
Spence, C. (2011) Crossmodal correspondences: A tutorial review. Attention, Perception, & Psychophysics, 73(4): 971-995
|
|
|
Timothy, C.L. (2020) On the topological characterization of gestures in a convenient category of spaces. Journal of Mathematics and Music, 1-25, https://www.tandfonline.com/doi/full/10.1080/17459737.2020.1716403
|
|
1
|
Xenakis, I. (2001) Formalized Music. New York: Pendragon
|
|
|
Zbikowski, L. (2017) Music, Analogy, and Metaphor. in: Ashley R., Timmers R. [ed.] The Routledge Companion to Music Cognition, New York: Routledge, 501-512
|
|
|
|
|