Multimodal analysis is quintessential in all forms of digital communication and is expressed through visuals, speech, music, and gesture. In conjunction with this, ‘affect’ is the primary response to these mediums and has the power to influence and shape a user experience. As such, communication and multimodality is exemplified in all aspects of life and is an essential component that combines ‘reason’ with ‘passion’.
Furthermore, multimodality is the driving factor that influences ‘affect’ and articulates a complex system of communication. A multitude of methods are employed in these situations as they elucidate the modes that resonate with a number of different audiences. This type of interrelationship is evident in a recent article at the University of Delaware. Jared Medina, neuroscientist at the National Science Foundation is exploring the role of cognition and learning to “probe the complex relationship between existing knowledge already stored in the brain and new information obtained through sensory perception” (U Daily 2016, para 3). Through an analysis of the physical and digital environment, the brain processes and stores information so that people can view the world from a different lense. To elaborate, the team is set to utilise a number of research techniques that will ultimately showcase the accessibility of multimodal analysis.
Moreover, the team will be investigating statistical learning, action, attention and working memory by conducting a number of experiments on the participants. Subsequently, participants will be analysed and results will be recorded based on their response to the sensory environment, speech, visual scenes, and auditory stimuli. By undertaking these experiments, participants will be able to better understand their individual subjectivities as well as how they interact, respond and construct their physical surroundings.
The significance of being able to examine an aspect of cognitive function, using more than one technique is reiterated by Medina as he says, “A multimodal approach allows us to more fully understand mind-brain relationships” (U Daily 2016, para. 18). Research will then be conducted at a deeper level through techniques of neuroimaging, neuropsychology and neurostimulation. Therefore, multimodal analysis has the capability to further comprehend the psychology of brain-damaged individuals and stroke victims. The benefits of this means that researchers and analysts can immerse themselves in the science of how injury to specific regions of the brain can influence and affect the relevant cognitive functions.
The demand to initiate scientific progress worldwide is enhanced by the notion of multimodal analysis. It is constantly present in all aspects of life and is a model to explain the impact of diverse modes on specific areas of the brain. This in turn, initiates a response and has the power to affect the individual. It is the way people embody these responses that makes a difference, in terms of cognitive function and the ability to make sense of the world. Consequently, the link between ‘reason’ and ‘passion’ is evident, thus providing strong sense of awareness that is reinforced by the sentiments of Medina in this article.
U Daily 2016, EPSCOR Track-2 Award, Delaware, viewed 26 August 2016, <http://www.udel.edu/udaily/2016/august/nsf-epscor-grant-neuroscience/>