RESEARCH
Mario Ortiz1, José L. Contreras-Vidal2, José M. Azorín1,2
1 Brain-Machine Interface Systems Lab, Miguel Hernández University of Elche, Av. de la Universidad S/N, 03202 Elche, Spain
2 IUCRC BRAIN Center, Noninvasive Brain–Machine Interface System Laboratory, Department of Electrical and Computer Engineering, University of Houston, Houston, TX 77004, USA
Music sounds, body dances and hands draw responding to each other while a projector screen shows brain waves of a saxophonist, a dancer and a visual artist, and the real-time motion captured of the dancer shown as an avatar (Fig. 1A). It is a creative variant of Exquisite Corpse, the collaborative, chance-based game made famous by the Surrealists [1]. In another demonstration, an enthusiast painter improvises in acrylic the feather headdress of the famous Irma Brunner of Manet, whereas his brain signals are acquired using a sensor cap that records the scalp electroencephalogram (EEG). His alpha EEG components are shown at the screen at the same time the brush finishes the black feathers of Irma’s hat (Fig. 1B). Both demonstrations were performed at the 2017 International Conference of Mobile Brain-Body Imaging and the Neuroscience of Art, Innovation and Creativity (“Your Brain on Art Conference”; http://yourbrainonart2017.egr.uh.edu/)1 , held in Valencia (Spain) from 10th to 13th of September 2017, and chaired by Prof. José M. Azorín (Miguel Hernández University of Elche, Spain) and Prof. José “Pepe” L. Contreras-Vidal (University of Houston, USA).
Figure 1. Your Brain on Art Performances. (A) Your Brain on Art: Creativity in Sound, Movement and Visual Arts, performed by Woody Witt (musician), Becky Valls (dancer) and Dario Robleto (conceptual visual artist), from University of Houston (USA); (B) Analyzing EEG During the Painting Process, performed by Mario Ortiz (painter), from Miguel Hernández University of Elche (Spain).
This conference brought together innovators and thought leaders from academia, the arts, federal agencies and industry to address the challenges of working at the intersection of science and art – two areas which on the surface appear to be far way from each other, but not so distant if you consider creativity and innovation associated with art as cognitive processes emerging from complex neural network dynamics. Thus, the conference highlighted several important challenges: Can creativity be assessed from the objective point of view of science? How science helps us to understand art and how art could help us to advance and promote science? This opened up interesting neuroscience, psychological and cognitive science discussions of how the state of mind can be changed and affect others. These are not new questions, but certainly as mobile brain-body imaging (MoBI) technology evolves, they are closer to be answered.
The conference gathered professionals, researchers and graduate students from science, the creative arts, medicine and engineering to discuss these and other emerging questions and challenges at the nexus of arts and science. There were 79 attendees from 16 countries (USA, Mexico, Spain, Germany, China, etc.): 27 scientists, 11 artists, 8 educators, 8 industry professionals, and 25 trainees.
During the 3 days, several demonstrations in the field of STEM+Art (Science, Technology, Engineering, Mathematics + Art) were combined with oral sessions and discussions. There were four main topics covered in the conference: 1) Wearable Mobile Brain-Body Imaging (MoBI), both hardware and software developments; 2) Cognitive and Brain-Computer Interfaces (BCI); 3) Cognitive, Medical and Pedagogical Applications; and 4) Neuroaesthetics. In addition, there was a doctoral consortium of trainees, composed by graduate students and postdoctoral researchers, who exposed their research findings on poster sessions and got the feedback of prominent researchers. The doctoral consortium also included a Brain-Computer Interface Hackaton, where teams of students worked with industry-provided MoBI equipment to design innovative Science-Art projects that addressed a broad range of problems.
The session titled “How do the creative arts engage the human mind and promote creativity and innovation across fields?” focused on the ways art and science can work together and explain how to represent or analyze the human experience and its surroundings. Therefore, they are committed to a multidisciplinary collaboration and to share spaces. One of the trainees, Jorge Gaxiola indicated: “I think the multidisciplinary approach is very interesting, mainly in the identification and creation of neuroscience tools focused on knowing about mental processes. This conference was my first experience merging neuroscience + art + engineering and I honesty loved it, because knowing more about the mental states that are presented could generate therapies, educational methods, etc.”
Another interesting topic was treated during the session “Brain Mechanisms of Aesthetic Perception and Artistic Brain-Computer Interfaces”, which offered important hints about the research paths to follow at the crossroad of art, engineering and neuroscience, and their derivative processes: creativity, neuroaesthetics and Brain-Computer Interfaces (BCIs). The fact that aesthetic stimuli activate several brain areas and processes and humans seek out aesthetic experiences could help developing models of the dynamical free behaving brain. In order to know more about the mental processes, it is necessary to study the brain “in action and in context” in real natural complex settings, which is now made possible by the use of context-aware MoBI technology.
Neurodegenerative diseases are a main concern in most countries where the aging population keeps growing. Everybody wants to live more. However, as important as longevity might be, high quality of live must also be maintained. The session “How can the arts and neuroscience research improve physical and mental health and promote wellbeing” focused on the benefits of art therapy not only for aged people, but also patients with neurological disorders, physical disability or trauma. There are different methodologies that are already used in art therapy [2]. Nevertheless, therapists emphasize the need for trans-disciplinary teams and more quantitative measurements in order to define the processes taking place and improve the therapy protocols.
Art is a way of communication, connecting with emotion and spurring creativity on trans-disciplinary teams. During the session “How disruptive neurotechnologies are changing science, arts and innovation”, it was shown how this intersection is changing neuroscience, art, cultural perception of art and technology and even technology itself. Suzanne Dikker (Utrecht University and New York University) said that it was inspiring to hear from other people that tried to breach the arts and science world.
In parallel to the conference, the consortium of trainees participated in the “Designers Hackathon on Artistic Brain-Computer Interfaces”. In this hackathon, multidisciplinary teams composed by one engineer, one scientist and one artist were capable of coming up with working prototypes in only three days. Brain Products and g.Tec provided hardware and software support to the teams, and OpenBCI also provided hardware. Fig. 2 shows the BCI project awarded as the Best Artistic Prototype. In this project, the user’s brain activity was used to control a robotic ball to paint.
Figure 2. Painting drawn in acrylic by a robotic ball controlled by a BCI. The hackathon project developed by Guillermo Herrera, Daniel Pimentel and Julia Anna Adrian was awarded as the Best Artistic Prototype at the Brain on Art Hackathon.
Discussion
BCIs can be developed to provide a quantitative approach to evaluate creativity and neuroaesthetics. There are several studies that investigate the relationship between artistic activities or neuroaesthetics and EEG pattern changes. For instance, there are studies that show that during visual art contemplation beta frequency band is suppressed [3], that during verbal creativity tasks alpha oscillations appear [4] or that the spectrum power in beta and gamma bands differs from non-creative activities [5]. However, further research is needed in order to develop the understanding and interpretation of the feedback EEG provides during artistic creation or observation. It is important to remark that there are still technical challenges to overcome. For instance, the unpredictable nature of the artistic performance makes it hard to neutralize artifacts due to muscle movement; the interpretation of the brain signal to what the artist meant to express; or the lack of control in the stimuli, challenges the analysis of the EEG signal. There are also technical issues to improve the usability of EEG acquisition systems without compromising the user comfort or the quality of the signal [6].
In addition, many artists are reluctant to participate in experimental research groups, as they consider art and creativity not susceptible to measurement and hard to be developed under the restrictive conditions of an experimental setup. Experimental design has to be approached involving artists and scientific researchers. Enhanced scientific rigor with balanced artistic nuance will lead to novel opportunities in the field. The fact is that, art can influence science and technology, and science and technology can in turn inspire art [7]. Therefore, they are part of a relationship that should not be split up.
Scientists, engineers and artists need to continue working on finding a common language to solve the problems ahead. Ashley Bear (National Academies of Sciences, Engineering, and Medicine, USA) noted that this conference was a great environment for scientists, engineers, artists and other professionals to create a shared language to learn how to communicate to solve the problems that our complex systems now generate.
Creative people are able to adapt with innovative ideas to the common daily problems. Employers look for people who can think broadly, work as a team and communicate with eloquence. All of those characteristics can easily be developed on art related classes and experiences. Could creativity indices obtained by BCIs be part of an employee selection process in the future? Who knows? The future will tell us.
Acknowledgments
We would like to thank all the agencies that supported this conference: National Science Foundation (Award IIS-1745835), Office of Naval Research Global (Award N62909-17-1-2191) and Conselleria de Educación, Investigación, Cultura y Deporte of Generalitat Valenciana, Spain (AORG/2017/123); the academic sponsors: University of Houston (USA), Miguel Hernandez University of Elche (Spain), and the Alliance for the Arts in Research Universities; the IEEE Brain Initiative, and the IEEE Systems, Man, and Cybernetics Society; the exhibitors: Brain Products, g.Tec and Technaid; and the rest of sponsors: Institut Valencià d’Art Modern (IVAM), Emotiv, Muse and OpenBCI.
References
[1] W. Bohn, “From Surrealism to Surrealism : Apollinaire and Breton,” J. Aesthet. Art Crit., vol. 36, no. 2, pp. 197–210, 1977.[2] J. Rubin, Approaches to Art Therapy: Theory and Technique, 3rd ed. New York, 2016.
[3] G. Herrera-Arcos et al., “Modulation of Neural Activity during Guided Viewing of Visual Art,” Front. Hum. Neurosci., vol. 11, p. 581, Nov. 2017.
[4] A. Fink and A. C. Neubauer, “EEG alpha oscillations during the performance of verbal creativity tasks: Differential effects of sex and verbal intelligence,” Int. J. Psychophysiol., vol. 62, no. 1, pp. 46–53, Oct. 2006.
[5] S. G. Danko, N. V. Shemyakina, Z. V. Nagornova, and M. G. Starchenko, “Comparison of the effects of the subjective complexity and verbal creativity on EEG spectral power parameters,” Hum. Physiol., vol. 35, no. 3, pp. 381–383, May 2009.
[6] J. G. Cruz-Garza et al., “Deployment of Mobile EEG Technology in an Art Museum Setting: Evaluation of Signal Quality and Usability,” Front. Hum. Neurosci., vol. 11, p. 527, Nov. 2017.
[7] A. Camurri and G. Volpe, “The Intersection of Art and Technology,” IEEE Multimed., vol. 23, no.
About the Authors
Mario Ortiz (M’09) He received a degree in Industrial Engineering in 2002, and Ph.D. in 2016 from the Universidad Politécnica de Cartagena. Currently, he is Associate Professor at the Mechanical Engineering and Energy Department and member of the Brain-Machine Interface Systems Lab at Miguel Hernández University of Elche (Spain). Mario Ortiz research activities include mathematical transforms applied to electrical signal processing, Neural Network Applications for signal classification and Brain-Machine Interfaces.
José L. Contreras-Vidal (M’88–SM’08) received the Engineer’s degree from Tecnológico de Monterrey (Mexico, 1987), the M.S.E.E. from the University of Colorado (Boulder, 1990), and the Ph.D. degree from Boston University (Boston, 1994). He was an Assistant and Associate Professor at the University of Maryland, College Park, in 1999 and 2005, respectively. He is Cullen Distinguished Professor of Electrical and Computer Engineering at the University of Houston, Houston, TX. He is Associate Editor for IEEE Human Machine Systems, Frontiers in Neuroprosthetics, and the International Journal of Mobile Human Computer Interaction and Senior member of the IEEE EMBS, RAS and SMC Societies. Dr. Contreras-Vidal is the IEEE Robotics and Automation Society representative to the IEEE Systems Council and the University of Houston Director of the BRAIN Center – An Industry-University Cooperative Research Center funded by the National Science Foundation.
José M. Azorín is Director of the Brain-Machine Interface Systems Lab and Professor of the Systems Engineering and Automation Department at Miguel Hernández University of Elche (Spain). He holds an M.Sc from the University of Alicante (1997, Spain) and a Ph.D. from the Miguel Hernández University of Elche (2003, Spain). His current research interests are Brain-Computer Interfaces, Neuro-robotics, Assistive Robotics and Rehabilitation Robotics. He is Distinguished Lecturer of the IEEE Systems Council, and IEEE Senior Member of RAS, EMBS and SMC Societies.
1Videos summarizing each day of the conference are available here: https://vimeo.com/257146685,
https://vimeo.com/258267458, https://vimeo.com/259329743
2Disclaimer: Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the Office of Naval Research.
