Omron Plenary Lecture
Fuzzy Bio-interface: Can fuzzy sets be an interface with brain?
sponsored by OMRON Corporation
by Isao Hayashi and Suguru N. Kudo
Recently, many attractive brain-computer interface and brain-machine interface have been proposed[1,2]. The outer computer and machine are controlled by brain action potentials detected through a device such as near-infrared spectroscopy (NIRS) and electroencephalograph (EEG), and some discriminant model determines a control process. However, under the condition where spontaneous action-potentials and evoked-action potentials are contained in brain signal asynchronously, we need a model that serves as an interface between brain and machine for a better stable control in order to prevent runaway reaction of machine. This interface plays a very important role to secure the stability of outer computer and machine. The interface has two kinds of functions:
- a decoding of the response action potentials to the control signal of outside machine and computer, and
- an encoding of the sensor signal of the outside machine and computer to pattern of stimuli in brain and neuronal networks.
Unfortunately, it is very difficult to identify such a function for the interface between machine and living brain and neuronal networks. Here we consider such an interface within the framework of fuzzy system. As a result, our study is supportive of this framework as a strong tool of the bio-interface. During the Japanese fuzzy boom in 1990's, fuzzy logic has been proven effective to translate human experience and sensitivity into control signals of machines. Tsukamoto has argued a concept of fuzzy interface such that fuzzy sets is regarded as a useful tool to intermediate between language and mathematics. We believe that the framework of fuzzy system is essential for BCI and BMI, thus name this technology “fuzzy bio-interface.”
In this lecture, we introduce a fuzzy bio-interface between a culture dish of rat hippocampal neurons and the khepera robot. We propose a model to analyze logic of signals and connectivity of electrodes in a culture dish, and show the bio-robot hybrid we developed[5,6]. Rat hippocampal neurons are organized into complex networks in a culture dish with 64 planar microelectrodes. A multi-site recording system for extracellular action potentials is used in order to record their activities in living neuronal networks and to supply input from the outer world to the vitro living neural networks. The living neuronal networks are able to express several patterns independently, and such patterns represent fundamental mechanisms for intelligent information processing.
First, we discuss how to indicate the logicality and connectivity from living neuronal network in vitro. We follow the works of Bettencourt et al. such that they classify the connectivity of action potentials of three electrodes on multi-site recording system according to their entropies and have discussed the characteristic of each classification. However, they only discuss the static aspects of connectivity relations among the electrodes but not the dynamics of such connectivity concerning how the strength of electrode connection changes when a spike is fired. To address this issue, we develop a new algorithm using parametric fuzzy connectives, that consist of both t-norms and t-conorms[9,10], in order to analyze those three electrodes (Figure 1 -- Algorithm to Analyze Action Potentials in Cultured Neuronal Network). We have obtained the experimental result such that the parameter(s) of fuzzy connectives become infinity. Given this result, we conclude that a pulse at the 60th channel (60el) propagates to the spreading area: (51el, 59el), (43el, 50el) and (35el, 42el); and that the logic of signals among the electrodes was shifted to the logical sum from the drastic product. Consequently, the logic of signals among electrodes drastically changes from the strong AND-relation to the weak OR-relation when a crowd of the pulses was fired.
Next, to control a robot, several characteristics of the living neuronal networks are represented as fuzzy IF-THEN rules. There are many works of robots that are controlled by the responses from living neuronal networks[11-15]. Unfortunately, they have not yet achieved a certain task that experimenter desired. We show a robot system that controlled by a living neuronal network through the fuzzy bio-interface in order to achieve such a task (Figure 2 -- Living Neuronal Network and Robot). This fuzzy bio-interface consists of two sets of fuzzy IF-THEN rules:
- to translate sensor signals of robot into stimuli for the living neuronal network, and
- to control (i.e. to determine the action of) robot based on the responses from the living neuronal network.
We estimated the learning of living neuronal networks with an example of straight running with neuro-robot hybrid. Among 20 trials, the robot completed the task 16 times, and it crashed on the wall and stopped there 4 times. In this result, we may conclude that the logic of signals among living neuronal networks represented as fuzzy IF-THEN rules for the fuzzy bio-interface is rather efficient and effective comparing to the other similar works. In such works, the success rate of 80% is considered extremely high.
- M.A.Lebedev, J.M.Carmera, J.E.O'Doherty, M.Zacksenhouse, C.S.Henriquez, J.C.Principe, and M.A.L.Nicolelis: Cortical Ensemble Adaptation to Represent Velocity of an Artificial Actuator Controlled by a Brain-machine Interface, Journal of Neuroscience, Vol.25, No.19, pp.4681-4693, 2005.
- L.R.Hochberg, M.D.Serruya, G.M.Friehs, J.A.Mukand, M.Saleh, A.H.Caplan, A.Branner, D.Chen, R.D.Penn, J.P.Donoghue: Neuronal Ensemble Control of Prosthetc Devices by a Human with Tetraplegia, Nature, Vol.442, pp.164-173, 2006.
- Y.Tsukamoto: Fuzzy Sets as an Interface between Language Model and Mathmatics Model, Proc. of the 24th Fuzzy System Symposium, pp.251-254, 2008.
- I.Hayashi, M.Kiyotoki, A.Kiyohara, M.Tokuda, and S.N.Kudoh: Acquisition of Logicality in Living Neuronal Networks and its Operation to Fuzzy Bio-Robot System, Proc. of 2010 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE2010) in 2010 IEEE World Congress on Computational Intelligence (WCCI2010), pp.543-549, 2010.
- S.N.Kudoh, I.Hayashi, and T.Taguchi: Synaptic Potentiation Re-organized Functional Connections in a Cultured Neuronal Network Connected to a Moving Robot, Proc. of the 5th International Meeting on Substrate-Integrated Micro Electrode Arrays (MEA2006), pp.51-52, Reutlingen, Germany, 2006.
- S.N.Kudoh, C.Hosokawa, A.Kiyohara, T.Taguchi, and I.Hayashi: Biomodeling System - Interaction between Living Neuronal Network and Outer World, Journal of Robotics and Mechatronics, Vol.19, No.5, pp.592-600, 2007.
- S.N.Kudoh and T.Taguchi: Operation of Spatiotemporal Patterns Stored in Living Neuronal Networks Cultured on a Microelectrode Array, Advanced Computational Intelligence and Intelligent Informatics, Vol.8, No2, pp.100-107, 2003.
- L.M.A.Bettencourt, G.J.Stephens, M.I.Ham, and G.W.Gross: Functional Structure of Cortical Neuronal Networks Grown in Vitro, Phisical Review, Vol.75, p.02915, 2007.
- B.Schweizer and A.Sklar: Associative Functions and Statistical Triangle Inequalities, Publicationes Mathematicae Debrecen, Vol.8, pp.169-186, 1961.
- I.Hayashi, E.Naito, and N.Wakami: Proposal for Fuzzy Connectives with a Learning Function Using the Steepest Descent Method, Japanese Journal of Fuzzy Theory and Systems, Vol.5, No.5, pp.705-717, 1993.
- D.J.Bakkum, A.C.Shkolnik, G.Ben-Ary, P.Gamblen, T.B.DeMarse, and S.M. Potter: Removing Some `A' from AI: Embodied Cultured Networks, in Embodied Artificial Intelligence, editered by F.Iida, R.Pfeifer, L.Steels, and Y.Kuniyoshi, New York, Springer, pp.130-145, 2004.
- T.B.DeMarse and K.P.Dockendorf: Adaptive Flight Control with Living Neuronal Networks on Microelectrode Arrays, Proc. of 2005 IEEE International Joint Conference on Neural Networks (IJCNN2005), pp.1549-1551, Montreal, Canada , 2005.
- Z.C.Chao, D.J.Bakkum, and S.M. Potter: Shaping Embodied Neural Networks for Adaptive Goal-directed Behavior, PLoS Comput Biol, Vol.4, No.3, e1000042, 2008.
- P.Marks: Rat-brained Robots take Their First Steps, New Scientist, Vol.199, No.2669, pp.22-23, 2008.
- K.Warwick: Implications and Consequences of Robots with Biological Brains, Journal of Ethics and Information Technology, Vol.12, No.3, pp.223-234, 2010.
Isao Hayashi is Professor of Informatics at Kansai University, Japan. After he received his B.Eng. degree in Industrial Engineering from Osaka Prefecture University, he worked at Sharp Corporation, Japan. After he received his M.Eng. degree from Osaka Prefecture University in 1987, he was a Senior Research Fellow of the Central Research Laboratory of Matsushita Electric Industrial (Panasonic) Co. Ltd and proposed a neuro-fuzzy system on intelligent control.
He received his D.Eng. degree based on his contributions to the neuro-fuzzy model from Osaka Prefecture University in 1991. He then joined Faculty of Management Information of Hannan University in 1993 and joined Faculty of Informatics of Kansai University in 2004. He is an editorial member of International Journal of Hybrid Intelligent Systems, Journal of Advanced Computational Intelligence and Intelligent Informatics, and has served on many conference program and organizing committees. He is the president of Kansai Chapter of Japan Society for Fuzzy Theory and Intelligent Informatics (SOFT), and the chair of the Technical Group on Brain and Perception in SOFT. He research interests include visual models, neural networks, fuzzy systems, neuro-fuzzy systems, and brain-computer interface.
Suguru N. Kudoh received his Master‘s degree in Biophysical Engineering in 1995 and PhD from the Osaka university in 1998. He was a research fellow of JST(Japan science and technology agency) from 1997 to 1998, and a research scientist of National Institute of Advanced Industrial Science and Technology (AIST) from 1998 to 2009. Now he is an associate professor at Kwansei Gakuin university.
The aim of his research is to elucidate relationship between dynamics of neuronal network and brain information processing. He analyses spatio-temporal pattern of electrical activity in rat hippocampal cells cultured on multi-electrode arrays or acute slice of basal ganglia. He is also developing Bio-robotics hybrid system in which a living neuronal network is connected to a robot body via control rules, corresponding to agenetically provided interfaces between a brain and a peripheral system. He believes that mind emerges from fluctuation of dynamics in hierarchized interactions between cells.