Model Em Neural Network

NEURAL NETWORK MODEL WITH AN ADDITIONAL INFORMATION INTERCHANGE CHANNEL BETWEEN NEURONS K. N. Shevchenko, N. V. Shevchenko, B. V. Shulgin The Urals State Technical University – UPI Russia, Yekaterinburg, Mira street, 19, 620002 E-mail: [email protected] We here present a simple neural network model, the electromagnetic (EM) neuron that includes an additional information interchange channel between neurons by means of the electromagnetic field generating by neuron firing. The numerical solution of the system of the differential equations describing EM neuron is presented. The principle of EM neuron is also implemented in a physical system and the experimental results are compared with the theoretical predictions to demonstrate the validity of the model. We show that phase synchronization of firing patterns in EM neuron may be generated within a network of neurons that do not have direct synaptic connections. The presence of a feedback circuit through the power supply stimulates self-organization of the network. The potential significance of these findings for an understanding of the neural dynamics in the brain is discussed.

Introduction Nowadays the work to create artificial neural nets is underway in two directions: the informational direction and the biological one. The supporters of the first direction assume that artificial systems need not imitate the structure and processes inherent in biological systems. It is of the only importance that, using some means, we can achieve the behavior typical for human beings and other biological systems.

Fig. 1. McCulloch-Pitts analogue model of a biological neuron

The supporters of the second direction suppose that we cannot achieve this on the purely informational level. According to these experts, it is biological structure and its functions that 163

manifest themselves in the phenomena of human behavior, the human capacity to study and adaptability. The neural network model under discussion belongs to the second, biological, direction, W. McCulloch being one of its fathers [1]. See Fig.1 for the classical analogue model of a biological neuron by McCulloch and Pitts. According to the model a neuron of an artificial network contains input synapses 1 possessing weight coefficients w1 , w2 , ...wi and dendrites with output signals x1w1 , x2 w2 , ...xi wi , an adder 2, a nonlinear transfer function 3 with threshold defined θ , and an axon 4 which is coupled to other neurons. In the model axons are considered as reliable transmission cables, where stable propagation of an action potential takes place, and which link brain areas. However, experimental and theoretical data recently published say that the axon architecture is much more complex and its functionality more diverse than it was suspected before [2]. The particular part of the axon, which it plays in the process of neural information transmission, underlies the new neural network model. 1. Neural network model with an additional information interchange channel between neurons The difference between the proposed model [3] and the models known is that in the model of neural network nonlinear transformation of the input signal into the digital (impulse) one is

Fig 2. The structure diagram of a neural network

164

introduced, which allows the creation of additional communication channels between the neurons (i.e. it increases the degree of calculating process paralleling and, therefore, the speed of information processing too). Besides it decreases energy consumption which level is related to the character of the calculating processes in the network. Fig. 2 shows the structure diagram of the neural network model proposed, which consists of an artificial neural network 1, analogue neurons (see fig. 1), a block of axon-like nodes 2, controlled power units 3, uncontrolled power unit 4. Fig. 3 shows the structural diagram of an axon-like node of a neuron. The axon-like node of a neuron consists of chains of circuit in-series for the radio pulse envelope matching and separation 5, a self-oscillator 6 with a self-quenching circuit 8 and with the output of the control signal by adjustable power units 9, the section of the coaxial line 7, and the power bus of the axon-like node 10. The additional circuit of matching and separation of the radio pulse envelope 5 is at the output of an axon-like node.

Fig 3. The structure diagram of an axon-like node

The neural network model works as follows. The information for processing enters in the form of signals the input of the artificial neural network of analogue neurons 1 which function on the basis of the well-known principles of a biological neuron. When the sum of signals at the input of an analogue neuron is more then the level of the given threshold, at its output a sequence of frequency-modulated impulses (activation impulses) forms, the frequency of the impulses depends on the total level of the neuron input signals. Then the train of the frequency-modulated impulses comes to the input of the axon-like node 2 which performs nonlinear transformation. The axon-like node (see fig. 3) works as follows. An activation impulse – spike (then we shall call it as video pulse), coming to the radio pulse envelope matching and separation circuit 5, performs a “shock-free” [4] (without dc circuit switching) repetitive self-oscillator 6 actuation to implement the conditions of electromagnetic oscillation receiving. At the moment of generation the radio pulse, induced in the vibrating system of the self-oscillator – coaxial line 7, is emitted to space by the coaxial line functioning like an antenna, and, at the same time, comes to the input of the 165

following radio pulse envelope matching and separation circuit 5. From the output of the radio pulse envelope matching and separation circuit 5 the selected video pulse comes to the actuation of the following self-oscillator 6 and the process repeats again. From the output of the radio pulse envelope matching and separation circuit 5, which is the last in the chain, the selected video pulse comes to the input of the following neuron or to the output of the artificial neural network directly. Radio pulse emitted to space can be received by any other axon-like node located far away from the first one. Because “shock free” actuation of self-oscillator takes place, the conditions of electromagnetic oscillation receiving are implemented (i.e. in this case self-oscillator is a transceiver [4]) and the first axon-like node is able to receive radio pulses from other axon-like nodes being a part of other neurons of the network. Thus mutual sync [5] of radio pulse radiation frequencies may carry out among great many neurons. The presence of the self-quenching network in each self-oscillator leads not only to synchronization of radio pulse radiation frequency but to synchronization of the radio pulse repetition rate, i.e. by transforming the initial sequence of video pulses from neuron outputs to radio pulses in the process of synchronization the exchange of information between neurons takes place. Many scientists suppose that the synchronized activity of different brain areas is the most probable mechanism for the solution of many cognitive problems [6]. Besides the presence of the selfquenching system leads to the leading part in the process of synchronization of those neurons whose pulse frequency at the output is higher, that is, more “fast” neurons cause the “slow” ones to coincide with them. Because the additional information exchange between neurons occurs by means of electromagnetic field, the exchange speed is equal to the speed of electromagnetic waves propagation. And because the electromagnetic field can provide the information exchange between a great number of neurons (the interconnection of “all to all” type is possible potentially), the degree of calculating process paralleling and the speed of information processing increase. By reaction circuits 9 of axon-like nodes from the outputs of self-quenching networks 8 (see fig. 3) to the units of controlled voltages 3 (see fig. 2) self-regulation of network neuron activity depending on the character of calculating processes takes place. Series connection of several chains of the radio pulse envelope matching and separation circuit, self-oscillator with self-quenching circuit, coaxial line section increases the probability of the information interchange between neurons due to the increase of the radiated radio pulse number during information transmission from the output of the analogue neuron through an axon-like node to the output of the neural network. The reliability of the network functioning also increases since in 166

case of failure of a neuron, other neurons, which can be induced at the moment, can undertake its functions The model offered is implemented by the means of analogue electronics; it is suitable technologically for the microcircuit modification, including nanotechnological means. 2. The mathematical model of EM neuron The mathematical model of a neuron with an additional information interchange channel – the model of an electromagnetic (EM) neuron – is built on the basis of two widely used models by FitzHugh – Nagumo and Van der Pol. The first two-dimensional FitzHugh – Nagumo model [7] is the result of simplification of four-dimensional Hodgkin-Huxley equations [8] and their reduction to the form of two-dimensional oscillator with cubic nonlinearity, one fast and one slow variable. We use the FitzHugh – Nagumo equation in the form [9]:

 du  dt = u (u − a )(1 − u ) − v;   dv = ε (u − I ),  dt

(1)

where u and v are the variables which correspond with quality to the transmembrane voltage and to the variable of ionic current activation respectively; a characterizes the degree of cubic nonlinearity;

ε is the parameter of time scale correlation, which characterize relative speed of activation (deactivation) of ionic current; I is the control parameter responsible for excitation threshold. The phase portrait of the FitzHugh – Nagumo system (1) is shown on fig. 4. The second system describes the Van der Pol oscillator in the following normalization [10]:  dx  dt = y;   dy = −ω 2 x + (λ − x 2 ) y,  dt

(2)

where ω is the base oscillation frequency; λ is the control parameter. In the system (2) there is a stable stationary point when the control parameter λ is of a negative value. When λ = 0 , the point loses its stability and the phase pathway of the system goes to the limit cycle through the bifurcation of Andronov – Hopf (see fig. 5). 167

2

1

1.5

0.8

1 0.6

0.5

u

dx/dt

0.4

0

0.2

-0.5 0

-1

-0.2

-1.5

-0.4 -0.02

0

0.02

0.04

0.06

0.08 v

0.1

0.12

0.14

0.16

-2 -3

0.18

Fig. 4. The phase portrait of the FitzHugh – Nagumo system in the plane of the dynamic variables v − u , obtained by the numerical solution of the system (1) for the following parameter values of a dynamic system a = 0.001 , ε = 0.001 , I = 0.001

-2

-1

0 x

1

2

3

Fig. 5. The phase portrait of the Van der Pol system in the plane of dynamic variables x − dx / dt , obtained by the numerical solution of the system (3) for the following parameter values of a dynamic system a = 0.001 , ε = 0.001 , I = 0.001 ,

ω =1

The mathematical model, which consists of four nonlinear differential equations and features the dynamics of EM neuron, looks like this:  dx  dt = y;   dy = −ω 2 x + (u − x 2 ) y;  dt   du = u (u − a)(1 − u ) − v;  dt  dv  = ε (u − I ).  dt

(3)

We get the solution of the equations of the spike generation in Ranvier’s node choosing relevant parameters a , ε , and I . The value of the fast variable u obtained by solution of the first subsystem is used as the control parameter λ in the Van der Pol equation modeling the process of generation of a high-frequency radio pulse. Mathematical modeling was performed by numerical solution of four differential equations system (3) using the Runge – Kutta method of the fourth-order with fixed pitch h = 0.01 . The results of the numerical solution of the system (3) for the values of dynamic system parameters a = 0.001 , ε = 0.001 , I = 0.001 , ω = 1 are given in fig. 6-9.

168

One should note that for the system (3) the significant difference of the time scales of the dynamic variables x , u , v , occurs:

τ 1 << τ 2 < τ 3 .

(4)

The time scale τ 1 of the Van der Pol system is not connected directly to the variables of the system (1) it is affected only by the control parameter u defining the period and conditions of the

1

0.8

0.8

0.6

0.6

0.4

0.4 u

u

1

0.2

0.2

0

0

-0.2

-0.2

-0.4

0

1000

2000

3000

4000

5000 t

6000

7000

8000

-0.4

9000 10000

50

2

1.5

1.5

1

1

0.5

0.5

0

0

-0.5

-0.5

-1

-1

-1.5

-1.5

0

500

1000

1500

2000

2500 t

3000

3500

4000

4500

-2

5000

Fig. 8. The dynamic variable x modeling multiple burst of radio pulses emitted by EM neuron, obtained by the numerical solution of the system (3) for the following parameter values a = 0.001 , ε = 0.001 , I = 0.001 ,

ω =1

169

150

200

250 t

300

350

400

450

500

I = 0.001

2

-2

100

Fig. 7. The dynamic variable u modeling a single spike of EM neuron obtained by the numerical solution of the system (1) for the following parameter values a = 0.001 , ε = 0.001 ,

x

x

Fig. 6. The dynamic variable u modeling burst activity of EM neuron obtained by the numerical solution of the system (1) for the following parameter values a = 0.001 , ε = 0.001 , I = 0.001

0

0

50

100

150

200

250 t

300

350

400

450

500

Fig. 9. The dynamic variable x modeling a single radio pulses emitted by EM neuron, obtained by the numerical solution of the system (3) for the following parameter values of a dynamic system a = 0.001 , ε = 0.001 , I = 0.001 , ω = 1

system (2) actuation and not affecting the base oscillations frequency. Therefore the τ 2 to τ 1 ratio (the inequality (4)) can reach a significant value: during the model realization the ratio τ 2 τ 1 = 10 6 was obtained. The ratio of time scales of the dynamic variables u and v of the two-dimensional FitzHugh – Nagumo model is determined using the parameter ε (see the system (1)): τ 3 τ 2 = 1 ε = 10 3 . These significant differences of the time scales of the dynamic variables are shown in fig. 10, where the numerical solutions for the three dynamical variables of the system (3) are given.

2 x u v

1.5

Values variable x,u,v

1 0.5 0 -0.5 -1 -1.5 -2

0

50

100

150 t

200

250

300

Fig. 10. The time history of the dynamic variables x , u , v obtained by the numerical solution of the system (3) for the following parameter values of dynamic system a = 0.001 , ε = 0.001 , I = 0.001 , ω = 1

The time scales of the dynamic variables will characterize the speed of the processes passing in the functional blocks of the neural network model described above (see fig. 2): τ 3 is the change of voltages in power supply units controlled 3, τ 2 is the spike generation in the artificial analogue neural network 1, τ 1 is radio pulses radiation in axon-like nodes 2. Thus, the time scales of the dynamic variables of the system (3) will define the technological requirements for the element base, which is the important fact of the implementation of the neural network model under discussion.

3. Discussion of the experimental data The experimental verification was implemented using the physical modeling of the system consisting of two EM neurons with geometrical sizes of axons, 1000 – 10000 as much as the

170

biological ones. The working models showed the following phenomena which are characteristic of the biological neurons and the biological neural networks:

-

mutual synchronization of spike running frequencies in two neurons located far away one another (it is modeling neurotransmission through “gap junctions”);

-

spike induction by an active neuron in the second neuron in the idle state (it is modeling “ephaptic” transmission);

-

the spike observed at the outputs of the radio pulse envelope matching and separation circuits and at the output of a neuron doesn’t change its amplitude, duration, or form under 20 fold change the frequency of nervous impulses.

The oscillograms in the fig. 11-16 illustrate the results of the experiments. By comparing the experimental data with the results of the numerical solution of the differential equation system, one can notice their great coincidence. In experimental conditions the dynamics of the phase synchronization process, which, as it was above mentioned, is demonstrated by the simplest system consisting of two EM neurons without direct synaptic links between was investigated. Fig. 15 is a photo of a phase portrait of a dynamic system consisting of two EM neurons in the plane of parameters U1(t) – dU1(t)/dt in the synchronous spike propagation mode (fig. 15, on the left) and in the asynchronous mode (fig. 15, on the right). The impulse voltage from the output of the first EM neuron U1(t) and the signal from the output of a differential chain proportional to the time derivative dU1(t)/dt were applied to the inputs of the horizontal and vertical deflection of the electronic oscillograph beam respectively.

Fig. 11. The burst activity of EM neuron

Fig. 12. A single spike of EM neuron

171

Fig. 13. The sequence radio pulses emitted by an EM neuron in the burst activity mode

Fig. 14. A single radio pulse of EM neuron

It is clear (fig. 15) that in the mode of phase synchronization the take-off runs of the pathways of the dynamic system movement is narrower than those in the asynchronous mode. The rather wide residual noise “path” in the synchronous mode (fig. 15, on the left) may be explained by the fact that the EM neurons are the devices of high-sensitivity and in course of the experiment they were under the influence of the natural field of electromagnetic noise. The difference between the synchronous and asynchronous modes of spike propagation is reflected evidently by the spectral characteristics of the two EM neurons radiation. In the synchronous mode the radiation spectrum is the discrete one and its envelope is defined by the spike durability, the internal structure is a lattice of equidistant spectral constituents being apart at the spike repetition frequency (fig. 16, 17; on the left ). In the asynchronous mode the radiation spectrum is solid and its envelope is as previous one and its internal structure is a spectrum of quasiperiodic signal of “beating” between the frequencies of spike running in each neuron with (in the experiment) external noise which ultimately has the spectrum of random noise signal (fig. 16, 17; on the right).

Conclusion The article describes the neural network model with an additional information interchange channel between neurons. It is shown that the experimental data is in accordance with the results of the numerical solution of the model system of differential equations which describe the dynamics of EM neurons. The principal possibility of physical implementation of artificial neural nets from EM

172

Fig. 15. A phase portrait of a dynamic system consisting of two EM neurons in the plane of parameters U1(t) dU1(t)/dt; on the left – under mutual synchronization of spike propagation, on the right – the same in the asynchronous mode

Fig. 16. The one-sided spectrum relative to the carrier frequency of radiation f=47.0 MHz of the dynamic system consisting of two EM neurons: on the left – under mutual synchronization of spike propagation, on the right – in the asynchronous mode

Fig. 17. “Subtle” structure of the spectrum near the carrier frequency of radiation f=47.0 MHz of the dynamic system consisting of two EM neurons: on the left – under mutual synchronization of spike propagation, on the right – in the asynchronous mode

173

neurons, which are able implementing the process of mutual phase-loch synchronization and information transfer using electromagnetic field. This ability of a network may become the fundamental one for the solution of the binding problem [11] – the problem of integration of symptoms of attention, identification, and memorizing which allow the creation of artificial neural networks with a wide range of intellectual abilities.

Acknowledgements The authors express their thanks to G.I. Kravchenko for fruitful cooperation, useful discussions and valuable help during the period investigation.

Bibliographic list 1. McCulloch W.S. A Logical calculus of the Ideas immanent in neural nets/ W.S. McCulloch, W. Pitts // Bull. Math. Biophys. 1943. № 5. P. 115-137. 2. Debanne D. Information processing in the axon/ Debanne D. Nature Reviews Neuroscience. 2004. Vol. 5. № 4. P. 304-316. 3. Neural network model/ Pa t en t R U 230 94 57, B ul l . N o.3 0, 20 07, K.N .Shevchenko, N.V. Shevchenko, B.V. Shulgin. 4. Superregenerators / M.K. Belkin, G.I. Kravchenko, J.G. Skorobutov, B.A. Stryukov. M.: Radio and communication, 1983. 248 p. 5. Dvornicov A.A. Self-oscillators in radio engineering / A.A. Dvornicov, G.M. Utkin. M.: Radio and communication, 1991. 224 p. 6. Models of neural activity dynamics in course of information processing in brain – the decade results. Reviews of the topical problems / G.N. Borisyuk, R.M. Borisyuk, Y.B. Kazanovitch, G.R. Ivanitsky // UFN. 2002 . Vol. 172, № 10. P. 1189-1214. 7. FitzHugh R.A. Impulses and physiological states in theoretical model of nerve membrane / R.A. FitzHugh // Biophys. J. 1961. Vol. 1. P. 445 -466. 8. Hodgkin A.L. A quantative description of membrane current and its application conduction and excitation in nerve / A.L. Hodgkin, A.F. Huxley // J. Physiol. 1952. Vol. 117, P. 500-544. 9. Keener J. Mathematical Physiology / J. Keener, J. Sneyd // Springer, 1998. P. 594-607. 10. Kuznetsov A.P. Nonlinear oscillations: training aid for higher educational institutions/ A.P. Kuznetsov, R.P Kuznetsov, N.M. Ryskin. M.: FML publishing house, 2002. 292 p. 11. McFadden J. Synchronous Firing and Its Influence on the Brain’s Electromagnetic Field. Evidence for an Electromagnetic Field Theory of Consciousness / J. McFadden // J. of Consciousness Studies. 2002. Vol. 9. № 4. P. 23-50.

174