Sourav_report_brain_gate.docx

Brain Gate A Technical Seminar Report submitted for fulfilment of the requirements for the Degree of Bachelor of Technology Under Biju Pattnaik University of Technology BY

Sourav Kumar Dey

Roll No. 201518307

August - 2018

Under the guidance of

Shrabani Mahato

NATIONAL INSTITUTE OF SCIENCE & TECHNOLOGY PALUR HILLS, BERHAMPUR, ODISHA– 761008, INDIA

BONAFIED CERTIFICATE

This is to certify that the Seminar entitled “Night Vision Technology “ is a bona fide record by Sourav Kumar Dey (Roll No. BTECH 201518307) under my supervision and guidance, in partial fulfilment of the requirements for the award of Degree of Bachelor of technology from National Institute of Science and Technology under Biju Pattnaik University of Technology for the year 2018.

Shrabani Mahato Asst. Professor Dept. of Computer Science & Engineering

ABSTRACT Who would have thought that in only a decade we would be able to read minds or at least translate thoughts into actions without having to say a word. Advances in Neuro Technology have made these things possible. Brain Gate is a brain implant system developed by the bio-tech company Cyber kinetics in 2003 in conjunction with the Department of Neuroscience at Brown University. The device was designed to help those who have lost control of their limbs, or other bodily functions, such as patients with amyotrophic lateral sclerosis (ALS) or spinal cord injury. The computer chip, which is implanted into the brain, monitors brain activity in the patient and converts the intention of the user into computer commands. Cyber kinetics describes that "such applications may include novel communications interfaces for motor impaired patients, as well as the monitoring and treatment of certain diseases which manifest themselves in patterns of brain activity, such as epilepsy and depression." Currently the chip uses 100 hair-thin electrodes that sense the electro-magnetic signature of neurons firing in specific areas of the brain, for example, the area that controls arm movement. The activities are translated into electrically charged signals and are then sent and decoded using a program, which can move either a robotic arm or a computer cursor.

i

ACKNOWLEDGEMENT I would like to take this opportunity to thank all those individuals whose invaluable contribution in a direct or indirect manner has gone into the making of this project a tremendous learning experience for me. It is my proud privilege to epitomize my deepest sense of gratitude and indebtedness to my faculty guide, Shrabani Mahato for his valuable guidance, keen and sustained interest, intuitive ideas and persistent endeavour. His guidance and inspirations enabled me to complete my report work successfully. I give my sincere thanks to Mr. Ashish Kumar Dass, Seminar Coordinator, for giving me the opportunity and motivating me to complete the project within stipulated period of time and providing a helping environment.

I acknowledge with immense pleasure the sustained interest, encouraging attitude and constant inspiration rendered by Prof. Sangram Mudali (Director) ,Prof. (Dr.) Ajit Kumar Panda (Dean), Prof. (Dr.) Shom Prasad Dash (HOD , School of Computer Science) N.I.S.T. Their continued drive for better quality in everything that happens at N.I.S.T. and selfless inspiration has always helped us to move ahead.

Sourav Kuamr Dey Roll No: 201518307

ii

TABLE OF CONTENTS ABSTRACT .................................................................................... i ACKNOWLEDGEMENT ................................................................ ii TABLE OF CONTENTS ................................................................ iii LIST OF FIGURES ........................................................................ v 1.INTRODUCTION ........................................................................ 1 2. Brain Gate Background……………………………………………..3 3. Brain Gate Neural Interface System………………………………5 4. Technology…………………………………………………………..7 4.1 Implanting The Chip……………………………………….8 4.2 Brain – Computer Interface (BCI)………………………..8 4.3 BCI VERSUS NEUROPROSTHETICS………………….9 5. Components in Brain Gate……….………………………………10 5.1 The Neuro chip…………………………………………..10 5.2 The connector…………………………………………….10 5.3 The converter……………………………………………..10 5.4 The computer……………………………………………..10 6. Working of Brain Gate…………………………………………….11 6.1 How Information is Transmitted…………………………12 7. Research and Experimental Result……………………………...13 8. Application………………………………………………………….14 9.Advantages,Dis Advantages and Future Scope………………...15 9.1 Adv1antages………………………………………………15 9.2 DIS ADVANTAGES………………………………………16 iii

9.3 Future Scope……………………………………………….16 10. Conclusion………………………………………………………...17 REFERENCES………………………………………………………..18

iv

LIST OF FIGURES

Fig.3.1. Brain Gate Pilot Device Fig4.1. Design of a Brain Gate interface Fig5.1.1 The Nuero Chip Fig 5.4.4 The Computer Fig 6.1. Working of Brain Gate Fig 6.2 Motor Cortex Area Fig6.1.1: Reading of Electrical Impulses Fig 6.1.2:Information Transmission Fig 8.1.1 Advantages of Brain Gate

v

Brain Gate

1. INTRODUCTION Brain Gate is a brain implant system built and previously owned by Cyber kinetics, currently under development and in clinical trials, designed to help those who have lost control of their limbs, or other bodily functions, such as patients with amyotrophic lateral sclerosis (ALS) or spinal cord injury. The Brain Gate technology and related Cyber kinetic’s assets are now owned by privately held Brain Gate. The sensor, which is implanted into the brain, monitors brain activity in the patient and converts the intention of the user into computer commands. The Brain Gate Neural Interface System is grounded on Cybernetics podium technology to sense, transmit, analyse and apply the language of neurons. The System consists of a sensor that is entrenched on the motor cortex of the brain and examines brain signals. The principle behind the Brain Gate system is that, signals are generated in the motor cortex and they cannot be sent directly to the arms, hands and legs due to spinal cord injury, stroke or other condition. The brain signals are construed and translated into cursor movements, offering the user a substitute pathway via the Brain Gate System to control a computer simply by thinking, in the same way as individuals who have the ability to move a computer mouse using their hands. Since coming of new technologies, computers are becoming more intelligent than they were in the past. Man and machine interface has been one of the growing fields of research and development in the recent years. Most of the effort has been dedicated to the design of user friendly or ergonomic systems by means of innovative interfaces such as voice recognition, virtual reality. A brain computer interface, sometimes called direct neural interface or brain machine interface is a direct communication pathway between human or animal brain and an external device. Brain computer interface is a staple of science fiction writing. Over the past 15 years, productive brain computer interface research program have arisen. Present day brain computer interfaces determine the intent of the user from a variety of different electrophysiological signals. These signals include slow cortical potentials, P300 potentials or beta rhythms recorded from the scalp. They are translated in real time into commands that operate a computer display or any other device. In one way brain computer interface, computer either accepts commands from brain or sends signals to 1

Brain Gate

it. In two way brain computer interface, brains and external devices exchange information in both directions. With assistive technologies computers adapt and change to user’s needs, from uniquely designed hardware peripherals to innovative software. Some scientists claim that it will not take long before computers become more intelligent than humans and humans can take advantage of these machines. A repercussion of this would be a world where humans and machines are getting melt with each other. An example of this is the Brain Gate system which is a clinical trial to turn thoughts into action. Many different disorders can disrupt the neuro muscular channels through which the brain communicates with and controls its external environment. Amyotrophic lateral sclerosis (ALS), brainstem stoke, brain or spinal cord injury and numerous other diseases impair the neural pathways that control the muscles or impair the muscles themselves. Those most severely affected may lose all voluntary muscles control, unable to communicate in any way. In the absence of methods for repairing the damage done by these disorders, a variety of methods for monitoring brain activity might serve as a BCI. Brain Gate is a brain implant system developed by biotech computer cyber kinetics in 2003 in conjunction with the department of neuro science at Brown University. The Brain Gate system is a boon to the paralyzed. It is a mindto-movement system that allows a quadriplegic man to control a computer using his thoughts. It is based on cyber kinetics platform technology to sense, transmit, analyse and apply the language of neurons.

2

Brain Gate

2. Brain Gate Background The Brain Gate technology platform was designed to take advantage of the fact that many patients with motor impairment have an intact brain that can produce movement commands. This allows Brain Gate system to create output signal directly from the brain, bypassing the route through the nerves to the muscles that cannot be used in paralyzed people. Brain Gate is a culmination of ten years of research by Dr. John Donoghue who is the chairman of the neuroscience department at Brown University and chief scientific officer for cyber kinetics. Dr. Gerhard Freighs helped him by experimenting on monkeys to control the cursor by thoughts alone. These researches cofounded cyber kinetics, Inc.in. The company bears all the expenses required for the study. According to the Cyber kinetics website three patients have been implanted with the Brain Gate system. The company has confirmed that one patient (Matthew Nagle) has a spinal cord injury while another has advanced ALS. The implant, Brain Gate, allowed Matthew Nagle, a 25 year old Massachusetts man who has been paralyzed from the neck down since 2001, because of a severe spinal cord injury, to control a cursor on a screen and to open and close the hand on a prosthetic limb just by thinking about the actions. After few minutes spent calibrating the implant, Mr Matthew Nagle could read emails and plays the computer game Pong. He was able to draw circular shapes using a paint program and could also change channel and turn up the volume on a television, even while talking to people around him. After several months he could also operate simple robotic devices such as prosthetic hand, which he used to grasp and move objects. With practice the user can refine movements using signals from only a sample of cells. Brain Gate is currently recruiting patients with a range of neuromuscular and neuro degenerative conditions for pilot clinical trials being conducted under an Investigational Device Exemption (IDE) in the United States. Cyber kinetics hopes to refine the Brain Gate in the next two years to develop a wireless devise that is completely implantable and doesn’t have a plug, making it safer and less visible. And

3

Brain Gate

once the basics of brain mapping are worked out, there is potential for a wide variety of further applications. The system is designed to restore functionality for a limited, immobile group of severely motor-impair individuals. It is expected that people using this system will employ a personal computer as a gateway to a range of self directed activities. These activities extend beyond typical computer functions. Cyber kinetics is further developing the Brain Gate system to provide limb movement to people with severe motor disabilities. The goal of this program would be to allow these individuals to one day use their arms and hands again. In addition Cyber kinetics is also developing products to allow for robotic control such as a thought controlled wheel chair.

4

Brain Gate

3. Brain Gate Neural Interface System The Brain Gate Neural Interface System is currently the subject of a pilot clinical trial being conducted under an Investigational Device Exemption (IDE) from the FDA. The system is designed to restore functionality for a limited, immobile group of severely motor-impaired individuals. It is expected that people using the Brain Gate System will employ a personal computer as the gateway to a range of self-directed activities. These activities may extend beyond typical computer functions (e.g., communication) to include the control of objects in the environment such as a telephone, a television and lights. The Brain Gate System is based on Cyber kinetics' platform technology to sense, transmit, analyse and apply the language of neurons. The System consists of a sensor that is implanted on the motor cortex of the brain and a device that analyses brain signals. The principle of operation behind the Brain Gate System is that with intact brain function, brain signals are generated even though they are not sent to the arms, hands and legs. The signals are interpreted and translated into cursor movements, offering the user an alternate "Brain Gate pathway" to control a computer with thought, just as individuals who have the ability to move their hands use a mouse. IBM, in partnership with scientists at Switzerland's Ecole Polytechnique Federale de Lausanne's (EPFL) Brain and Mind Institute will begin simulating the brain's biological systems and output the data as a working 3-dimensional model that will recreate the high-speed electro-chemical interactions that take place within the brain's interior. These include cognitive functions such as language, learning, perception and memory in addition to brain malfunction such as psychiatric disorders like depression and autism. From there, the modelling will expand to other regions of the brain and, if successful, shed light on the relationships between genetic, molecular and cognitive functions of the brain. The Brain Gate Neural Interface Device is a proprietary brain-computer interface that consists of an internal neural signal sensor and external processors that convert neural signals into an output signal under the users own control. The sensor consists of a tiny chip smaller than a baby aspirin, with one hundred electrode sensors each thinner than a hair that detect brain cell electrical activity. The chip is implanted on the surface of the brain in the motor cortex area that controls movement. In the pilot version of the device, a cable connects the sensor to an external signal processor in a cart that contains computers. The computers translate brain activity and create the communication output using custom decoding software. 5

Brain Gate

Importantly, the entire Brain Gate system was specifically designed for clinical use in humans and thus, its manufacture, assembly and testing are intended to meet human safety requirements. Five quadriplegics patients in all are enrolled in the pilot study, which was approved by the U.S. Food and Drug Administration (FDA). The brain gate pilot device consists of a Sensor of the size of a contact lens, a cable and pedestal, which connects the chip to the computer, a cart which consists the signal processing unit .

Fig.3.1. Brain Gate Pilot Device

6

Brain Gate

4. Technology In its current form, Brain Gate consists of a sensor implanted in the brain and an external decoder device, which connects to some kind of prosthetic or other external object. The sensor is in the form of a Multi electrode array, formerly known as the Utah Array, which consists of 100 hair-thin electrodes that sense the electromagnetic signature of neurons firing in specific areas of the brain, for example, the area that controls arm movement. The sensor translates that activity into electrically charged signals, which are then sent to an external device and decoded in software. The decoder connects to and can use the brain signals to control an external device, such as a robotic arm, a computer cursor, or even a wheelchair. In essence, Brain Gate allows a person to manipulate objects in the world using only the mind. In addition to real-time analysis of neuron patterns to relay movement, the Brain Gate array is also capable of recording electrical data for later analysis. A potential use of this feature would be for a neurologist to study seizure patterns in a patient with epilepsy. Brain Gate was originally developed by researchers in the Department of Neuroscience at Brown University in conjunction with bio-tech company Cyber kinetics, Inc. Cyber kinetics later spun off the device manufacturing to Blackrock Microsystems, who now manufactures the sensors and the data acquisition hardware.[3] The Brain Gate Company purchased the intellectual property and related technology from Cyber kinetics and continues to own the intellectual property related to Brain Gate.

Fig4.1. Design of a Brain Gate interface 7

Brain Gate

4.1 Implanting The Chip There will be two surgeries, one to implant the Brain Gate and one to remove it. Before surgery, there will be several precautionary measures in order to prevent infection; patients will have daily baths with antimicrobial soap and take antibiotics. In addition, MRI scans will be done to find the best place on the brain for the sensor. Under sterile conditions and general anesthesia, Doctor will drill a small hole into the skull and implant the sensor using the same methods as in the monkey studies. Patients will receive post-surgical care including a CT scan, some blood tests, and wound care in the hospital for 1 to 5 days after surgery. After surgery, one of the study doctors will see the patients at least once a week for six weeks, then monthly and as needed. A nurse will also check the patients regularly and will always carry a 24-hour pager. The skin around the pedestal will need to be carefully monitored during the study. Detailed instructions will be provided so that the patient’s daily care provider can help with skin care.

4.2 Brain – Computer Interface (BCI) A brain-computer interface (BCI), sometimes called a direct neural interface or a brain-machine interface, is a direct communication pathway between a human or animal brain (or brain cell culture) and an external device. In one-way BCIs, computers either accept commands from the brain or send signals to it (for example, to restore vision) but not both. Two-way BCIs would allow brains and external devices to exchange information in both directions but have yet to be successfully implanted in animals or humans. In this definition, the word brain means the brain or nervous system of an organic life form rather than the mind. Computer means any processing or computational device, from simple circuits to silicon chips (including hypothetical future technologies such as quantum computing). Research on BCIs began in the 1970s, but it wasn't until the mid-1990s that the first working experimental implants in humans appeared. Following years of animal experimentation, early working implants in humans now exist, designed to restore damaged hearing, sight and movement. The common thread throughout the research is the remarkable cortical plasticity of the brain, which often adapts to BCIs, treating prostheses controlled by implants as natural limbs. With recent advances in technology and knowledge, pioneering researchers could now conceivably attempt to produce BCIs that augment human functions rather than simply restoring them, previously only the realm of science fiction.

8

Brain Gate

4.3 BCI VERSUS NEUROPROSTHETICS Neuroprosthetics is an area of neuroscience concerned with neural prostheses — using artificial devices to replace the function of impaired nervous systems or sensory organs. The most widely used neuroprosthetic device is the cochlear implant, which was implanted in approximately 100,000 people worldwide as of 2006. There are also several neuroprosthetic devices that aim to restore vision, including retinal implants, although this article only discusses implants directly into the brain. The differences between BCIs and neuroprosthetics are mostly in the ways the terms are used: neuroprosthetics typically connect the nervous system, to a device, whereas the term “BCIs” usually connects the brain (or nervous system) with a computer system. Practical neuroprosthetics can be linked to any part of the nervous system, for example peripheral nerves, while the term "BCI" usually designates a narrower class of systems which interface with the central nervous system. The terms are sometimes used interchangeably and for good reason. Neuroprosthetics and BCI seek to achieve the same aims, such as restoring sight, hearing, movement, ability to communicate, and even cognitive function. Both use similar experimental methods and surgical techniques.

9

Brain Gate

5. Components in Brain Gate 5.1 The Neuro chip A 4-millimeter square silicon chip studded with 100 hair-thin microelectrodes is embedded in the primary motor cortex the region of the brain responsible for controlling movement.

Fig5.1.1 The Nuero Chip

5.2 The connector When the user thinks “move cursor up and down”, the cortical neurons fire in a distinctive pattern: the signal is transmitted through the pedestal plug attached to the skull.

5.3 The converter The signal travels to a shoebox-sized amplifier mounted on the user’s wheelchair, where it’s converted to optical data and bounced by fibre - optic cable to a computer.

5.4 The computer The computer translates brain activity and creates the communication output using custom decoding software.

Fig 5.4.4 The Computer 10

Brain Gate

6. Working of Brain Gate The sensor of the size of a contact lens is implanted in brain’s percental gyrus which control hand and arm movements. A tiny wire connects the chip to a small pedestal secured in the scull. A cable connects the pedestal to a computer. The brain's 100bn neurons fire between 20 and 200 times a second. The sensor implanted in the brain senses these electrical signals and passes to the pedestal through the wire. The pedestal passes this signals to the computer through the cable. The computer translates the signals into a communication output, allowing a person to move a cursor on a computer screen merely by thinking about it.

Fig 6.1. Working of Brain Gate

11

Brain Gate

The Brain Gate neural interface device is a propriety brain-computer interface that consist of an Inter neural signal sensor and External Motor Cortex Processors. The sensor consists of a tiny chip containing 100 microscopic electrodes that detect brain cell electrical activity. The chip is implanted on the surface of brain in the motor cortex area that controls movement. External Processors convert neural signals into an output signal under the users own control. In the pilot version of the device, a cable connects the sensor to an external processor in a cart that contains computers. The computers translate brain activity and create the communication output using custom decoding software.

Fig 6.2 Motor Cortex Area

6.1 How Information is Transmitted When a work is done through any part of body then a potential difference is created in the brain. This potential difference is captured by the electrodes and is transmitted via fibre optic to the Digitizer(external processor).The digitizer converts the signal into some 0’s and 1’s and that is feed into the computer. Thus a new path for propagation of brain commands from the brain to the computer via Brain Gate are created. Now when external devices are connected to the computer, then they work according to the thought produced in the motor cortex.

Fig6.1.1: Reading of Electrical Impulses

Fig 6.1.2:Information Transmission

12

Brain Gate

7. Research and Experimental Result The first reported experiments involving the implantation of the Multielectrode array in one human subject were carried out in 2002 by Kevin Warwick, Mark Gasson and Peter Kyberd.The procedure, which was performed at the Radcliffe Infirmary, involved the implantation of the array in the peripheral nerves of the subject in order to successfully bring about both motor and sensory functionality, i.e. bi-directional signalling. The subsequent full clinical trial of Brain Gate, was led by researchers at Massachusetts General Hospital, Brown University, and the Department of Veterans Affairs and ran from 2004 to 2006, involving the study of four patients with tetraplegia. The results, published in a 2006 article in the journal Nature, showed that a human with tetraplegia was able to control a cursor on a computer screen just by thinking, enabling him to open emails, and to operate devices such as a television. One participant, Matt Nagle, had a spinal cord injury, whilst another had advanced ALS. In July 2009, a second clinical trial (dubbed "BrainGate2") was initiated by researchers at Massachusetts General Hospital, Brown University, and the Providence VA. In November 2011, researchers from the Stanford University Neural Prosthetics Translational Laboratory joined the trial as a second site. This trial is on going. In May 2012, Brain Gate researchers published a study in Nature demonstrating that two people paralyzed by brainstem stroke several years earlier were able to control robotic arms for reaching and grasping. One participant, Cathy Hutchinson, was able to use the arm to drink coffee from a bottle, the first time she was able to drink unaided in 15 years. This took place on site at The Boston Home in Dorchester, Massachusetts, a specialized residence where Ms Hutchinson resided. The study included researchers at Brown University, the Department of Veterans Affairs, Massachusetts General Hospital, Harvard Medical School, and the German Aerospace Centre.

13

Brain Gate

8. Application The mind-to-movement system that allows a quadriplegic man to control a computer using only his thoughts is a scientific milestone. It was reached, in large part, through the brain gate system. This system has become a boon to the paralyzed. The Brain Gate System is based on Cyber kinetics platform technology to sense, transmit, analyse and apply the language of neurons. The principle of operation behind the Brain Gate System is that with intact brain function, brain signals are generated even though they are not sent to the arms, hands and legs. The signals are interpreted and translated into cursor movements, offering the user an alternate Brain Gate pathway to control a computer with thought, just as individuals who have the ability to move their hands use a mouse. Matthew Nagle, a 25-year-old Massachusetts man with a severe spinal cord injury, has been paralyzed from the neck down since 2001.After taking part in a clinical trial of this system, he has opened e-mail, switched TV channels, turned on lights. He even moved a robotic hand from his wheelchair. The application area includes the following:      

In classification of EEG signal. In multimedia communication. In evaluation of spike detection algorithms. Actuated control of mobile robot by human EEG. As a brain controlled switch for asynchronous control. In evaluating the machine learning algorithms.

14

Brain Gate

9.Advantages,Dis Advantages and Future Scope 9.1 Adv1antages         

Controlling remote devices Making and receiving telephone calls Accessing the internet. Turn on or off the lights Control robotic arm Watch and control television Use the pc Locking or unlocking doors Motorized wheelchair

Fig 8.1.1 Advantages of Brain Gate

COMPETITIVE ADVANTAGES The Brain Gate Neural Interface System is being designed to one day allow the user to interface with a computer and/or other devices at a level of speed, accuracy and precision that is comparable to, or even faster than, what is possible with the hands of a non-disabled person. The Brain Gate System may offer substantial improvement over existing assistive technologies. Currently available assistive devices have significant limitations for both the person in need and the caregiver. For example, even simple switches must be adjusted frequently, a process that can be time consuming. In addition, these devices are often obtrusive and may prevent the user from being able to simultaneously use the device and at the same time establish eye contact or carry on conversations with others. 15

Brain Gate

POTENTIAL ADVANTAGES Potential advantages of the Brain Gate System over other muscle driven or brainbased computer interface approaches include: its potential to interface with a computer without weeks or months of training; its potential to be used in an interactive environment, where the user's ability to operate the device is not affected by their speech, eye movements or ambient noise; and the ability to provide significantly more usefulness and utility than other approaches by connecting directly to the part of the brain that controls hand movement and gestures.

9.2 DIS ADVANTAGES     

Expensive. Risky Surgery. Not Wireless yet. Difficulty in adaptation and learning. Limitation in information transform rate.

9.3 Future Scope 

 

Current new advances include a second-generation interface software M*Power controller that will enable users to perform a wide variety of daily activities without assistances of technician. Smaller and wireless device . The user will have an improved control of respiratory system, limb with muscle stimulation or robotics.

16

Brain Gate

10. Conclusion The invention of Brain gate is such a revolution in medical field. The remarkable breakthrough offers hope that people who are paralyzed will one day be able to independently operate artificial limbs, computers or wheelchairs. Brain Gate research in human is a boon to the paralyzed. The idea of moving robots or prosthetic devices not by manual control, but by mere “thinking” (i.e., the brain activity of human subjects) has been a fascinated approach. The Brain Gate technology is used for the future implementation of nueral networks. In the future, the Brain Gate system could be used by those individuals whose injuries are less severe. Next generation products may be able to provide an individual with the ability to control devices that allow breathing, bladder and bowel movements.

17

Brain Gate

11. REFERENCES 1. Brain Gate gets a new lease on life, The Boston Globe, August, 2009. 2. Brain Gate intellectual property and patents

3. Warwick, K, Gasson, M, Hutt, B, Goodhew, I, Kyberd, P, Andrews, B, Teddy, P and Shad, A: "The Application of Implant Technology for Cybernetic Systems", Archives of Neurology, 60(10), pp1369-1373, 2003. 4. Legato, M Editor: ”Principles of Gender-Specific Medicine”, Academic Press, 2017.

18