Technical Seminar.docx

CHAPTER 1

INTRODUCTION The 21st century depends entirely on technology. The importance of technology is everywhere such as in health, finance, education, corporate world etc. The technology which we use is constantly evolving due to which innovations takes place very frequently in today’s world. Thus the size of computer devices are getting smaller day by day. One such innovation is Sixth Sense Technology. Steve Mann, is regarded as the brain behind this technology, who made the neck worn computers in 1997. Later on this work was continued by Pranav Mistry, a PhD student at the MIT Media Lab, who is now Vice President of Research at Samsung. The information around humans regarding touch, vision, taste, smell, hearing is perceived by using the five senses. But the most important information, i.e. data, which is available online through internet, is not naturally recognized by human senses, withsix sense technology such information will be readily available to the users as the gap between the digital world and the physical world is eradicated. The Sixth Sense technology will revolutionize the world in all aspects, the tasks which take few minutes today will be performed in few milliseconds due to sixth sense technology. This technology will give humans the freedom to use computer anywhere and everywhere, the entire world will be a computer due to this technology. The device is comprised of components such as data projector, camera, mobile component, color markers, mirrors and microphone. All the components have certain functions and they work in collaboration with each other.

1.1 HISTORY The history of “Sixth sense technology” goes back into 1990’s when Steve Mann first attempted to propose a neck worn projector and a camera combination. The idea of implementing computer technologies to daily tasks as ours Sixth Sense was future developed by Pranav Mistry who also appears to be an MIT student as well as Steve Mann. The first prototype of Sixth sense technology was actually bigger than what it looks like today and it was not working properly to use in a daily life. In an article called The Sixth Sense technology Arjun K.R says that “They started with a larger projector that was mounted on a helmet. But that proved cumbersome if someone was projecting data on to a wall then turned to speak to friend-the data would project on the friend’s face”(Arjun K.R 2011).Therefore, Mistry came up with different and more convenient device which is a neck worn portable camera that allows users more space for their daily actions. On the other hand, Mistry first tried out his idea on a simple computer mouse. First, he put two rollers into one mouse and see if he could obtain data and guide the movements of the mouse. Two rollers did not work properly so he decided to use four rollers and see if it could work better. Four rollers gave him the idea that he could use the same idea on fingers and that’s what he next moved on to. The fig 1.1 below shows how four rolls were tried out in Mistry’s experiment

Figure 1.1: Four mouse rollers tried in the experiment

1.2 APPLICATIONS The Sixth Sense prototype implements several applications that demonstrate the usefulness, viability and viability of the system. The Sixth Sense device has a huge number of applications. The following are few of the applications of Sixth Sense Technology. 

Call using palm as dialer: You can use the Sixth Sense to project a keypad onto your hand, and then use that virtual keypad to make a call. It gives the provision of using the hand as a screen onto which the dialer is projected, using which an individual can make a call. Calling a number also will not be a great task with the introduction of Sixth Sense Technology. No mobile device will be required, just type in the number with your palm acting as the virtual keypad. The keys will come up on the fingers. The fingers of the other hand will then be used to key in the number and call as shown in fig.1.2

Fig1.2: Palm as dialer 

Capturing Photos using Fingers: The burden of carrying a camera and keeping it safe and sound on a trip can now be avoided by simply using the Sixth sense technology. The sixth sense computer works as a camera. By making a square with our fingers and specifying the image we want to frame, the camera takes the photo.

Fig1.3: Capture Pictures Using Frame Gesture



Get product information: Sixth Sense uses image recognition or marker technology to recognize products you pick up, and then feeds you information on those products. For example, if you're trying to shop "green" and are looking for paper towels with the least amount of bleach in them, the system will scan the product you pick up of the shelf and give you guidance on whether this product is a good choice for you as shown in the fig 1.4

Fig1.4: Product information 

Reviewing Flight Status: Any background can be used to project a screen, on which the status of flight can be checked by placing the ticket in front of this technology. The system will recognize your boarding pass and let you know whether your flight is on time and if the gate has changed as shown in fig 1.5

Fig1.5: Flight updates



Time without watch: Time can be viewed merely by drawing a circle on the wrist instead of physically wearing a wristwatch. . The computer tracks the red marker cap or piece of tape, recognizes the gesture, and instructs the projector to ash the image of a watch onto his wrist.

Fig1.6: Wrist Watch 

Video newspaper:

It determines the news headline and the displays the appropriate video. The Sixth Sense system also augments physical objects the user is interacting with by projecting more information about these objects projected on them. For example, a newspaper can show live video news or dynamic information can be provided on a regular piece of paper as show in fig 1.5. Thus a piece of paper turns into a video display.

Fig1.7: Video in newspaper 

Zoom in Zoom out: It helps to view images and related things in the way one wants to see.

Fig1.8: Zoom in and out



Feed information on people

Figure 1.9: Information on people Sixth Sense also is capable of "a more controversial use. When you go out and meet someone, projecting relevant information such as what they do, where they work, and also it could display tags about the person floating on their shirt as shown in fig 1.9 It could be handy if it displayed their facebook relationship status so that you knew not to waste your time.

• Get book information

Figure 1.10: Book information Sixth Sense uses image recognition or marker technology to recognize products you pick up, and then feeds you information on books as shown in fig 1.10. The system can project Amazon ratings on that book, as well as reviews and other relevant information

Chapter 2

SYSTEM DESIGN 2.1 System Architecture

Figure 2.1: Six Senses Sixth Sense in scientific (or non-scientific) terms is defined as Extra Sensory Perception or in short ESP. It involves the reception of information not gained through any of the five senses. Nor is it taken from any experiences from the past or known. Sixth Sense aims to more seamlessly integrate online information and tech into everyday life. By making available information needed for decision-making beyond what we have access to with our five senses, it effectively gives users a sixth sense The fig 2.1 shows the additional sixth sense to our five sense.

2.2 Earlier Sixth Sense Prototype

Figure 2.2: Earlier Device Maes MIT group, which includes seven graduate students, were thinking about how a person could be more integrated into the world around them and access information without having to do something like take out a phone as shown in fig 2.2. They initially produced a wristband that would read a Radio Frequency Identification tag to know, for example, which

book a user is holding in a store. They also had a ring that used infrared to communicate by beacon to super-market smart shelves to give you information about products. As we grab a package of macaroni, the ring would glow red or green to tell us if the product was organic or free of peanut traces whatever criteria we program into the system.They wanted to make information more useful to people in real time with minimal effort in a way that doesn’t require any behavior changes. The wristband was getting close, but we still had to take out our cell phone to look at the information. They when they struck on the idea of accessing information from the internet and projecting it. So someone wearing the wristband could pick up a paperback in the bookstore and immediately call up reviews about the book, projecting them onto a surface in the store or doing a keyword search through the book by accessing digitized pages on Amazon or Google books. They started with a larger projector that was mounted on a helmet. But that proved cumbersome if someone was projecting data onto a wall then turned to speak to friend the data would project on the friends face.

2.3 Recent Prototype

Figure 2.3: Present Device Now they have switched to a smaller projector and created the pendant prototype to be worn around the neck. The Sixth Sense prototype is composed of a pocket projector, a mirror and a camera. The hardware components are coupled in a pendant-like mobile wearable device. Both the projector and the camera are connected to the mobile computing device in the users pocket. We can very well consider the Sixth Sense Technology as a blend of the computer and the cell phone. It works as the device associated to it is hanged around the neck of a person and thus the projection starts by means of the micro projector attached to the device. Therefore, in course, you turn out to be a moving computer in yourself and the fingers act like a mouse and a keyboard. The prototype was built from an ordinary webcam and a battery-powered 3M projector, with an attached mirror all connected to an internetenabled mobile phone. The setup, which costs less than $350, allows the user to project information from the phone onto any surface walls, the body of another person or even your hand. Mistry wore the device on a lanyard around his neck, and colored Magic Marker caps on four fingers as shown in fig

2.3 (red, blue, green and yellow) helped the camera distinguish the four fingers and recognize his hand gestures with software that Mistry created.

2.4 Advantages          

The digital information and its objects are integrated into the physical world by using the Sixth sense interface, thereby making the entire world as our own computer. Sixth Sense makes machines like computers to adapt to human needs and not the other way round. Hand gestures are used to communicate with digital information, multi-touch and multi-user interaction are also supported. Data from machine is directly accessed into real time. It is open source and it is cost effective and map idea can be minded anywhere. Our relevant information is provided by the gesturecontrolled wearable computing device that manipulates any surface into a display. It is portable Comparatively easy to carry as can be worn in our neck Even a naïve man with little or no knowledge of mouse and computer can use this device. Need to carry a camera no longer persist. The cost that arose for the making of the sixth sense technology proto type is exceptionally low. A basic sixth sense device sums up to $300.

2.5 Disadvantages    

Excessive use of a technology can cause addiction and can hamper social life as well. It will in-turn diminish humanness. Exposure of rays on surfaces like human arms can lead to health problems. This technology will affect the hardware market and will result in less revenue being generated and lower the cash inflow. It can affect the vision of the user because of its peculiar and better use at night time and in dark areas as compared to mornings and bright places.

CHAPTER 3

IMPLEMENTATION 3.1 Components The main components of Sixth Sense Technology are as follows:      

Camera Color Markers Mobile Component Projector Mirror Microphone

3.1.1 Camera This component is present at the head of the Sixth Sense Technology pendant. The function of this camera is just like that of a human eye, the only difference being it will provide digital information. This camera is capable of capturing the images that falls within its vision and also it provides information about the objects present in front of it. The hand gestures performed by the user can be tracked with this component. Additionally, facial recognition is supported by this device. The information collected by camera is processed by the connected mobile component.

Fig 3.1: Camera 3.1.2 Color Markers This technology makes use of four color markers: red marker, green marker, blue marker and yellow marker. The user can wear these markers at the tip of their fingers which helps the camera to track hand gestures. These gestures can perform various tasks such as painting, taking a picture and many more.

Fig 3.2: Color Markers 3.1.3 Mobile Component One of the most important components of this technology is the mobile module. The function of this component can be carried out by any device such as smartphone, personal digital assistant, laptop etc. as long as it is mobile and web-enabled. It is a processing engine that processes the data obtained from camera and sends the output to projector.

Fig 3.3: Mobile Component 3.1.4 Projector The projector is basically an output gadget which is used to display any information provided by the mobile component. Projector can project information on any kind of physical surface such as a wall, palm of user’s hand, a paper etc. and the user can interact with it.

Fig 3.4: Projector 3.1.5 Mirror Mirror is placed just below the projector. The projector can project the information in any direction with the help of a mirror which can be tilted in any direction as per the user’s requirement.

Fig 3.5: Mirror 3.1.6 Microphone

Fig 3.6: Microphone Clip This component is used only when the projection is to be done on a piece of paper. Microphone in form of a clip is affixed on that piece of paper. The information written by user on that paper is recognized by the camera by interpreting the sound of user’s touch. The camera is also capable of detecting the motion of the paper held in hand.

3.2 Working Sixth sense device analyses whatever the user is watching and it augments the surfaces and physical objects user is interacting with. A number of standard gadgets are connected including a mini projector, camera, mobile phone, microphone and a mirror. The mobile phone acts as a computer and our connection to web where all the information is stored.

Fig 3.6: Working of the Sixth Sense Technology The components in the device are managed by a mobile phone device carried by the user. The microphone and the projector are connected to the mobile phone device. The projector, projects the information enabling surfaces, walls and physical objects around the user to be used as interfaces. The microphone is trained frequently with commands; the device stores these commands which were trained by the user and executes the corresponding action through the projector. The user gives the input to the device and when the input is received to the microphone, the command is processed and is sent to the mobile device .The mobile device interprets the command and is passed on to the projector. The projector output is seen on the screen through the mirror for accurate projection. The tips of the user’s fingers are covered with red, yellow, green and blue colored tapes. Therefore it helps the camera to recognize the hand gestures. The movements of the user’s fingers recognized into gestures that acts as an interaction instruction for the device. The Sixth Sense technology is a blend of computer along with a cell phone.

CHAPTER 4

CONCLUSION AND FUTURE SCOPE 4.1 Conclusion The key here is that Sixth Sense recognizes the objects around you, displaying information automatically and letting you access it in any way you want, in the simplest way possible. Clearly, this has the potential of becoming the ultimate "transparent" user interface for accessing information about everything around us. If they can get rid of the colored finger caps and it ever goes beyond the initial development phase, that is. But as it is now, it may change the way we interact with the real world and truly give everyone complete awareness of the environment around us.

4.2 Future Scope Further development in this technology will lead to evolvement of new markets. Hardware used in the current technology can be perfected as it plays an important role in this technology by interacting with the user. Security of the current technology can further be improved and more accuracy should be aimed for. Enhancements should be made so that visually impaired people can use this technology. The current technology is a little bulky to carry around, so few improvements can make it user friendly.

REFERENCES [1]Sixth Sense 2009 < http://en.wikipedia.org/wiki/SixthSense> [2]Monika Arora, (2012), Basic Principles of Sixth Sense Technology, VSRD-IJCSIT, Vol. 2 (8), 687-693. [3]Abhinav Sharma, Mukesh Agarwal, Anima Sharma, Sachin [4]Gupta, (2013),Sixth Sense Technology, IJRITCC, Vol. 1(4), 277-282. [5]S. Sadhana Rao, (2010),Sixth Sense Technology, Proceedings of the International Conference on Communication and Computational Intelligence, 336-339. [6]Nalen Anand, Neha Gaur, Kavita Choudhary, (2013), Sixth Sense in a State of Radical Emergence International Journal of Information and Computation Technology,Vol. 3,527-532. [7]Sixth Sense Technology 2011< http://www.slideshare.net/ Sanjay Medichetty/sixthsense-technology> [8]Sixth Sense Technology< http://www.engineersgarage.com/ articles/sixth-sensetechnology >