03-09-2016, 04:01 PM
1452522375-jinsaseminarreport.docx (Size: 359.65 KB / Downloads: 4)
INTRODUCTION
1.1 AUGMENTED REALITY
Computers are becoming increasingly portable and ubiquitous, as recent progress in hardware technology has produced computers that are small enough to carry easily or even to wear. However, these computers often referred to as PDAs (Personal Digital Assistant) or palmtops are not suitable for traditional user-interface techniques such as the desk-top metaphor or the WIMP (window, icon, mouse, and a pointing device) interface. The fundamental limitations of GUIs are: Explicit operations, Unaware of the real world situations, Gaps between the computer world and the real world. Inorder to address these problems, a research field called computer augmented environments has been emerged.
Augmented reality (AR) is a variant of virtual reality that uses see-through head mounted displays to overlay computer generated images on the user's real sight. AR systems currently developed use only locational information to generate images. This is because the research focus of AR is currently on implementing correct registration of 3D images on a real scene. However, by incorporating other external factors such as real world IDs, the usefulness of AR could be much more improved.
The goal of Augmented Reality (AR) is to improve and enhance our perception of the surroundings by combining sensing, computing and display technologies.Most AR research addresses human vision, as it is generally considered to be our most important sense. Visual systems are also the focus in this overview, but it is worth noting that other stimuli, such as feedback from auditory, tactile or olfactory displays, may be equally or even more important, depending on the specific scenario and individual.The characteristics of these systems can be further understood from three classical and widely used criteria for AR systems [Azuma 1997]:
1 “Combines virtual and real”
AR requires display technology that allows the user to simultaneously see virtual and real information in a combined view. Traditional displays can show only computer-generated images and are thus insufficient for AR.
2 “Registered in 3-D”
AR relies on an intimate coupling between the virtual and the real that is based on their geometrical relationship. This makes it possible to render the virtual content with the right placement and 3D perspective with respect to the real.
3.“Interactive in real time”
The AR system must run at interactive frame rates, such that it can superimpose information in real-time and allow user interaction.
The fundamental idea of AR is to combine, or mix, the view of the real environment with additional, virtual content that is presented through computer graphics. Its convincing effect is achieved by ensuring that the virtual content is aligned and registered with the real objects. As a person moves in an environment and their perspective view of real objects changes, the virtual content should also be presented from the same perspective.The Reality-Virtuality Continuum [Milgram and Kishino 1994] spans the space between reality, where everything is physical, and virtual reality, where virtual and synthesized computer graphics replace the physical surroundings. Mixed reality is located between them, and includes AR and augmented virtuality.
AR adds virtual content to a predominantly real environment, whereas augmented virtuality adds real content to a predominantly virtual environment. Although both AR and augmented virtuality are subsets of mixed reality by definition, most of the research in the area focuses on AR, and this term is therefore often used interchangeably with mixed reality.
AR techniques exploit the spatial relationships between the user, the digital information, and the real environment, to enable intuitive and interactive presentations of data. An AR system can, for example, achieve medical see-through vision, by using a special display in which the images displayed by the computer are seen overlaid on the patient [State et al. 1996, Stetten et al. 2001], as shown in Figure 1 and Figure 2.Such configurations rely on the proper acquisition and registration of internal medical imagery for the relevant perspective, and careful calibration to establish the geometrical relationship between the display, the viewer, and the patient, to ensure that the correct image is accurately presented.
1.2TECHNOLOGY
Hardware
Hardware components for augmented reality are: processor, display, sensors and input devices. Modern mobile computing devices like smartphones and tablet computerscontain these elements which often include a camera and MEMS sensors such as accelerometer, GPS, and solid state compass, making them suitable AR platforms.[9]
Display
Various technologies are used in Augmented Reality rendering including optical projection systems, monitors, hand held devices, and display systems worn on the human body.
Head-mounted
A head-mounted display (HMD) is a display device paired to a headset such as a harness or helmet. HMDs place images of both the physical world and virtual objects over the user's field of view. Modern HMDs often employ sensors for six degrees of freedom monitoring that allow the system to align virtual information to the physical world and adjust accordingly with the user's head movements.[10][11][12] HMDs can provide users immersive, mobile and collaborative AR experiences.[13]
Eyeglasses
AR displays can be rendered on devices resembling eyeglasses. Versions include eyewear that employ cameras to intercept the real world view and re-display its augmented view through the eye pieces[14] and devices in which the AR imagery is projected through or reflected off the surfaces of the eyewear lens pieces.
HUD
Near eye augmented reality devices can be used as portable head-up displays as they can show data, information, and images while the user views the real world. Many definitions of augmented reality only define it as overlaying the information.[18][19] This is basically what a head-up display does; however, practically speaking, augmented reality is expected to include tracking between the superimposed information, data, and images and some portion of the real world.[20]
CrowdOptic, an existing app for smartphones, applies algorithms and triangulation techniques to photo metadata including GPS position, compass heading, and a time stamp to arrive at a relative significance value for photo objects.[21] CrowdOptic technology can be used by Google Glass users to learn where to look at a given point in time.[22]
In January 2015, Microsoft introduced HoloLens, which is an independent smartglasses unit. Brian Blau, research director of consumer technology and markets at Gartner, said that "Out of all the head-mounted displays that I've tried in the past couple of decades, the HoloLens was the best in its class.".[23] First impressions and opinions have been generally that HoloLens is a superior mdevice to the Google Glass, and manages to do several things "right" in which Glass failed.[23][24]
Contact lenses
Contact lenses that display AR imaging are in development. These bionic contact lenses might contain the elements for display embedded into the lens including integrated circuitry, LEDs and an antenna for wireless communication.[25][26][27][28] Another version of contact lenses, in development for the U.S. Military, is designed to function with AR spectacles, allowing soldiers to focus on close-to-the-eye AR images on the spectacles and distant real world objects at the same time.[29][30] The futuristic short film Sightfeatures contact lens-like augmented reality devices.[31][32]
Virtual retinal display
A virtual retinal display (VRD) is a personal display device under development at the University of Washington's Human Interface Technology Laboratory. With this technology, a display is scanned directly onto the retina of a viewer's eye. The viewer sees what appears to be a conventional display floating in space in front of them.[33]
EyeTap
The EyeTap (also known as Generation-2 Glass)captures rays of light that would otherwise pass through the center of a lens of an eye of the wearer, and substitutes synthetic computer-controlled light for each ray of real light. The Generation-4 Glass[34] (Laser EyeTap) is similar to the VRD (i.e. it uses a computer controlled laser light source) except that it also has infinite depth of focus and causes the eye itself to, in effect, function as both a camera and a display, by way of exact alignment with the eye, and resynthesis (in laser light) of rays of light entering the eye.[35]
Handheld
Handheld displays employ a small display that fits in a user's hand. All handheld AR solutions to date opt for video see-through. Initially handheld AR employed fiducial markers,[36] and later GPS units and MEMS sensors such as digital compasses and six degrees of freedom accelerometer–gyroscope. Today SLAM markerless trackers such as PTAM are starting to come into use. Handheld display AR promises to be the first commercial success for AR technologies. The two main advantages of handheld AR is the portable nature of handheld devices and ubiquitous nature of camera phones. The disadvantages are the physical constraints of the user having to hold the handheld device out in front of them at all times as well as distorting effect of classically wide-angled mobile phone cameras when compared to the real world as viewed through the eye.[37]
Spatial
Spatial Augmented Reality (SAR) augments real world objects and scenes without the use of special displays such as monitors, head mounted displays or hand-held devices. SAR makes use of digital projectors to display graphical information onto physical objects. The key difference in SAR is that the display is separated from the users of the system. Because the displays are not associated with each user, SAR scales naturally up to groups of users, thus allowing for collocated collaboration between users.
Examples include shader lamps, mobile projectors, virtual tables, and smart projectors. Shader lamps mimic and augment reality by projecting imagery onto neutral objects, providing the opportunity to enhance the object’s appearance with materials of a simple unit- a projector, camera, and sensor.
Other applications include table and wall projections. One innovation, the Extended Virtual Table, separates the virtual from the real by including beam-splitter mirrors attached to the ceiling at an adjustable angle.[38] Virtual showcases, which employ beam-splitter mirrors together with multiple graphics displays, provide an interactive means of simultaneously engaging with the virtual and the real. Many more implementations and configurations make spatial augmented reality display an increasingly attractive interactive alternative.
A SAR system can display on any number of surfaces of an indoor setting at once. SAR supports both a graphical visualisation and passive haptic sensation for the end users. Users are able to touch physical objects in a process that provides passive haptic sensation.[7][39][40][41]
Tracking
Modern mobile augmented reality systems use one or more of the following tracking technologies: digital cameras and/or other optical sensors, accelerometers, GPS,gyroscopes, solid state compasses, RFID and wireless sensors. These technologies offer varying levels of accuracy and precision. Most important is the position and orientation of the user's head. Tracking the user's hand(s) or a handheld input device can provide a 6DOF interaction technique.[42][43]
Input devices
Techniques include speech recognition systems that translate a user's spoken words into computer instructions and gesture recognition systems that can interpret a user's body movements by visual detection or from sensors embedded in a peripheral device such as a wand, stylus, pointer, glove or other body wear.[44][45][46][47]
Computer
The computer analyzes the sensed visual and other data to synthesize and position augmentations.
1.3 SOFTWARE AND ALGORITHMS
A key measure of AR systems is how realistically they integrate augmentations with the real world. The software must derive real world coordinates, independent from the camera, from camera images. That process is called image registration which uses different methods of computer vision, mostly related to video tracking. Many computer vision methods of augmented reality are inherited from visual odometry. Usually those methods consist of two parts.
First detect interest points, or fiducial markers, or optical flow in the camera images. First stage can use feature detection methods like corner detection, blob detection, edge detection or thresholding and/or other image processing methods.[50][51] The second stage restores a real world coordinate system from the data obtained in the first stage. Some methods assume objects with known geometry (or fiducial markers) present in the scene. In some of those cases the scene 3D structure should be precalculated beforehand. If part of the scene is unknown simultaneous localization and mapping (SLAM) can map relative positions. If no information about scene geometry is available, structure from motion methods like bundle adjustment are used..Mathematical methods used in the second stage include projective (epipolar)geometry, geometrialgebra, rotationrepresentation with exponentialmap, kalman and particle filters, nonlinear optimization, robust statistics.
Augmented Reality Markup Language (ARML) is a data standard developed within the Open Geospatial Consortium (OGC),[52] which consists of an XML grammar to describe the location and appearance of virtual objects in the scene, as well as ECMAScript bindings to allow dynamic access to properties of virtual objects.
To enable rapid development of Augmented Reality Application, some software development kits (SDK) have emerged
1.4HOW TO "AUGMENT REALITY"
One of the first presentations of augmented reality appears in a special issue of Communications of the ACM , in July, 1993 [24]. We presented a collection of articles that "merge electronic systems into the physical world instead of attempting to replace them." This special issue helped to launch augmented reality research, illustrating a variety of approaches that use one or more of three basic strategies:
1 . Augment the user
The user wears or carries a device, usually on the head or hands, to obtain information about physical objects.
2 . Augment the physical object
The physical object is changed by embedding input, output or computational devices on or within it.
3 . Augment the environment surrounding the user and the object
Neither the user nor the object is affected directly. Instead, independent devices provide and collect information from the surrounding environment, displaying information onto objects and capturing information about the user's interactions with them.
AUGMENTTHEUSER
Beginning with the earliest head-mounted display by Sutherland in 1968 [21], researchers have developed a variety of devices for users to wear, letting them see, hear and touch artificially-created objects and become immersed in virtual computer environments that range from sophisticated flight simulators to highly imaginative games. Some augmented reality researchers have borrowed this "virtual reality" technology in order to augment the user's interactions with the real-world. Charade [2] involves wearing a data glove to control the projection of slides and video for a formal presentation. Charade distinguishes between the natural gestures a user makes when just talking or describing something and a set of specialized gestures that can be recognized by the system, such as "show the next slide" or "start the video".
1.4.2 AUGMENTTHEENVIRONMENT
The third type of augmented reality enhances physical environments to support various human activities. In Krueger's Video Place , a computer-controlled animated character moved around a wall-sized screen in response to a person's movements in front of the screen. Another early example was Bolt's "Put That There" , in which a person sits in a chair, points at objects that appear on a wall-sized screen and speaks commands that move computer-generated objects to specified locations. Elrod and his colleagues use embedded sensors to monitor light, heat and power in the building, both to make the environment more comfortable for the occupants when they are there and to save energy when they are not.
1.4.3AUGMENT THEOBJECT
Another approach involves augmenting physical objects directly. In the early 1970's, Papert [19] created a "floor turtle", actually a small robot,that could be controlled by a child with a computer language called Logo. LEGO/Logo [20] is a direct descendant, allowing children to use Logo to control constructions made with LEGO bricks, motors and gears. Electronic bricks contain simple electronic devices such as sensors (light, sound, touch, proximity), logic devices (and-gates, flip-flops, timers) and action bricks (motors, lights). A child can add a sound sensor to the motor drive of a toy car and use a flip-flop brick to make the car alternately start or stop at any loud noise. Children (and their teachers) have created a variety of whimsical and useful constructions, ranging from an "alarm clock bed" that detects the light in the morning and rattles a toy bed to a "smart" cage that tracks the behavior of the hamster inside. Another approach is "ubiquitous computing" [23], in which specially-created objects are detected by sensors placed throughout the building. PARCTabs fit in the palm of your hand and are meant to act like post-it notes. The notebook-sized version acts like a scratch pad and the Liveboard, a wall-sized version, is designed for collaborative use by several people. A related project at Xerox EuroPARC [17] uses Active Badges (from Olivetti Research laboratory, England) to support collaborative activities, such as sharing documents, and personal memory, such as triggering reminders of important or upcoming events or remembering people or meetings in the recent past.
Another approach is "ubiquitous computing" [23], in which specially-created objects are detected by sensors placed throughout the building. PARCTabs fit in the palm of your hand and are meant to act like post-it notes. The notebook-sized version acts like a scratch pad and the Liveboard, a wall-sized version, is designed for collaborative use by several people. A related project at Xerox EuroPARC [17] uses Active Badges (from Olivetti Research laboratory, England) to support collaborative activities, such as sharing documents, and personal memory, such as triggering reminders of important or upcoming events
1.5RESEARCH CHALLENGES
The user experience for an AR system is primarily affected by the display type, the system’s sensing capabilities, and the means for interaction. The display and sensing techniques determine the effectiveness and realism possible in the blending of the two realities, but may at the same time have ergonomic and social consequences.
It may, in particular, be desirable to achieve walk-up-and-use scenarios that support spontaneous interaction with minimal user preparation [Encarnacao et al. 2000]. Unencumbering technology can also be emphasized, avoiding setups that rely on user-worn equipment [Kaiser et al. 2003, Olwal et al. 2003], such as head-worn displays [Cakmakci and Rolland 2006] or motion sensors [Welch and Foxlin 2002]. It can also be useful to, to the greatest extent possible, preserve the qualities of the real space, while augmenting and assisting the user with unmediated view and control. The excessive use of artificial elements, such as visual reference patterns used for tracking, may, for example, have negative side-effects by cluttering or occluding the real environment that the system is meant to augment. Some display technologies may also result in significantly reduced visual quality due to optical properties, or the use of a downsampled view of the real environment.
1COMBINING GRAPHICS AND THE REAL WORLD
A fundamental characteristic of AR systems is that they allow the user to see a combined view of virtual imagery and real objects.The display hardware used in these systems can be head-worn (retinal displays, miniature displays, and projectors), handheld (displays and projectors) or spatial (displays or projectors in the environment) [Bimber and Raskar 2005].The following sections focus on display technology for handheld and spatial AR systems. The first three sections discuss classical AR display technologies in this context, where optical see-through, video see-through and direct projection display systems make it possible to visually merge the real and the virtual. The last section discusses spatially aware handheld displays, which use a tracked display to provide a virtual view of data associated with the real environment. We are particularly interested in the four approaches described in these sections, since they can be used in configurations that avoid encumbering technology and visual modifications to the environment.
2.1.1 OPTICAL SEE-THROUGH DISPLAYS
Optical see-through capabilities are achieved by using an optical combiner, such as a half-silvered mirror or a holographic material.The role of the combiner is to provide an optically direct view of the environment, with a simultaneous presentation of computer-generated imagery. The combiner is typically able to transmit light from the environment, while also reflecting light from a computer display. The combined light reaches the user’s eyes