Codefocus Acessibility Web

.N ET

Windows W indows A Accessibility ccessibility

Volume 5 / Issue 4

What's New in Windows 7 Automation API Testing Tools vs. Accessibility Guidelines Internet Explorer 8 New Accessibility Features

TABLE OF CONTENTS

Features 6

A Letter from the Director of Accessibility at Microsoft “Our vision is to create innovative technology that is accessible to everyone and that adapts to each person’s needs. Accessible technology eliminates barriers for people with disabilities and it enables individuals to take full advantage of their capabilities.” —Bill Gates, Chairman, Microsoft Corporation

Rob Sinclair

8

Open Accessibility Open accessibility is a term that describes both the need for and the realization of interoperability in assistive technologies. This article describes open accessibility and the ways Microsoft and other software development organizations, like Novell and the AIA, are working together to help user access technology across platforms.

Norm Hodne

10 Accessibility 101 What is accessibility? Learn about the fundamentals of accessibility, the people that accessible technologies help, and the things you can do to help them interact smoothly and successfully with each other.

Jennifer Linn, Annuska Perkins

16 Windows Automation API 3.0 Overview The ecosystem of Windows® automation technologies includes Microsoft® Active Accessibility and Windows implementations of the UI Automation specification, which provides more flexible programmatic access to UI elements.

Masahiko Kaneko

22 What's new in Windows 7 Automation API Windows® 7 offers end–to-end accessibility with better performance, seamless interoperability, and improved framework design.

Nazia Zaman, Matthew Karr

28 Creating Accessibility-aware Silverlight 2 Content This article provides you with guidelines for taking advantage of Silverlight(TM) 2 features and building an accessible Silverlight application.

Mark Rideout

36 Making Custom Controls Accessible While there are many ways to make custom controls accessible, choosing the right solution can be tricky. With Microsoft® technologies, you can implement UI Automation, create or override properties with Dynamic Annotation, or close the gap between Microsoft Active Accessibility and UI Automation.

LeAnne Fossmeyer, Michael Bernstein, David Tseng, Vidhya Sriram

42 Microsoft Accessibility Testing Tools The few informal tests described in this article can expose a plethora of usability and accessibility shortcomings, oversights, and other issues in your application. But how do you test, assess, and rectify them?

Karl Bridge

48 Internet Explorer 8 New Accessibility Features Windows® Internet Explorer® 8 has a lot of cool new features that make Web page browsing more accessible, including Caret Browsing, Zoom version 2, high DPI, and support for ARIA and UI Automation.

JP Gonzalez-Castellan

56 A Pragmatic Approach to WPF Accessibility A Microsoft MVP shares his experience addressing accessibility with Windows® Presentation Foundation and adaptive technology vendors.

Alvin Bruney

62 Writing a UI Automation Provider UI Automation providers retrieve data about UI elements for assistive technologies and automation tools. This article walks through code samples for a UI Automation provider for a Win32-based custom control.

Brendan McKeon

70 Creating UI Automation Client Applications UI Automation enables a client application to investigate and manipulate the UI of all running applications on the system. This article walks through codes sample that demonstrate the basic features of UI Automation for client applications.

Matthew Karr

Departments 7 Code Compilers 59 Advertisers Index

US subscriptions are US $29.99 for one year. Subscriptions outside the US pay US $44.99. Payments should be made in US dollars drawn on a US bank. American Express, MasterCard, Visa, and Discover credit cards are accepted. Bill me option is available only for US subscriptions. Back issues are available. For subscription information, email [email protected] or contact customer service at 832-717-4445 ext 10. Subscribe online at www.code-magazine.com CoDe Component Developer Magazine (ISSN # 1547-5166) is published bimonthly by EPS Software Corporation, 6605 Cypresswood Drive., Suite 300, Spring, TX 77379. POSTMASTER: Send address changes to CoDe Component Developer Magazine, 6605 Cypresswood Drive., Suite 300, Spring, TX 77379.

4

Table of Contents

www.code-magazine.com

ONLINE QUICK ID 0811012

A Letter from the Director of Accessibility at Microsoft “Our vision is to create innovative technology that is accessible to everyone and that adapts to each person’s needs. Accessible technology eliminates barriers for people with disabilities and it enables individuals to take full advantage of their capabilities.” —Bill Gates, Chairman, Microsoft Corporation t Microsoft®, our commitment to creating the most accessible development platform in the world runs deep. In 1997, Microsoft Active Accessibility® was released, providing a solid foundation for the first generation of ac-

A

cessible applications. Eleven years later, Microsoft Active Accessibility still powers millions of applications around the globe. UI Automation extends the legacy of Microsoft Active Accessibility and provides support for managed application development through the Windows Presentation Foundation (WPF). Windows 7 introduces additional accessibility features for end users, as well as UI Automation improvements in performance, extensibility, and expanded platform support. In November of 2007, Microsoft entered into an agreement with Novell to extend UI Automation support to the Linux Accessibility Toolkit. This agreement, entered into under the Microsoft Community Promise Specification initiative, extends UI Automation support to the Linux platform, increasing interoperability and further demonstrating Microsoft’s commitment to the accessibility community. Within these pages you’ll find a number of technical articles describing the state of accessible application development, as well as overviews of the latest platform support for accessibility in some of Microsoft’s most exciting products, including Windows Presentation Foundation (WPF), Silverlight, and Internet Explorer 8. For those interested in learning the basics, this issue provides a thorough overview of accessible application development, intended for those who are completely new to the space. Our mission at Microsoft is to help people everywhere meet their full potential. Our commitment to accessibility is core to who we are—it’s a firstclass service in every product that we ship, and a set of investments we continue to grow year after year, product after product. Quite simply, making the computer easier to see, hear, and use is what we’re all about.

Rob Sinclair Director of Accessibility Microsoft Corporation

6

A Letter from the Director of Accessibility at Microsoft

www.code-magazine.com

developer

magazine

Group Publisher Markus Egger Associate Publisher Rick Strahl Managing Editor Ellen Whitney Content Editors Kevin Fansler Writers In This Issue Michael Bernstein Karl Bridge Alvin Bruney LeAnne Fossmeyer JP Gonzalez-Castellan Norm Hodne Masahiko Kaneko Matthew Karr Jennifer Linn Brendan McKeon Annuska Perkins Mark Rideout Rob Sinclair Vidhya Sriram David Tseng Nazia Zaman Technical Reviewers Markus Egger Art & Layout King Laurin GmbH [email protected] Production Franz Wimmer King Laurin GmbH 39057 St. Michael/ Eppan, Italy Printing Fry Communications, Inc. 800 West Church Rd. Mechanicsburg, PA 17055 Advertising Sales Lake Quincy Media LLC 330-677-2440 [email protected] [email protected] Sales Managers Erna Egger +43 (664) 151 0861 [email protected] Tammy Ferguson 832-717-4445 ext 26 [email protected] Circulation & Distribution General Circulation: EPS Software Corp. Newsstand: Ingram Periodicals, Inc. Media Solutions Worldwide Media Subscriptions Circulation Manager Cleo Gaither 832-717-4445 ext 10 [email protected] US subscriptions are US $29.99 for one year. Subscriptions outside the US are US $44.99. Payments should be made in US dollars drawn on a US bank. American Express, MasterCard, Visa, and Discover credit cards accepted. Bill me option is available only for US subscriptions. Back issues are available. For subscription information, email [email protected] or contact customer service at 832-717-4445 ext 10. Subscribe online at www.code-magazine.com CoDe Component Developer Magazine EPS Software Corporation / Publishing Division 6605 Cypresswood Drive, Ste 300, Spring, Texas 77379 Phone: 832-717-4445 Fax: 832-717-4460

CODE COMPILERS

component

Volume 5 Issue 4

ONLINE QUICK ID 0811022

Open Accessibility

Norm Hodne [email protected], +1 (425) 704-7324 Norm Hodne is the Lead Program Manager in the Windows Experience Group responsible for the accessibility APIs, accessibility applications, and the speech commanding and dictation features shipped in Windows.

Creating a natural user interface requires designers, testers, and developers working in concert to develop the right support that makes multi-modal access to an operating system and applications possible. To assist in this work through the Accessibility Interoperability Alliance (AIA), Microsoft® released its UI Automation Specifications with a Community Promise and released testing tools as open source projects via CodePlex. Microsoft is committed to interoperable accessibility. Making Technology Accessible and Interoperable The purpose of the AIA, formed recently by a coalition of the world’s leading information and assistive technology (AT) companies, is to foster industry-wide collaboration aimed at reducing many of the barriers that people with disabilities frequently encounter when they try to access information and communication technologies, including Web sites.

number of users access technology in their daily lives. By making UI Automation freely available Microsoft hopes to increase the accessibility of many information and communication technologies and to drive cross-platform interoperability.

Microsoft UI Automation: An Innovative Accessibility Model

UI Automation, a next-generation accessibility programming To create accessible technolFast Facts model, simplifies development ogy products today, developers and reduces costs for AT develwork across multiple platforms, Novell will implement opers as well as IT application application models, and types of UI Automation for Moonlight developers who want to make hardware. To address interoperthat will provide accessibility their software compatible with ability issues arising from this capabilities for Silverlight AT products, such as screen current environment, the AIA is applications on Linux. readers for people who are using a two-part strategy: First, blind. UI Automation, which the group is working to harmois available for Windows® XP, nize current technologies to in® Windows Vista , Windows 7, and is built into the teroperate more easily, reducing costs for developers Windows Presentation Foundation (WPF), Silverand creating more usable products for customers. light™ 2 and in additional products, offers developers Second, it is working to create a unified accessibility a richer and more efficient way to convey user intermodel that will serve the entire industry. In order to face commands and behaviors to a person through support backwards compatibility and bridging techtheir AT product. nology to a full UI Automation implementation, Microsoft reUI Automation represents what the interface can leased the UI Automation SpeciUI Automation do rather than how it is presented visually, making fication with the UI Automation, allows developers to it easier for both the application developer and asMicrosoft Active Accessibility, sistive technology developers to create rich experiand IAccessibleEx specifications. create assistive technologies ences for their users, including people who are blind. across all platforms and Microsoft also released the UI AuAT products built using UI Automation will require tomation Specification to the AIA, fewer updates and fewer customizations for specific help a larger number of users an engineering working group applications, because they can intelligently interact access technology dedicated to making it easier for with user interfaces and controls that have yet to be developers to create software, invented. in their daily lives. hardware, and Web-based products that are accessible to people Linux Implementation with disabilities. The UI Automation specification describes the latest accessibility Microsoft developed UI Automation from the start framework technology developed by Microsoft, and to be compatible for implementation on platforms will help developers include advanced accessibility other than Windows. Starting in 2008, a Novell functionality into implementations designed for use team is implementing UI Automation for Linux and on any operating system. will support SUSE, Ubuntu, and Red Hat and make UI Automation allows developers to create assistive the code available to other Linux distributions. Their technologies across all platforms and help a larger implementation work has several goals:

Norm also is a representative to the INCITS V2 standards organization where he works on standards related to accessibility.

8

Open Accessibility

www.code-magazine.com

…Microsoft has agreed to grant a royalty-free license for any patents necessary to implement required portions of the UI Automation Specification… • Make Windows Forms accessible • Make Moonlight accessible • Implement UI Automation technologies Mono is an open source project sponsored by Novell which implements software to develop and run .NET client and server applications on Windows, Linux, Solaris, Mac OS X, and UNIX. Based upon the Mono Web site, Novell believes that Mono is well positioned to become the leading application development platform for Linux. Implementing UI Automation on Linux retains the existing Linux accessibility frameworks. UI Automation will integrate with the AT-SPI and ATK accessibility frameworks by implementing the client interfaces in a bridge that will act as a translation layer between the frameworks.

Open Source Tools Keeping with its commitment to help the community build better experiences for people that use assistive technology, Microsoft released two new testing tools as open source on the CodePlex shared code site. The UI Accessibility Checker (AccChecker) a Microsoft Active Accessibility® testing tool and UI Automation Verify (UIA Verify), a UI Automation testing tool, were released in March 2008. These tools are targeted at both developers and testers to provide support for the development process to create and ship products that meet the needs of assistive technology users.

UI Accessibility Checker AccChecker enables testers, without prior experience with accessibility testing, to easily discover accessibility problems with Microsoft Active Accessibility and other user interface (UI) implementations. AccChecker was born from the realization that existing Microsoft Active Accessibility tools, such as Inspect and AccExplorer, provided in-depth details on the implementation, but no information on the correctness of the accessibility implementation.

UI Automation Verify Test Automation Framework UIA Verify is the second accessibility-testing tool available on CodePlex. UIA Verify is a test automa-

www.code-magazine.com

tion framework that features the User Interface Automation Test Library (UIA Test Library) and Visual UI Automation Verify (Visual UIA Verify), the graphical user interface (GUI) tool. The framework facilitates manual and automated testing of the Microsoft UI Automation Provider implementation of a control or application. The majority of the UIA Verify functionality is provided through a dynamic link library (such as UIATestLibrary.dll) that contains the code for testing specific UI Automation functionality Microsoft is continuing and supports logging of the test results. its commitment to accessibility

by creating innovative accessibility APIs and testing tools and releasing them with a Community Promise or as open source projects.

About the Community Promise

As a member of the AIA, Microsoft has agreed to grant a royalty-free license for any Microsoft patents necessary to implement required portions of the UI Automation Specification, as the specification may be modified and eventually published by the AIA. Companies also can implement the latest version of the UI Automation Specification, which is publicly available from Microsoft. The Community Promise that accompanies the UI Automation Specification permits royalty-free access to Microsoft patent claims necessary to implement required portions of both mandatory and optional parts of the UI Automation Specification.

AIA Membership Committed to Open Accessibility Microsoft is continuing its commitment to accessibility by creating innovative accessibility APIs and testing tools and releasing them with a Community Promise or as open source projects. This open approach to accessibility technology allowed Novell to start development of UI Automation for Linux; it should also enable the development of lower-cost, interoperable solutions that ultimately make technology accessible to greater numbers of people throughout the world.

Resources Accessibility Interoperability Alliance http://www.accessinteropalliance.org/ UI Automation Community Promise http://msdn.microsoft.com/en-us/accessibility/ bb892136.aspx Novell UI Automation Accessibility Project http://www.mono-project.com/Accessibility CodePlex project hosting Web site http://codeplex.com/ Norm Hodne

The Accessibility Interoperability Alliance offers membership to IT and AT companies, as well as to nongovernmental organizations (NGOs). The AIA is committed to developing and enhancing standards and promoting the adoption of the standards to support the estimated one in four computer users that can benefit from AT applications.

AIA Steering Committee The Steering Committee for the Accessibility Interoperability Alliance includes five members. The Steering Committee includes Qualilife and GWMicro in the AT member category, IBM and Adobe in the IT member category, and Microsoft as the at-large member.

Open Accessibility

9

ONLINE QUICK ID 0811032

Accessibility 101

Jennifer Linn [email protected] Jenny is a writer in the Windows Experience group and has been working with Accessibility for 8 years. Currently her main focus is on the MSDN Accessibility Developers Center (http://msdn.microsoft.com/enus/accessibility/default.aspx) and internal accessibility guidelines for product groups. In her spare time, she enjoys walking homeless dogs at the Seattle Humane Society (http://www.seattlehumane.org) and reading about history.

We know what you’re thinking: Why should I read an article about the fundamentals of accessibility? Well, if you already know the percentage of computer users who have disabilities, can name at least ten different categories of assistive technologies, and can describe the key concepts involved in designing an accessible application, then you can probably skip to the next article. However, if you’re unsure what accessible technology is, then take a few minutes and keep reading. You’ll learn about the main concepts around accessible technology, the people they help, and things you can do to help them interact smoothly and successfully with each other. What We Mean When We Talk about Accessibility

Accessible technology is a piece of software or hardAs a developer, one of your main concerns is to enware that makes it easier for someone to see, hear, sure that you provide programmatic access to your and use a technology product, such as a computer application’s user interface elements. This enables the or a mobile device. It can be an assistive technology assistive technologies to get information about the UI (AT) product, a specially designed piece of software elements and expose that information to the assistive or hardware that accommodates technologies, which in turn relays someone’s disability (or multiple the information to the user. For Fast Facts disabilities) and enables them to example, important information 57% of computer users interact with a computer. These that an AT needs includes type, are likely or very likely to products are developed to work name, location, and current state benefit from the use of with a computer's operating sysof a UI element. They also need accessible technology. tem and software. For example, a to know when changes in the blind computer user may use asUI occur. For information about 44% of computer users sistive technology called a screen how to provide programmatic use some form of reader to navigate an applicaaccess using Microsoft’s accesaccessible technology. tion’s interface and a computer sibility frameworks, Microsoft user with ALS may use a word UI Automation and Microsoft Users seek solutions prediction program to help faActive Accessibility, see the Acto make their computers cilitate communicating thoughts cessibility Developer Center on easier to use, not for and ideas. MSDN (http://msdn.microsoft. solutions based on their com/en-us/accessibility/default. health or disability. Many companies offer hardware aspx). Making accessibility options devices, accessories, aids, and Programmatic access is just one software applications that fall easier to discover and of the key concepts of creating under the umbrella of assistive use will result in computers accessible applications. The technology. For an overview of that are easier to use, following section presents adassistive technology categories, more convenient, ditional concepts about accessee Descriptions of Assistive and more comfortable for sible design. Technology Products at the end computer users. of this article. Accessible technology can also be a feature built into a product that allows someone to adjust the settings to meet their needs. Examples of accessibility features include those that allow a user to increase font size, change font settings, or choose different colors for their computer screen display. Other examples are the option for users to receive announcements from their computer through sound notifications (a "ding" when new e-mail messages arrive), or visual notifications (a modal dialog flashes and beeps when the user tries to click away from it). So overall it is helpful to think of accessible technology as a range

10

Accessibility 101

of solutions that makes computer use more comfortable for some people and possible for others.

Who Benefits from Accessible Technology? Although most accessible technology was originally intended and designed for individuals with severe disabilities, accessible technology is widely used by computer users of all abilities today. Among adult computer users in the United States: • 1 in 4 has a visual disability. • 1 in 4 has a dexterity disability. • 1 in 5 has a hearing disability.

www.code-magazine.com

Although many accessible technologies are designed to help people with disabilities optimize their abilities, research shows that the majority of all computer users can benefit from adjusting their display, mouse, keyboard, and sound settings. Users can find accessible technologies helpful when, for example, recovering from shoulder surgery. Another example is using a mobile device when driving a car or riding a bus, in which case, voice recognition or output can be helpful. According to a two-part study commissioned by Microsoft and conducted by Forrester Research (The Market for Accessible Technology—The Wide Range of Abilities and Its Impact on Computer Use: http://www. microsoft.com/enable/research/phase1.aspx and Accessible Technology in Computing—Examining Awareness, Use, and Future Potential: http://www. microsoft.com/enable/research/phase2.aspx), the majority of computer users can benefit from using accessible technology. Figure 1 shows that 57% (74.2 million) of computer users are likely or very likely to benefit from the use of accessible technology due to having mild or severe disabilities. Specifically: • 40% (51.6 million) of computer users are likely to benefit from the use of accessible technology due to mild difficulties/impairments. • 17% (22.6 million) of computer users are very likely to benefit from the use of accessible technology due to severe difficulties/impairments. What does that mean for you? That’s a lot of people who will depend on the accessibility of your application. Accessible technology has the potential to improve computer use for a wider audience because it makes computers easier to use. A key component of encouraging the use of accessible technology is to make it easier to find and highlight the functionality and benefits rather than the impairments they seek to ameliorate.

Profile: Low Vision Olivia, a managing editor, has had low vision as a result of glaucoma. Low vision is a term commonly used to mean partial sight, or sight that isn't fully correctable with surgery, pharmaceuticals, contact lenses, or glasses. It includes moderate vision impairment, such as tunnel vision or blind spots, as well as legal blindness and almost total blindness. Sometimes it involves a lack of acuity, meaning that objects appear blurred. Other times, it involves a reduced ability to distinguish colors, see contrasts, or determine spatial relationships among objects. Olivia can see type and images on her computer screen when they are enlarged to about 1.5 inches in height. As her glaucoma advances, print will appear faded and words will be difficult to read. By using a screen enlargement program, Olivia can effectively view and interact with documents and tools that are an important part of her job. In addition, she uses high DPI for sharpness and turns on the high contrast accessibility setting with the enlarger because having the magnified screen displayed in black on white improves the readability for her. While she is currently able to use a mouse, she knows she needs to learn more about keyboard access because she will have to switch to keyboard use as her vision continues to deteriorate. Even with her high level of magnification, Olivia tends to sit extremely close to the monitor. In addition, due to the increased level of magnification, she needs to scroll horizontally and vertically many times in order to see what other customers will see all at once on the screen. She has to work to remember where items are in order to retain context, for example, where controls and text are meant to be seen together but aren’t due to her level of magnification. These factors make her prone to headache, eyestrain, and fatigue.

Annuska Perkins Annuska Perkins has called Microsoft home since 1998. She began as a program manager for MSN Sidewalk.com, an online city guide. Afterwards, she joined the Accessibility group, where her role evolved from program manager, to product planner, to User Experience expert. Check out her blog and projects on the MSDN Accessibility site: http://msdn.microsoft.com/ accessibility.

To get a more personal view of how accessible technology helps people, here is a look at some profiles of computer users today. The following profiles describe how people with common types of disabilities might use accessible technology to help them in their job and everyday life. Each profile is followed by design and development considerations for creating accessible applications.

Computer Users with Vision Disabilities Vision disabilities include low vision, color blindness, and blindness. Among adult computer users in the United States, 1 in 4 (27%) have a vision disability. There are many options for individuals with vision disabilities to modify the computer displays and appearance so it is more legible, or receive information through sound or touch. Those who are blind cannot use a computer monitor, but have the option to receive information from their computers through hearing or touch offered through screen readers and Braille displays.

www.code-magazine.com

Figure 1: This chart also shows the percentages of computer users who are likely or very likely to benefit from the use of accessible technology due to a range of mild to severe difficulties and impairments.

Accessibility 101

11

Design and Development Considerations: Low Vision To ensure that a low vision computer user’s experience is equal to that of users not requiring assistive technology, keep the following design guidelines in mind:

Accessibility Resources Whether you’re developing accessibility technology or designing an accessible Web site, you’ll need a bounty of resources. The following are tools, specifications, guidelines, and resources you can use to make computerbased content and technology accessible to all.

• Ensure that individual objects in the UI can easily be distinguished and that each object has a clear, unique, meaningful label that makes sense when read out of context. • Respect the Windows user system settings related to accessibility, such as color choices and icon sizes. • Do not specify hard-coded fonts. Instead, enable users to change the font size and face. Use default font size of 10 points or larger. • Ensure that users can set color settings on a system-wide basis and have them respected within their applications. • Check for appropriate use of color in foreground and background combinations. • Ensure programmatic access to UI elements. • Provide full keyboard access to your applications functionality via keyboard focus, access keys, and shortcut keys. • Ensure that the UI adapts well to high DPI settings. Profile: Blindness

Tools ®

accChecker is a Microsoft Active Accessibility® API test tool with additional accessibility test automation features. http:// www.codeplex.com/AccCheck UIA Verify is a UI Automation API test tool with additional accessibility test automation features. http://www.codeplex. com/UIAutomationVerify Fujitsu ColorDoctor simulates display content according to grayscale and color characteristics. http://design. fujitsu.com/en/universal/ assistance/colordoctor/ Wat-C Color Contrast Analyser tests whether foreground and background color combinations provide good color visibility. It also simulates certain visual conditions such as color blindness. http://www. paciellogroup.com/resources/ contrast-analyser.html

12

Accessibility 101

Jorge, a financial analyst, has been blind for five years, ever since a car accident. He can see a little light but no images and uses a guide dog to navigate within buildings and outside. After his car accident, Jorge went through extensive rehabilitation to learn how to use assistive technology to continue working. He uses a screen reader to relay information from his computer, along with an earphone so that he can work without disturbing co-workers. He also uses a refreshable Braille display to supplement the voice information he receives from the screen reader. He uses Braille copies of the firm’s reports and analyses when needing to work with hardcopy. Jorge also relies on the keyboard to navigate and on memorization of spatial relationships and keyboard shortcuts. Consistency and predictability within and across applications are great aids to his productivity. Design and Development Considerations: Blind Computer Users To ensure that a blind computer user’s experience is equal to that of users not requiring assistive technology, keep the following design guidelines in mind: • Provide programmatic access to the UI for assistive technologies such as screen readers and Braille displays. • Use text that is easily understood and that makes sense when read out loud with no visual association. Supply alternative text for images. • Group information logically and place the main idea of each paragraph in the first sentence.

• Ensure that your feature can be used without a mouse. Provide powerful, intuitive keyboard shortcuts. • Support prefix navigation (typing the first few characters to reach a known destination). • Ensure tab orders are logical.

Computer Users with Hearing Disabilities Hearing disabilities encompass a wide range of conditions—from slight hearing loss to deafness. People who have hearing disabilities might be able to hear some sound, but might not be able to distinguish words. Among adult computer users in the United States, 1 in 5 (21%) have a hearing disability. Profile: Deafness Tim is a network architect and has been deaf from birth. He works very effectively with his computer; in addition to using e-mail, he uses instant messaging to communicate with his colleagues and co-workers. Since he is deaf, he does not need sound information from his computer so he has turned sound off completely. To be sure that he gets information visually that would otherwise be conveyed by sound, he uses SoundSentry to provide visual warnings for system sounds, ShowSounds to display captions for speech and sounds, and Visual Notification to provide visual warnings when features are turned on or off. He has turned on the captioning features where available in products. Tim also uses a TTY (Telephone Typewriter) device (replacing a handset phone) and a text pager with the keyboard set to vibrate attached to his belt. This pager does text to voice for outgoing messages and voice to text for incoming messages. Design and Development Considerations: Deaf Computer Users To ensure that a deaf computer user’s experience is equal to that of users not requiring assistive technology, keep the following design guidelines in mind: • If your application plays video, provide a way to turn on and off captioning. Use the ShowSounds system setting as a cue to the user’s preference. • Be sure that your feature works with Visual Notification (Sound Sentry) system setting so that users receive visual cues in lieu of sound notifications. • Providing visual customizations can greatly enhance the customer experience. For example, provide ways for users to change the size, color, and flashing characteristics of sound alternatives.

Computer Users with Dexterity Disabilities Individuals with dexterity disabilities experience pain, discomfort, or complete loss of feeling in their

www.code-magazine.com

fingers, hands, wrists, or arms, making it difficult to use a standard keyboard or mouse. Among adult computer users in the United States, 1 in 4 (26%) have a dexterity difficulty. Dexterity difficulties and impairments can be caused by a wide range of common illnesses and accidents such as carpal tunnel, arthritis, stroke, cerebral palsy, Parkinson's disease, multiple sclerosis, loss of limbs or digits, spinal cord injuries, and repetitive stress injury, among others.

talking dictionary, phonetic spellchecking, word-prediction, and homophone support to address key aspects of reading and writing processes. In addition, she has turned on features in the operating system to help her focus on tasks, and she sometimes uses a headset to block out background noise.

Profile: Repetitive Stress Injury Jason is a medical researcher, studying molecular biology. In high school, Jason played tennis and has been playing regularly for the last 15 years. Unfortunately, years of playing have been hard on his joints and he recently had surgery for a rotator cuff injury. Jason is undergoing physical therapy to strengthen the muscles of his rotator cuff. He is able to conduct his work by using a speech recognition program installed on his computer. He also answers phone calls right on his computer and has a mobile phone equipped with voice command. Design and Development Considerations: Dexterity Disabilities To ensure that a computer user’s experience is equal to that of users not requiring assistive technology, keep the following design guidelines in mind: • Provide full keyboard access to your applications functionality via keyboard focus, access keys, and shortcut keys. • Enable programmatic access.

Computer Users with Learning Disabilities Learning disabilities can range from conditions such as dyslexia and attention deficit disorder to intellectual disabilities. Some individuals with learning disabilities are better able to process information when it is presented to them in a form and at a pace that is appropriate to them individually. During the learning process, many people with learning disabilities benefit from having a multisensory experience of audio speech paired with a visual representation. Reducing visual and auditory distractions can also aid the learning process for many people. Profile: Dyslexia Amelia is a fifth grade student who wants to be a photographer when she grows up. A bright and capable student, she struggles with dyslexia and admits that she used to be scared of reading books aloud in class. Fortunately, Amelia's parents and teacher recognized the dyslexia early, and the school’s assistive technology specialist helped find a software program that helps her read text in class and at home. Amelia's computer is now outfitted with a reading and writing software program. The program highlights text as it reads the text aloud and includes a

www.code-magazine.com

Design and Development Considerations: Learning Disabilities To ensure that a computer user with a learning disability has an end-user experience equal to that of users not requiring assistive technology, keep the following design guidelines in mind:

Microsoft’s Commitment to Accessibility: “Our vision is to create innovative technology that is accessible to everyone and that adapts to each person's needs. Accessible technology eliminates barriers for people with disabilities and it enables individuals to take full advantage of their capabilities.” —Bill Gates, Microsoft Corporation

• Enable users to perceive information in multiple ways, such as visually and aurally. • Avoid unnecessary animations and distracting UI elements. Provide a way to turn off animations. • Adhere to common UI visual guidelines to provide a consistent and intuitive experience. For more information, see the Windows Vista User Experience Guidelines on MSDN (http://msdn. microsoft.com/en-us/library/aa511258.aspx).

Aging Computer Users Do you have trouble seeing things on your computer screen? By the time we reach our fifties, twothirds of us have vision, hearing, or dexterity impairments that will impact our use of the computer. Some functional losses are accelerated by the onset of age-related degenerative diseases and ailments, including hypertension, osteoporosis, diabetes, and macular degeneration. Disabling conditions, including arthritis and orthopedic impairments resulting from sports, vehicle, and occupational injuries experienced earlier in life tend to manifest themselves as the body ages. Profile: Aging Population Fran is a retired librarian and spends her mornings hosting a book club and taking an aerobics class. She and her husband, Roy, bought a new computer when their kids started sharing photos and videos of their families. Both Fran and Roy have difficulty reading text on the computer screen and prefer different color schemes and settings. Roy prefers larger icons and a slower mouse cursor, while Fran prefers photos of her grandchildren on the desktop, large blue text, and a stylized mouse pointer for easier visibility. They each set up a profile with their preferred settings so they can share files while personalized settings make it easier for each to read text on the computer screen.

Microsoft® Internet Explorer® Developer Toolbar provides a variety of tools for quickly creating, understanding, and troubleshooting Web pages. http://www.microsoft. com/downloads/details. aspx?FamilyID=e59c3964672d-4511-bb3e2d5e1db91038&DisplayLang=en

ARIA Resources Web Accessibility Initiative WAI provides guidelines, resources, and strategies to make the Web accessible to people with disabilities. http://www.w3.org/WAI/ WAI-ARIA Primer provides background on the accessibility problem space solved with ARIA. http://www.w3.org/TR/wai-ariaprimer/ ARIA Specification defines the ARIA syntax. http://www.w3.org/TR/wai-aria/ WAI-ARIA Best Practices describes how to develop accessible rich Web applications using ARIA. http://www.w3.org/ TR/wai-aria-practices

Accessibility 101

13

They now use the computer for nearly every aspect of their lives including banking and filling prescriptions while traveling, managing digital photos, and communicating with family members using e-mail and video phone conferencing.

Among adult computer users in the United States: • 1 in 4 has a vision difficulty • 1 in 4 has a dexterity difficulty • 1 in 5 has a hearing difficulty

Microsoft Resources Profiles of Accessibility in Action details inspiring profiles of accessibility in action among five individuals using accessibility options and assistive technology products on their computers. http:// www.microsoft.com/enable/ profiles/default.aspx Microsoft® Accessibility Developer Center offers guidance, essential information, and tools for developing accessible applications and writing accessible code. http://msdn. microsoft.com/accessibility Microsoft Active Accessibility on MSDN Library describes Microsoft Active Accessibility in detail. http://msdn. microsoft.com/en-us/library/ ms697707(VS.85).aspx UI Automation Specification and Community Promise http://msdn.microsoft.com/ en-us/accessibility/bb892133. aspx Microsoft Accessible Technologies http://www. microsoft.com/enable/

Design and Development Considerations: The Aging Population To ensure that customers like Fran and Roy have an end-user experience equal to that of users not requiring assistive technology, keep the design guidelines for low vision, dexterity, and cognitive changes in mind.

High-level Concepts for Creating Accessible UI In addition to providing programmatic access and following the considerations described above, what are the high-level concepts to consider to ensure that your application’s UI is accessible? • Ensure that all functionality of the UI is usable by people with vision, dexterity, hearing, and cognitive limitations. • Make your application’s default UI accessible. For example, chose a color palette with contrasting colors and avoid color combinations that are not perceivable by people with colorblindness. • Make your UI flexible and customizable so that users can optimize the UI. As the user specifies their preferences through Windows settings ensure your application responds appropriately. (The user’s settings are discoverable programmatically through the Windows SystemParametersInfo() API: http://msdn.microsoft.com/ en-us/library/ms724947(VS.85).aspx.) • Make the UI easy to navigate regardless of the input device. Ensure that it can be accessed whether a customer uses a mouse, keyboard, on-screen keyboard, or speech recognition.

Descriptions of Assistive Technology Products Assistive technology products are designed to provide additional accessibility to individuals who have physical or cognitive difficulties, impairments, and disabilities. When selecting assistive technology products, it is crucial to find products that are compatible with the computer operating system and programs on the particular computer being used. More than 100 companies offer hardware devices, accessories, aids, and software applications that fall under the umbrella of assistive technology. These alternative input products include speech recognition systems, alternative keyboards, electronic pointing devices, sip-and-puff systems, wands, sticks, joy-

14

Accessibility 101

sticks, trackballs, and touch screens; and alternative output systems such as speech synthesizers, Braille embossers and displays, and screen readers. Below are descriptions of the various types of assistive technology products that are currently available on the market today. For more information, you can search the catalog of assistive technology products for products compatible with the Windows operating system on the Microsoft.com/enable site. Screen magnifiers (or screen enlargers) work like a magnifying glass for the computer by enlarging a portion of the screen, increasing legibility, and making it easier to see items on the computer. Some screen magnifiers allow a person to zoom in and out on a particular area of the screen. Screen readers are used to verbalize, or "speak," items on the screen including text, graphics, control buttons, and menus into a computerized voice that is spoken aloud. In essence, a screen reader transforms a graphic user interface (GUI) into an audio interface. Screen readers are essential for computer users who are blind. Speech recognition, or voice recognition, programs allow people to give commands and enter data using their voices rather than a mouse or keyboard. Voice recognition systems use a microphone attached to the computer, which can be used to create text documents such as letters or e-mail messages, browse the Internet, and navigate among applications and menus by voice. Refreshable Braille displays provide tactile output of information represented on the computer screen. A Braille "cell" is composed of a series of dots. The pattern of the dots and various combinations of the cells are used in place of letters. Refreshable Braille displays mechanically lift small rounded plastic or metal pins as needed to form Braille characters. The user reads the Braille letters with his or her fingers, and then, after a line is read, can refresh the display to read the next line. Braille embossers transfer computer generated text into embossed Braille output. Braille translation programs convert text scanned-in or generated via standard word processing programs into Braille, which can be printed on the embosser. On-screen keyboards provide an image of a standard or modified keyboard on the computer screen that allows the user to select keys with a mouse, touch screen, trackball, joystick, switch, or electronic pointing device. On-screen keyboards often have a scanning option that highlights individual keys that can be selected by the user. On-screen keyboards are helpful for individuals who are not able to use a standard keyboard due to dexterity or mobility difficulties. Alternative input devices allow individuals to control their computers through means other than a standard keyboard or pointing device. Examples include:

www.code-magazine.com

• Alternative keyboards feature larger- or smaller-than-standard keys or keyboards, alternative key configurations, and keyboards for use with one hand. • Electronic pointing devices are used to control the cursor on the screen without use of hands. Devices used include ultrasound, infrared beams, eye movements, nerve signals, or brain waves. • Sip-and-puff systems are activated by inhaling or exhaling. • Wands and sticks are worn on the head, held in the mouth or strapped to the chin and used to press keys on the keyboard. • Joysticks are manipulated by hand, feet, chin, etc., and used to control the cursor on screen. • Trackballs are movable balls on top of a base that can be used to move the cursor on screen. • Touch screens allow direct selection or activation of the computer by touching the screen, making it easier to select an option directly rather than through a mouse movement or keyboard. Touch screens are either built into the computer monitor or can be added onto a computer monitor. • TTY/TDD conversion modems are connected between computers and telephones to allow an individual to type a message on a computer and send it to a TTY/TDD telephone or other Baudot-equipped device. • Light signaler alerts monitor computer sounds and alert the computer user with light signals. This is useful when a computer user cannot hear computer sounds or is not directly in front of the computer screen. As an example, a light can flash alerting the user when a new email message has arrived or a computer command has completed.

hear what they are typing and also provide a spoken voice for individuals who cannot communicate orally, but can communicate their thoughts through typing. Talking and large-print word processors are software programs that use speech synthesizers to provide auditory feedback of what is typed. Large-print word processors allow the user to view everything in large text without added screen enlargement.

Conclusion Whether you know it or not, many people with disabilities probably use your applications. Understanding their needs is the first step to designing accessible applications that can be used by everyone. Jennifer Linn Annuska Perkins

Keyboard filters are typing aids such as word prediction utilities and add-on spelling checkers that reduce the required number of keystrokes. Keyboard filters enable users to quickly access the letters they need and to avoid inadvertently selecting keys they don't want. Reading tools and learning disabilities programs include software and hardware designed to make text-based materials more accessible for people who have difficulty with reading. Options can include scanning, reformatting, navigating, or speaking text out loud. These programs are helpful for those who have difficulty seeing or manipulating conventional print materials; people who are developing new literacy skills or who are learning English as a foreign language; and people who comprehend better when they hear and see text highlighted simultaneously. Text-to-Speech (TTS) or speech synthesizers receive information going to the screen in the form of letters, numbers, and punctuation marks, and then "speak" it out loud in a computerized voice. Using speech synthesizers allows computer users who are blind or who have learning difficulties to

www.code-magazine.com

Accessibility 101

15

ONLINE QUICK ID 0811042

Windows Automation API 3.0 Overview

Masahiko Kaneko [email protected] Masahiko Kaneko is a Senior Program Manager on the User Interface Platform Team in the Windows Experience Division. A program manager in accessibility at Microsoft for more than 10 years, he has been involved with several releases of the Windows Operating System as well as many other Microsoft products. Masahiko enjoys road biking, XC skiing, music, and movies in his free time but most of it is now dedicated to his first son who is turning 18 months in September.

While general accessibility requirements (such as font colors in UI rendering) are important, programmatic access to the graphical user interface (GUI) is a crucial element to improving accessibility. On the Windows® operating system, Microsoft® Active Accessibility® and User Interface (UI) Automation support this programmatic access. This article provides a quick overview of Windows Automation API 3.0 featured in Windows 7.

Windows Automation API 3.0: a Bit of History

Today, the Windows operating system offers two application programming interfaces (API) specifiMicrosoft Active Accessibility offers a single COM cations for user interface accessibility and software interface with a fixed, small set of properties. UI Autest automation. The legacy API, Microsoft Active tomation offers a richer set of properties, as well as a Accessibility, was introduced to set of extended interfaces called Windows 95 as a platform addcontrol patterns to manipulate Fast Facts on in 1996. The new API is a automation elements in ways UI Automation is designed Windows implementation of Microsoft Active Accessibility the User Interface Automation for a modern and evolving cannot. specification called UI Automauser experience. The latest tion. UI Automation was introWhile UI Automation previWindows 7 implementation duced in Windows Vista® and ously had both managed and helps applications be .NET Framework 3.0. unmanaged API for providers, compatible with existing the original release had no unaccessibility applications Although the two technologies managed interfaces for clients. while fully supporting new are different, the basic design With Windows Automation API principles are similar. Both ex3.0, you can finally write UI Auuser interface paradigms. pose the UI object model as tomation clients entirely in unUI Automation is actually a tree hierarchy rooted at the managed code. faster and more reliable desktop. Microsoft Active Acthan its predecessor, The new API also provides supcessibility represents individual Microsoft Active Accessibility. port for transitioning from MiUI elements as accessible obcrosoft Active Accessibility servjects, and UI Automation repers to UI Automation providers. The IAccessibleEx resents them as automation elements. Both refer to interface enables legacy Microsoft Active Accessibilthe accessibility tool or software automation proity servers to add support for specific UI Automation gram as the client. However, Microsoft Active Acpatterns and properties without rewriting their whole implementation. The specification also allows in-process Microsoft Active Accessibility clients to access UI Automation provider interfaces directly, rather than through UI Automation client interface.

Figure 1: Microsoft Active Accessibility uses OLEACC.dll to communicate between clients, like screen readers, and servers, such as Windows applications.

16

Windows Automation API 3.0 Overview

cessibility refers to the application or control offering the UI for accessibility as the server, while UI Automation refers to this as the provider.

The ecosystem of Windows automation technologies, now called Windows Automation API, includes classic Microsoft Active Accessibility and Windows implementations of the UI Automation specification. At Microsoft, the UI Automation specification is implemented on Windows Vista, Windows Server 2008, Windows Presentation Foundation (WPF), XPS Viewers, and many other upcoming Microsoft products. Windows 7, Windows Internet Explorer 8, and Silverlight 2.0 are joining the pack soon.

www.code-magazine.com

The Architecture: Microsoft Active Accessibility, UI Automation, and Interoperability The goal of Microsoft Active Accessibility is to expose basic information about custom controls such as control name, location on screen, and type of control, as well as state information such as visibility and enabled/disabled status. The UI is represented as a hierarchy of accessible objects; changes and actions are represented as WinEvents. The following components comprise the Microsoft Active Accessibility architecture: • Accessible Object—A logical UI element (such as a button) that is represented by an IAccessible COM interface and an integer ChildID. • WinEvents—An event system that enables servers to notify clients when an accessible object changes. • OLEACC.dll—run-time, dynamic-link library that provides the Microsoft Active Accessibility API and the accessibility system framework.

Figure 2: User Interface Automation (UI Automation) uses the UI Automation Core to communicate between clients and providers and uses proxies to communicate with legacy implementations.

For Microsoft Active Accessibility, the system component of the accessibility framework (OLEACC. dll) helps the communication between accessibility tools and applications (Figure 1). The applications (Microsoft Active Accessibility servers) provide UI accessibility information to tools (Microsoft Active Accessibility clients), which interact with the UI on behalf of users. The code boundary can be a programmatic or process boundary. The goal of UI Automation is similar but broader, as described later in this article. From an architecture point of view, UI Automation loads the UI Automation Core component into both the accessibility tools’ and applications’ processes (Figure 2). This component manages cross-process communication and provides higher level services. This core component enables bulk fetching or caching of properties, which improves the cross-process performance over Microsoft Active Accessibility implementation.

Figure 3: UIA-to-MSAA Bridge enables Microsoft Active Accessibility clients to access UI Automation providers.

Interoperability between Microsoft Active Accessibility-based and UI Automation-based Applications The UIA-to-MSAA Bridge enables Microsoft Active Accessibility clients to access UI Automation providers by converting the UI Automation object model to a Microsoft Active Accessibility object model (Figure 3). Similarly, the MSAA-to-UIA Proxy (Figure 4) translates Microsoft Active Accessibility-based server object models for UI Automation clients. Now with IAccessibleEx, you can also improve existing Microsoft Active Accessibility server implementations by adding only required UI Automation object model information. The MSAA-to-UIA Proxy takes care of incorporating the added UI Automation object model.

www.code-magazine.com

Figure 4: The MSAA-to-UIA Proxy enables UI Automation clients to access Microsoft Active Accessibility servers.

Limitations of Microsoft Active Accessibility Microsoft designed the Microsoft Active Accessibility object model about the same time as Windows 95 released. The model is based on “roles” defined a decade ago, and you cannot support new

Windows Automation API 3.0 Overview

17

UI behaviors or merge two or more roles together. There is no text object model, for example, to help assistive technologies deal with complex Web content. Another limitation involves navigating the object model. Microsoft Active Accessibility represents the UI as a hierarchy of accessible objects. Clients navigate from one accessible object to another using interfaces and methods available from the accessible object. Servers can expose the children of The ecosystem of Windows an accessible object with properties of the accessible object automation technologies, now or with the IEnumVARIANT called Windows Automation API, COM interface. Clients, however, must be able to deal with includes classic Microsoft both approaches for any server. Active Accessibility and This ambiguity means extra work for client implementers, and the Windows implementations of the complexity can contribute unUI Automation specification. foreseen problems of server implementations. Just as important is the inability to extend Microsoft Active Accessibility properties or functions without breaking or changing the IAccessible COM interface specification. The result is that you cannot expose new control behavior or properties through the object model. The object model tends to be both static and stagnant.

UI Automation offers much better performance for out-of-process client practices (400% or faster for some scenarios than Microsoft Active Accessibility running out-of-process), while adding richness and flexibility to support the latest user interface designs.

UI Automation Specification The UI Automation specification provides more flexible programmatic access to UI elements on the desktop, enabling assistive technology products such as screen readers to provide information about the UI to end users and to manipulate the UI by means other than standard input. The specification can be supported across platforms other than Microsoft Windows.

The implementation of that specification in Windows is also called UI Automation. UI Automation

is broader in scope than just an interface definition. UI Automation provides: • An object model and functions that make it easy for client applications to receive events, retrieve property values, and manipulate UI elements. • A core infrastructure for finding and fetching across process boundaries. • A set of interfaces for providers to express tree structure and some general properties. • A set of interfaces for providers to express other properties and functionality specific to the control type. To improve on Microsoft Active Accessibility, UI Automation aims to address the following goals: • Enable efficient out-of-process clients, while continuing to allow in-process access. • Expose more information about the UI in a way that allows clients to be out-of-process. • Coexist with and leverage Microsoft Active Accessibility without inheriting its limitations. • Provide an alternative to IAccessible that is simple to implement. The implementation of the UI Automation specification in Windows features COM-based interfaces and managed interfaces.

UI Automation Elements UI Automation exposes every piece of the UI to client applications as an Automation Element. Providers supply property values for each element; invoking a method on an element invokes the corresponding method on the provider. Elements are exposed as a tree structure, with the desktop as the root element. Automation elements expose common properties of the UI elements they represent. One of these properties is the control type, which describes its basic appearance and functionality (for example, a “button” or a “check box”).

UI Automation Tree The Automation Tree represents the entire UI: the root element is the current desktop and child elements are application windows. Each of these child elements

Figure 5: This UI Automation tree represents all elements on the Run dialog.

18

Windows Automation API 3.0 Overview

www.code-magazine.com

can contain elements representing menus, buttons, toolbars, and so on. These elements in turn can contain elements like list items, as shown in Figure 5. UI Automation providers for a particular control support navigation among the child elements of that control. However, providers are not concerned with navigation between these control subtrees. This is managed by the UI Automation core, using information from the default window providers. To help clients process UI information more effectively, the framework supports alternative views of the automation tree: (1) Raw View, (2) Control View, and (3) Content View. As Table 1 shows, the type of filtering determines the views, and the client defines the scope of a view.

UI Automation Properties The UI Automation specification defines two kinds of properties: (1) automation element properties and (2) control pattern properties. Automation element properties apply to most controls, providing fundamental information about the element such as its name. Examples are listed in Table 2. Control pattern properties apply to control patterns, described next. Unlike with Microsoft Active Accessibility, every UI Automation property is identified by a GUID and a programmatic name, which makes new properties easier to introduce.

UI Automation Control Patterns A control pattern describes the attributes and functionality of an automation element. For example, a simple “clickable” control like a button or hyperlink should support the Invoke control pattern to represent the “click” action. Each control pattern is a canonical representation of possible UI features and functions, as shown in Table 3. There are 22 control patterns defined to date, some of which are new in Windows 7, and the Windows Automation API can support custom control patterns. Unlike with Microsoft Active Accessibility role or state properties, one automation element can support multiple UI Automation control patterns.

From a UI Automation client’s perspective, there is no difference between UI Automation providers and Microsoft Active Accessibility servers that implement IAccessibleEx correctly.

UI Automation Control Types A control type is another automation element property that specifies a well-known control that the element represents. Currently, UI Automation defines 38 control types, including; Button, Check Box, Combo Box, Data Grid, Document, Hyperlink, Image, ToolTip, Tree, and Window. Before you can assign a control type to an element, the element needs to meet certain conditions, includ-

Automation Tree

Description

Raw View

The full tree of automation element objects for which the desktop is the root.

Control View

A subset of the raw view that closely maps to the UI structure as the user perceives it.

Content View

A subset of the control view that contains content most relevant to the user, like the values in a drop-down combo box.

UI Automation Community Promise Specifications The entire accessibility framework specification is available under Microsoft Open Specification Promise (http://www.microsoft.com/ interop/osp/default.mspx). Windows Automation API 3.0 is a Windows implementation of the specification. Find more information at the MSDN Accessibility Developer Center http://msdn.microsoft. com/accessibility/

Table 1: Types of Automation Tree views.

Property

Description

AutomationId

A string containing the UI Automation identifier (ID) for the automation element. The AutomationId property of an element is expected to be the same in any instance of the application, regardless of the local language.

BoundingRectangle

The coordinates of the rectangle that completely encloses the element. The returned rectangle is expressed in physical screen coordinates.

Name

A string for the text representation of the element. This string should always be consistent with the label text on screen. For example, the Name property must be “Browse…” for the button labeled “Browse…”.

ControlType

A ControlType of the automation element, which defines characteristics of the UI element by well known UI control primitives such as button or check box.

FrameworkId

A string for the name of the underlying UI framework. FrameworkId enables client applications to apply special cases to a particular UI framework. Examples of property values are "Win32", "WinForm", and "DirectUI".

Table 2: Examples of UI Automation Element properties.

www.code-magazine.com

Windows Automation API 3.0 Overview

19

Control

Description

Dock

Used for controls that can be docked in a container, like toolbars or tool palettes. Used for controls that can be expanded or collapsed, like the File menu. Used for controls that support grid functionality such as sizing and moving to a specified cell, without header information. The “large icon view” in Windows® Explorer is an example of a control that follows the Grid control pattern. Used for controls within grids. For example, each cell in Explorer’s “details view” could follow the GridItem pattern. Used for controls that can be invoked, such as a button. Used for controls that hosts a number of children that may be virtualized. Used for controls that can switch between multiple representations of the same set of information, data, or children. For example, a list view control where data is available in thumbnail, tile, icon, list, or detail views. Used for controls that have a range of values. For example, a spinner control containing years might have a range of 1900 to 2010, while another spinner control presenting months would have a range of 1 to 12. Used for controls that can scroll. Used for selection container controls. For example, list boxes and combo boxes. Used for controls those have a grid as well as header information. For example, Microsoft Excel worksheets. Used for edit controls and documents that expose textual information. Used for controls where the state can be toggled. For example, check boxes and checkable menu items. Used for controls that can be resized, moved, and rotated. Typical uses for the Transform control pattern are in designers, forms, graphical editors, and drawing applications. Used for a virtualized object in a container that supports the ItemContainer pattern. Used for controls that provide fundamental window-based functionality within a traditional graphical user interface.

ExpandCollapse Grid

GridItem Invoke ItemContainer MultipleView

Supported Platform UI Automation is supported on the following platforms to date: • Windows XP

RangeValue

Scroll Selection

• Windows Vista • Windows Server 2003 • Windows Server 2008 • Windows 7

Table Text Toggle Transform

VirtualizedItem Window

Table 3: Examples of UI Automation Control Patterns. ing a particular automation tree structure, property values, control patterns, and events. However, you are not limited to these. You can extend a control with custom patterns and properties as well as with the pre-defined ones. The total number of pre-defined control types is significantly lower than Microsoft Active Accessibility accRole definitions, because you can combine UI Automation control patterns to express a larger set of features while Microsoft Active Accessibility roles cannot. You can also customize the description of control type by LocalizedControlType property while keeping the baseline type as defined.

UI Automation Events UI Automation events notify applications of changes to and actions taken with automation elements. The four different types of UI Automation events, as listed in Table 4, do not necessarily mean that the visual state of the UI has changed. The UI Automation event model is independent of the WinEvent framework in Windows, although the Windows Automation API can make UI Automation events interoperable with the Microsoft Active Accessibility framework.

20

Windows Automation API 3.0 Overview

IAccessibleEx Interface The IAccessibleEx interface helps existing applications or UI libraries extend their Microsoft Active Accessibility object model to support UI Automation without rewriting everything from scratch. With IAccessibleEx, you can implement only the differences between Microsoft Active Accessibility and UI Automation object models. Because the MSAA-to-UIA Proxy translates the object models of IAccessibleEx-enabled Microsoft Active Accessibility servers as UI Automation object models, UI Automation clients don’t have to do any extra work. The IAccessibleEx interface can enable classic in-process Microsoft Active Accessibility clients to interact directly with UI Automation providers, too.

Which to Support: Microsoft Active Accessibility, UI Automation, or IAccessibleEx? While IAccessibleEx can be a cost effective way of supporting UI Automation, a couple of technical considerations should be made prior to the decision.

www.code-magazine.com

Event

Description

Property change

Raised when a property on an UI Automation element or control pattern changes. For example, if a client needs to monitor an application's check box control, it can register to listen for a property change event on the ToggleState property. When the check box control is checked or unchecked, the provider raises the event and the client can act as necessary. Raised when a change in the UI results from end user or programmatic activity. For example, clients can listen for the Invoked event when a user clicks a button when the InvokePattern is invoked programmatically.

Element action

Structure change

General event

Raised when the structure of the UI Automation tree changes. The structure changes when new UI items become visible, hidden, or removed on the desktop. Raised when actions of global interest to the client occur, such as when the focus shifts from one element to another or when a window closes.

Table 4: Types of UI Automation Events.

If you are developing a new application or control, I recommend UI Automation as it provides the best flexibility. Microsoft Active Accessibility may look simpler in the short term, but there are a lot of serious limitations to overcome, like its Windows 95-inspired object model and the inability to support new UI behaviors or merge roles. These shortcomings surface quickly when you try to introduce new controls. The UI Automation object model is based on canonical functionalities of UI features. Control developers choose from abstracted features (control patterns) that best suit their controls’ behaviors. This is supplemented with control types that offer role information to users.

When to Use IAccessibleEx Consider the following requirements before you use the IAccessibleEx interface for your application. Rule #1: The baseline Microsoft Active Accessibility Server’s accessible object hierarchy must be clean. IAccessibleEx cannot fix problems with existing accessible object hierarchies. Rule #2: Your IAccessibleEx implementation must be compliant with both Microsoft Active Accessibility and UI Automation specifications. Tools are available to validate compliance with both specifications. If either of these baseline requirements is not met, you should consider implementing UI Automation natively. You can keep legacy Microsoft Active Accessibility server implementations for backward compatibility if it is necessary. From a UI Automation client’s perspective, there is no difference between UI Automation providers and Microsoft Active Accessibility servers that implement IAccessibleEx correctly.

www.code-magazine.com

Clients’ Golden Question: Microsoft Active Accessibility, UI Automation, or Something Else? Microsoft Active Accessibility is a “chatty” architecture and is slow for clients that run out of process. To mitigate this, many accessibility tool programs choose to hook into and run in the target application process. While assistive technologies widely employ this in-process code practice, the risk and complexity are extremely high. Thus, you need an out-ofprocess solution with better performance and reliability.

The UI Automation object model is based on canonical functionalities of UI features. Control developers choose from abstracted features (control patterns) that best suit their controls’ behaviors.

UI Automation offers much better performance for out-of-process client practices (400% or faster for some scenarios than Microsoft Active Accessibility running out-of-process), while adding richness and flexibility to support the latest user interface designs.

Conclusions and Resources Windows Automation API 3.0 in Windows 7 features the best of Microsoft Active Accessibility and the UI Automation specification, while providing a cleaner migration path from one to the other. The object model is easier to use and more flexible, the automation elements reflect the evolution of modern user interfaces, and developers can define custom UI Automation control patterns, properties, and events. UI Automation Specifications are offered as crossplatform accessibility API under the Microsoft Open Specification Promise. For more information about the specification and about how Microsoft implements it, visit the MSDN Accessibility Developer Center (http://msdn.microsoft.com/accessibility/). Masahiko Kaneko

Windows Automation API 3.0 Overview

21

ONLINE QUICK ID 0811052

What’s New in Windows 7 Automation API

Nazia Zaman

Windows® 7 offers end–to-end accessibility with better performance, seamless interoperability, and improved framework design. The Windows Automation API 3.0 combines the advantages of existing accessibility implementations in Microsoft® Active Accessibility® and a modern API design in UI Automation to provide easier migration from and integration with precursor technologies and industry standards.

[email protected], Program Manager, User Interface Platform

The major improvements in this accessibility framework are:

Once you have the Automation object, you can discover the entire user interface (UI). The UI is modeled as a tree of automation elements (IUIAutomationElement objects), each element Fast Facts representing single pieces of The Windows 7 UI: a button, a window, the Automation API is now desktop, and so on. The IUIavailable unmanaged AutomationElement interface from the client has methods relevant to all conto provider sides, boasting trols, such as checking propera 400%+ improvement in ties or setting focus. Here, you performance. get the element with focus and its name:

Nazia is a PM on the User Interface Platform Team in the Windows Experience Division, and is very passionate about Accessibility.

• End-to-end unmanaged solution, including a native client API, faster and more robust UI Automation implementations, extensible provider implementations, and Win32 proxy for performance and availability. • Harmonization with industry standards such as W3C ARIA and Section 508 specifications. • New properties and control patterns. • Custom properties, events, and patterns.

She graduated from Stanford University with a Masters in Computer Science. She is interested in Human Computer Interaction, usability, and kernel programming. Check out her blog at: http://blogs.msdn. com/dreamingreality/

An Unmanaged Client API For Windows 7, UI Automation has a new COM client API for use from unmanaged and managed clients alike. Unmanaged applications can now use UI Automation without changing languages or loading the CLR, while simultaneously benefiting from all the latest features. This new API is similar to the previous managed API, but is friendlier to C++ developers. applications

Unmanaged can now use UI Automation without changing languages or loading the CLR, while simultaneously benefiting from all the latest features.

The heart of the new COM API is the IUIAutomation interface, which enables clients to get automation elements, register event handlers, create objects, and access other helper methods. To begin using UI Automation, simply include the Automation header (UIAutomation.h), and then CoCreate the Automation object: IUIAutomation * pAutomation; CoCreateInstance(__uuidof(CUIAutomation), NULL, CLSCTX_INPROC_SERVER, __uuidof(IUIAutomation), (void **)&pAutomation);

22

What’s New in Windows 7 Automation API

IUIAutomationElement *pElement; pAutomation->GetFocusedElement(&pElement); BSTR name; pElement->get_CurrentName(&name); std::wcout << L"Focused Name:" << name << L"\n";

Furthermore, elements can expose functionality specific to a particular control through control patterns. Control patterns are collections of associated properties, events, and methods, and more than one can apply to an element. Here, I use the Invoke pattern to press a button: IUIAutomationInvokePattern * pInvoke; pElement->GetCurrentPatternAs(UIA_InvokePatternId, __uuidof(IUIAutomationInvokePattern), (void **)&pInvoke); pInvoke->Invoke();

To navigate the tree of automation elements, you can use tree walkers. Here, I create an IUIAutomationTreeWalker object to navigate to the first child of an element in the Control View of the tree: IUIAutomationTreeWalker * pWalk; pAutomation->get_ControlViewWalker(&pWalk); IUIAutomationElement * pFirst; pWalk->GetFirstChildElement(pElement, &pFirst);

To address performance issues when communicating across processes, clients can fetch multiple properties and patterns at a time. Here, I create a cache request (IUIAutomationCacheRequest ob-

www.code-magazine.com

ject) from the Automation object, identify the property (Automation ID property) to prefetch, cache it for an element, and then get the cached ID: IUIAutomationCacheRequest * pCR; pAutomation->CreateCacheRequest(&pCR)); pCR->AddProperty(UIA_AutomationIdPropertyId); IUIAutomationElement * pCachedElement; pElement->BuildUpdatedCache(&pCachedElement); BSTR autoID; pCachedElement->get_CachedAutomationId(&autoID); std::wcout << L"Cached ID:" << autoID << L"\n";

With UI Automation property conditions, looking for specific values for a property is easy; you can combine conditions with AND, OR, and NOT operators to search for UI scenarios. Here, I search for a check box: IUIAutomationCondition * pCheckBoxProp; VARIANT varCheckBox; varCheckBox.vt = VT_I4; varCheckBox.lVal = UIA_CheckBoxControlTypeId; pAutomation->CreatePropertyCondition( UIA_ControlTypePropertyId, varCheckBox, &pCheckBoxProp); IUIAutomationElement * pFound; pElement->FindFirst(TreeScope_Descendants, pCheckBoxProp, &pFound);

In addition to these features, UI Automation provides custom proxy registration, events, and the Text Pattern for manipulating documents.

Proxy Factory

Proxy factory registration consists of creating proxy factory entries and inserting them into a proxy factory mapping. The mappings are processed in order as clients register proxies. In this example, I create the proxy for all HWNDs with a class name containing the word “MYCUSTOMBUTTON”: // Instantiate the proxy factory. IUIAutomationProxyFactory * pCF = new CustomFactory(); // Create an entry with the factory. IUIAutomationProxyFactoryEntry * pCEnt; pAutomation->CreateProxyFactoryEntry(pCF, &pCEnt); pCEnt ->put_ClassName(L"MYCUSTOMBUTTON"); pCEnt ->put_AllowSubstringMatch(TRUE);

Matthew Karr

// Get the mapping. IUIAutomationProxyFactoryMapping * pMapping; pAutomation->get_ProxyFactoryMapping(&pMapping);

Matthew is a Software Development Engineer on the User Interface Platform Team, Accessibility, Application, and Automation group in the Windows Experience Division.

// Insert the entry at the start of the mapping. pMapping->InsertEntry(0, pCEnt);

Optimizations in the Event Model UI Automation clients, like screen readers, track events raised in the UI by UI Automation providers and notify users of the events. To improve efficiency in Windows 7, providers can raise an event selectively, notifying only those clients that subscribe to that event. Common UI Automation events that providers should support include the following: • Changes in a property or control pattern for a UI Automation element. • End user or programmatic activity that affects the UI. • Changes to the structure of the UI Automation tree. • Shift in focus from one element to another.

Clients can customize how UI Automation sees HWND-based controls by registering customized providers with a proxy factory implementation. Custom proxies affect only the client’s own view. The provider answers CreateProvider calls with the HWND to be proxied, as well as the idObject and idChild associated with the WinEvent, if needed.

To listen to an event, create a COM object implementing the event handler interface, and then call the corresponding method on the IUIAutomation object to subcribe the handler to the event. Here, I subscribe a handler to the Focus Changed event:

Here, I verify the expected values of OBJID_CLIENT and CHILDID_SELF and create a custom proxy:

class FocusChangedHandler: public IUIAutomationFocusChangedEventHandler;

IFACEMETHODIMP CustomFactory::CreateProvider( UIA_HWND hwnd, LONG idObject, LONG idChild, IRawElementProviderSimple **ppRetVal) { *ppRetVal = NULL; if(idObject == OBJID_CLIENT && idChild == CHILDID_SELF) { // Create the custom proxy. *ppRetVal = new CustomProxy((HWND)hwnd); } return S_OK; }

FocusChangedHandler * focusHandler = new FocusChangedHandler();

www.code-magazine.com

Matthew Karr has been a developer at Microsoft for 5 years, working on Automation and Accessibility. He’s helped ship UI Automation V1 and the Magnification API with Vista, and is currently working on the COM UI Automation API for Windows 7. In his free time he enjoys juggling, snowboarding, science fiction, and video games.

// Register the event with default cache request. pAutomation->AddFocusChangedEventHandler( NULL, focusHandler);

Interoperability with Microsoft Active Accessibility-based Controls To improve interoperability, UI Automation translates between Microsoft Active Accessibility and

What’s New in Windows 7 Automation API

23

Some frameworks such as HTML, Windows Presentation Foundation (WPF), and Silverlight™ have a metadata system that can associate accessibilityrelated properties with an element. Consequently, developers can easily fix common bugs such as an incorrect accessibility name. However, Win32 does not have a similar feature, and the very basic property system for HWNDs does not apply to subitems.

Figure 1: COM diagram shows the role of IAccessibleEx in extending legacy implementations. UI Automation implementations. UI Automation clients can use the new UI Automation services to interact with earlier Microsoft Active Accessibility implementations, and Microsoft Active Accessibility clients can interact with UI Automation providers. In Windows 7, Microsoft Active Accessibility implementations can add UI Automation properties and control patterns by supporting the IRawElementProviderSimple interface to expose patterns and Clients using the properties and the IAccessibleEx UI Automation client API interface to handle ChildIds. Figure 1 shows the relationship can use its services to between the IAccessible, IAccessearch existing sibleEx, IRawElementProviderSimple, and IRangeValueProIAccessible implementations. vider interfaces.

Code that writes to the UI automation provider interfaces will still be visible to existing IAccessible clients.

IAccPropServices *pAccPropServices; CoCreateInstance(CLSID_AccPropServices, NULL, CLSCTX_SERVER, IID_IAccPropServices, (void**)&pAccPropServices); pAccPropServices->SetHwndPropStr(hwnd, OBJID_CLIENT, CHILDID_SELF, AutomationId_Property_GUID, L"Foo ID");

Custom Control Patterns, Properties, and Events With Windows 7, you can extend the platform with custom control patterns, properties, and events. Because of this support, developers of UI Automation clients and providers can introduce new accessibility specifications independent of future operating system releases.

With the IAccessibleEx interface, developers can extend Microsoft Active Accessibility implementations by adding required UI Automation object model information. The new MSAA-to-UIA proxy featured in the Windows Automation API provides a variation of a UI Automation “provider” to Microsoft Active Accessibility implementations. UI Automation clients can interact with all variations of UI Automation implementations: native UI Automation, IAccessible (Microsoft Active Accessibility), and IAccessible + IAccessibleEx.

Developers register and use custom patterns, properties, and events on both the client side and the provider side. If a provider registers for a property that the client hasn’t, the client can’t retrieve it, and if a client registers for a property that the provider hasn’t, the provider can’t respond to a request for it. Neither of these cases causes errors; one side merely remains unaware of the other’s capabilities.

New Win32 Support via OLEACC Plus IAccessibleEx

Custom Properties

In Windows 7, OLEACC proxies expose information about common controls that Microsoft Active Accessibility cannot express. The MSAA-to-UIA proxy recognizes IAccessibleEx and forwards this additional information to UI Automation. For example, the OLEACC slider proxy adds the IRangeValue pattern, exposing minimum and maximum values that Microsoft Active Accessibility cannot expose. Extending the OLEACC proxies with IAccessibleEx has the dual benefit of leveraging existing code and keeping the OLEACC proxies up to date.

24

However, with Direct Annotation, developers can mark up properties on Win32 controls with accessibility property/value information, enabling developers to fix bugs without needing a full UI Automation or Microsoft Active Accessibility implementation. Here, I set the AutomationId on a simple control with the hwnd:

What’s New in Windows 7 Automation API

Once the new property, pattern, or event is registered, using it is just like using a built-in pattern, property, or event.

Registration of properties is identical for both client and provider. In the following sample, I have a native UI Automation control and I want to add a custom string property, LongName: // The PropertyId for the LongName property. PROPERTYID longNamePropertyId; // This is the predefined property GUID, // the name of the property, and the type. UIAutomationPropertyInfo longNamePropertyInfo = { GUID_LongNameProp,

www.code-magazine.com

L"LongName", UIAutomationType_String }; // This yields the property ID for new property. pAutomationRegistrar->RegisterProperty( &longNamePropertyInfo, &longNamePropertyId);

Retrieving the property is similar to retrieving a normal property; you use the property ID that the RegisterProperty method initialized: VARIANT longNameValue; pElement->GetCurrentPropertyValue( longNamePropertyId, &longNameValue); std::wcout << longNameValue.bstrVal << L"\n";

Again, on the provider side, you call the same registration method except you add an entry in the control’s implementation of IRawElementProviderSim ple::GetPropertyValue. In this code sample, the UI Automation implementation keeps a reference to the control in _pOurControl, and the control supports a GetLongName method: GetPropertyValue(PROPERTYID propertyId, VARIANT * pRet) { pRet->vt = VT_EMPTY; if(propertyId == longNamePropertyId) { // Get the long name from the control. pRet->bstrVal = _pControl.GetLongName(); pRet->vt = VT_BSTR; } // Deal with other properties... return S_OK

Custom Control Patterns Creating a custom control pattern requires the following: • Arrays of events, properties, and methods associated with the pattern. • IIDs of the pattern’s corresponding provider interface and client interface. • Code to create a client interface object. • Code to perform marshalling for the pattern’s properties and methods.

However, with Direct Annotation, developers can mark up properties on Win32 controls with accessibility property/value information...

On the client side, the code that registers a pattern must supply a factory for creating instances of a Client Wrapper that forwards property requests and method calls to an IUIAutomationPatternInstance provided by UI Automation. The UI Automation framework then takes care of remoting and marshalling the call. On the provider side, the code that registers a pattern must supply a “pattern handler” object that performs the reverse function of the Client Wrapper. The UI Automation framework forwards the property and method requests to the pattern handler object, which in turn calls the appropriate method on the target object’s provider interface. The UI Automation framework takes care of all communication between the client and provider, both of which register corresponding control pattern interfaces. For more details please refer to the Windows 7 SDK.

}

New Properties and Control Patterns

Custom Events

Some other useful properties for automation elements have been added to UI Automation for Windows 7:

Events follow a nearly identical model to properties: you register the event ID, and then you can use the custom EventID exactly as you use a normal EventID. However, custom events cannot have arguments; they are merely notifications that the event occurred.

ControllerFor is an array of elements manipulated by the automation element. Without this property it is hard to determine the impact of an element’s operation.

Here, I register the event. Both listening and raising would then be identical to other events: // The EventId for the ThingHappened event EVENTID thingHappenedEventId; // Event Information // This is the event GUID, the name of the event. UIAutomationEventInfo thingEventInfo = { GUID_ThingHappenedEvent, L"ThingHappened" }; // This gives you the Event ID for the new event. pAutomationRegistrar->RegisterEvent( &thingEventInfo, &thingHappenedEventId);

www.code-magazine.com

DescribedBy is an array of elements that provide more information about the automation element. Instead of using the object model to discover information about the element, clients can quickly access that information in the DescribedBy property.

UI Automation as an Automated Test Tool UI Automation is specifically designed to support automated testing, as well as accessibility. UI Automation separates two models of UI interaction: one exposes information about the UI and the other collects information needed by test tools. UI Automation providers are applications, such as Microsoft Word and other third-party applications, or controls based on the Microsoft Windows operating system. UI Automation clients include automated test scripts and assistive technology applications. Both a provider and client are required for UI Automation to be useful as an automated test tool.

To improve interoperability, UI Automation translates between Microsoft Active Accessibility and UI Automation implementations.

FlowsTo is an array of elements that suggest the reading order after the current automation element. FlowsTo is used when automation elements are not exposed or structured in the reading order users perceive.

What’s New in Windows 7 Automation API

25

IsDataValidForForm identifies whether data is valid in a form. ProviderDescription identifies source information for the automation element’s UI Automation provider, including proxy information.

Control Patterns for Virtualized Child Objects When a control has too many children to load at once, a common solution is to treat the excess children as virtualized controls. This creates problems for the UI Automation tree map because there are only a handful of real controls and the virtualized controls simply don’t exist offers in the UI Automation tree.

Windows 7 further extensibility through the registration of custom control patterns, properties, and events.

To manage this, UI Automation offers two control patterns. The ItemContainer pattern lets a user search a container of virtualized controls for specific properties. This gives the client a reference to a virtualized control, but the user can’t do anything with it. The VirtualizedItem pattern enables the client to force the item to exist, either by realizing it internally, or by having it scroll on screen. In this client-side code example, I search for a specifically named item in a virtualized list: // Get the ItemContainer pattern. IUIAutomationItemContainerPattern * pContainer; pElement->GetCurrentPatternAs(

W3C ARIA Role

UIA_ItemContainerPatternId, __uuidof(IUIAutomationItemContainerPattern), (void**)&pContainer)); // Search the container for the property. VARIANT varNameStr; varNameStr.vt = VT_BSTR; varNameStr.bstrVal = SysAllocString(name); IUIAutomationElement * pFound; pContainer->FindItemByProperty(NULL, UIA_NamePropertyId, varNameStr, &pFound); // Realize the virtual element. IUIAutomationVirtualizedItemPattern * pVirt; pFoundElement->GetCurrentPatternAs( UIA_VirtualizedItemPatternId, __uuidof(IUIAutomationVirtualizedItemPattern), (void**)&pVirt)); pVirtualizedItem->Realize();

New UI Automation Properties for Accessible Rich Internet Applications (ARIA) Every day, Web sites are increasing their utility with dynamic content and advanced UI controls by using technologies like Asynchronous JavaScript and XML (AJAX), HTML, and JavaScript. However, assistive technologies are frequently unable to interact with these complex controls or expose dynamic content to users. Accessible Rich Internet Applications (ARIA) is a W3C technical specification for developing Web content and applications so that they are accessible to people with disabilities.

MSAA Role

UIA Control Type

UIA AriaRole Property

button

ROLE_SYSTEM_PUSHBUTTON

button

button

checkbox

ROLE_SYSTEM_CHECKBUTTON

Checkbox

checkbox

combobox

ROLE_SYSTEM_COMBOBOX

Combobox

combobox

grid

ROLE_SYSTEM_TABLE

DataGrid

grid

gridcell

ROLE_SYSTEM_CELL

DataItem

gridcell

group

ROLE_SYSTEM_GROUPING

Grouping

group

img

ROLE_SYSTEM_GRAPHIC

Image

img

link

ROLE_SYSTEM_LINK

HyperLink

link

list

ROLE_SYSTEM_LIST

List

list

menu

ROLE_SYSTEM_MENUPOPUP

Menu

menu

presentation

ROLE_SYSTEM_PANE

Pane

presentation

progressbar

ROLE_SYSTEM_PROGRESSBAR

ProgressBar

progressbar

radio

ROLE_SYSTEM_RADIOBUTTON

RadioButton

radio

slider

ROLE_SYSTEM_SLIDER

Slider

slider

tooltip

ROLE_SYSTEM_TOOLTIP

Tooltip

tooltip

tree

ROLE_SYSTEM_OUTLINE

Tree

tree

treegrid

ROLE_SYSTEM_TABLE

DataGrid

treegrid

Table 1: W3C ARIA Roles can be mapped to Microsoft Active Accessibility roles and UI Automation control types and AriaRole properties.

26

What’s New in Windows 7 Automation API

www.code-magazine.com

W3C ARIA States and Properties

Microsoft Active Accessibility

UI Automation Control Patterns and Properties

UI Automation AriaProperties Property

checked

STATE_SYSTEM_CHECKED

Toggle Pattern, checked

checked

controls

n/a

ControllerFor

n/a

describedby

n/a

DescribedBy

n/a

disabled

STATE_SYSTEM_UNAVAILABLE

IsEnabled False

disabled

flowto

n/a

FlowsTo

n/a

invalid

n/a

IsDataInvalidForForm

invalid

labelledby

n/a

LabeledBy

n/a

live

n/a

n/a

live

multiselectable

STATE_SYSTEM_EXTSELECTABLE

CanSelectMultiple

multiselectable

readonly

STATE_SYSTEM_READONLY

IsReadOnly

readonly

required

STATE_REQUIRED

IsRequiredForForm

required

secret

STATE_SYSTEM_PROTECTED

IsPassword

secret

valuemax

n/a

Maximum Property in RangeValue Pattern

valuemax

valuemin

n/a

Minimum Property in RangeValue Pattern

valuemin

valuenow

IAccessible::get_accValue

Value Property in RangeValue Pattern

valuenow

Table 2: W3C ARIA States and Properties can be mapped to Microsoft Active Accessibilty properties and UI Automation control patterns and AriaProperties properties. To support the ARIA specification, the UI Automation specification enables developers to associate UI Automation AriaRole and AriaProperties attributes with W3C ARIA Roles, States, or Properties. This helps user applications such as Internet Explorer support the ARIA object model in the context of UI Automation while keeping a baseline accessibility object model. Some parts of the ARIA specification can be mapped to the desktop-oriented Microsoft Active Accessibility object model; however, much of the specification can only be applied to rich Internet applications. Table 1 lists some examples of mappings from W3C ARIA Roles to Microsoft Active Accessibility Roles and UI Automation Control Types. For example, the ARIA Role checkbox is supported in Microsoft Active Accessibility by the role ROLE_SYSTEM_CHECKBUTTON and in UI Automation by the combination of control type Checkbox and AriaRole checkbox. The ARIA state checked is supported in Microsoft Active Accessibility by the state STATE_SYSTEM_ CHECKED and in UI Automation by the control pattern Toggle Pattern and the AriaProperties property checked. ARIA States and Properties are supported by the UI Automation AriaProperties property with the following exceptions: ARIA properties that take object references (like the describedby property), and ARIA properties already supported by the accessibility object model. Table 2 lists examples of mappings from W3C ARIA States and Properties

www.code-magazine.com

to various properties and functions of Microsoft Active Accessibility and UI Automation. UI Automation also offers a simple text object model with the TextPattern pattern, which supports embedding objects in a document object. This enables user agents and client applications to treat Web content either as an HTML document or as a traditional desktop UI depending on the end-user scenarios. With these features, UI Automation enables both the support of and extension of the W3C ARIA specification without a dependency on a specific application or browser.

Conclusion With application user interfaces growing more and more complex, getting accessibility right is a challenge for developers. Programmatic access to the UI is critical in the development of assistive technologies like screen readers and magnifiers. To address this, the Windows 7 Automation API aims to provide a complete end-to-end, flexible, extensible, and consistent framework with improved design and performance. For further details, please refer to the Windows 7 SDK or the MSDN Accessibility Developer Center (http://msdn.microsoft.com/accessibility/). Nazia Zaman Matthew Karr

What’s New in Windows 7 Automation API

27

ONLINE QUICK ID 0811062

Creating Accessibility-aware Silverlight 2 Content If you haven't heard, accessibility is one of the most important aspects of a Web site experience. By using the accessibility features in Silverlight™ 2, you can provide the best experience for all users. Building a rich Internet experience can be a daunting task when you have to balance a cool visual design with usability.

Mark Rideout [email protected] (425)-703-1688 Mark Rideout is a Program Manager on the Silverlight Team at Microsoft. Mark is responsible for features around text and text input in addition to input and accessibility features. Mark has worked on multiple Microsoft UI frameworks such as Windows Forms and has been working with the .NET Framework since 1.0. Mark started at Microsoft in 1997, and prior to that, Mark worked as a consultant where he designed and developed business applications. Mark enjoys everything Lego, pinball, and Disneyland. He enjoys teaching his 2-year-old daughter about everything he likes.

28

sability goes beyond the initial determination that a user interface (UI) "makes sense" as the UI has to be accessible to everyone. Silverlight 2 enables building a cool and accessible experience, whether you are building an entire Silverlight application or just a few Silverlight controls. This article provides you with guidelines for taking advantage of these features and building an accessible Silverlight application.

U

elements into two types: simple visuals that usually derive directly from the FrameworkElement class, or functional controls that derive from the Control class. A TextBlock, Image, or MediaElement are examples of simple visuals while a Button, TextBox, or ListBox are examples of functional controls.

Along with the visual tree of elements, your Silverlight 2 application has a corresponding accessibility tree. UI Automation presents three accessibility If you attempted to add accessibility to your Silviews of the visual tree: a control view, a content verlight 1 content, you probably found that there view, and a raw view. Generally, all elements that wasn't much support except for derive from Control show up something analogous to "alt" in the control view while eleFast Facts text for screen readers. Silverments that derive directly from light 2 on the other hand proFrameworkElement show up Accessibility is an vides necessary features that in the content view. TextBlock ever-increasing requirement you can build on. can appear in either the conas you develop applications. tent view when used inside a Silverlight 2 provides In general, when you build your control or data template or the Web application with accessibilcontrol otherwise. If you crethe tools you need to make ity in mind, there are a general ate custom Silverlight controls, your application accessible set of areas to think about: you can identify whether it’s a but you need to understand control, content, or both (or how best to do it. • Screen reader support. none). • Keyboard navigation. • High contrast presentation. There are different ways to think of elements being visible or invisible. Silverlight 2 reports elements whose Visibility property is equal to Visibility.ColEnsuring Your Silverlight 2 Content lapsed as hidden, but reports elements that are in the tree and "hidden" from view as being visible elSupports Screen Readers ements. This distinction is important because you Silverlight 2 provides information to screen readers might style your app to have different UI "modes" about your application via the new UI Automation by just z- ordering the elements. If you have reaframework. At the application level, you work with sons for not setting the Visibility property, then you UI Automation using the AutomationProperties should at least disable controls that are "hidden" class and its corresponding properties. When creatfrom view to ensure that screen readers don't ating custom controls or extending controls, you work tempt to interact with them. with AutomationPeer and the UI Automation provider interfaces. This article looks at some imporXAML Tag Your Elements tant UI Automation concepts you should be aware of as you build your Silverlight 2 application. As you develop your Silverlight application, you sometimes need to provide additional accessibility Silverlight Accessibility Tree information to ensure your content is readable. For example, screen readers won't know anything about Your Silverlight 2 application is comprised of many your Image element except that it is an image. As visual elements from Borders to Control-based elewith HTML, you need to provide "alternative" textments. For the most part, you can split the visual based information. With Silverlight, you provide al-

Creating Accessibility-aware Silverlight 2 Content

www.code-magazine.com

ternative accessibility information by using the AutomationProperties static class. There are cases when you won't have to add additional markup via AutomationProperties. For example, controls that have text in them (like Buttons) already provide default information. In other cases, you do need to provide additional data.

tems, Microsoft® Active Accessibility®-based screen readers can read Silverlight 2 content via the UI Automation UIA-to-MSAA Bridge. Because the accessibility technologies are somewhat different, not all data transfers into Microsoft Active Accessibility properties and some concepts are simplified.



For more information on the UIA-to-MSAA bridge, see the Windows Automation API 3.0 Overview article by Masahiko Kaneko in this issue. In addition, you can search MSDN for Active Accessibility Bridge to UI Automation for specific details on the limitations.

In code, you use the appropriate attached property set/get method:

Ensure Keyboard Interactivity

Using AutomationProperties is very easy both in your XAML markup and in code. Using the Image element as an example, you set the AutomationProperties.Name attribute with XAML markup like this:

AutomationProperties.SetLabeledBy(this.nameInput, this.nameLabel);

For best practices on providing good alternative text, see the MSDN technical article "Creating Text Equivalents for Images" (http://msdn.microsoft. com/en-us/library/ms971334.aspx). Sometimes you might have a TextBox, Image, or other control labeled with text. For example, a TextBox for the user's name might have text to the left specifying "Name:". You can use the TextBlock element to provide that static text label, and the AutomationProperties.LabeledBy property to tell that the TextBlock is labeling the TextBox. Because Silverlight 2 does not support element Binding syntax (using the ElementName syntax) to set the LabeledBy property in XAML, you have to resort to some code, like this XAML: <StackPanel Orientation="Horizontal">

Then you set the LabeledBy property like so: AutomationProperties.SetLabeledBy(this.nameInput, this.nameLabel);

To properly support screen readers, you need to ensure that all your non-text controls that show up in the accessibility tree provide at least the AutomationProperties.Name value. I recommend that you spend the time to provide other values such as HelpText. There are some great accessibility verification tools recently released by Microsoft that work great with UI Automation. Check it out by searching for Microsoft Accessibility Labs on MSDN.

UI Automation and Microsoft Active Accessibility Screen Readers—Bridging the Gap As I've mentioned earlier, Silverlight 2 relies on the operating system's UI Automation support. This does not imply that only UI Automation-based screen readers are supported. On supported operating sys-

www.code-magazine.com

Along with the visual tree of elements, your Silverlight 2 application has a corresponding accessibility tree.

Creating an accessible Silverlight application means creating a keyboard-accessible application. Try using your application with only a keyboard to identify where functionality is unavailable to keyboardonly users. Sometimes you'll find that you require mouse actions to use certain features. Other times you'll find that you cannot tab to an element. By thinking about keyboard accessibility during application design and development, you will avoid these types of issues. For more information on keyboard best practices, see Guidelines for Keyboard User Interface Design in MSDN (http://msdn.microsoft.com/en-us/library/ms971323.aspx).

IsTabStop and Tab Order

AutomationPeer is Based upon the WPF Design The design for the AutomationPeer class and derived classes comes from the Windows Presentation Foundation set of classes. This provides you with an opportunity to write your AutomationPeer accessibility code once and use it both in WPF and Silverlight. Sharing designs also helps you as you learn about adding accessibility since there are already materials out about WPF and accessibility.

With the new keyboard and focus model in Silverlight 2, you can create a keyboard-friendly experience. You specify what controls can receive focus via the IsTabStop property. Most Silverlight controls, such as Button, already set the IsTabStop property to true. After you design the UI, take the time to set the TabIndex property to control the tab order. One thing to note is that only Control-based elements can receive focus.

Image and TextBlock (Non-Control-based Elements) If you don't think about keyboard accessibility, you might add interactivity to an Image, Path, or MediaElement with mouse input, but visual elements that don’t derive from the Control class cannot receive focus. When you enable mouse actions on these types of elements, you may also want to enable the functionality with a keyboard command or with a control in the UI. Alternatively, if you want to enable focus on an Image, you can create a custom UserControl element that contains the Image and add your keyboard logic there. Do note that you will also need to provide a focus visual indication, as described later in this article.

Creating Accessibility-aware Silverlight 2 Content

29

Custom Keyboard Actions The trick in providing custom keyboard actions with a hotkey combination is to make sure your combination doesn't conflict with the browser keyboard actions because the browser will Silverlight, always win.

With you provide alternative accessibility information by using the AutomationProperties static class and attached properties.

A good keyboard combination that I've used is combining a number key with Shift and Control keys. For example, to provide keyboard hotkeys for a particular view in my UI, I listen (hook up event handlers) for the Shift, Control, and number keys associated with the view. I add a KeyDown event handler to the root visual of my application since keyboard events bubble when not handled: if (e.Key == Key.D1 && ((Keyboard.Modifiers & (ModifierKeys.Control | ModifierKeys.Shift)) == (ModifierKeys.Control | ModifierKeys.Shift))) { // Perform hotkey action here }

Regardless of what custom keyboard actions you add you need to let users know that they exist. You can do this by setting the AutomationProperties.AcceleratorKey property. In addition, you should add a tooltip or UI that describes the hotkey combination.

With Great Power Comes Great Responsibility

To help you understand some of the ways people might use your application, think about what your application would look like if you printed it on a grayscale printer. Think about what it would be like to use it with the keyboard only. Think about whether your application is usable with the sound turned off. Finally, think about whether you could easily explain how to use your app over the phone. All of these will help your design be more usable for more people in more situations.

Support Changing Font Sizes Silverlight 2 doesn't automatically change size based upon the computer's DPI settings nor does the content automatically change size based upon the browser's zoom or text size settings. There are three ways that you can scale your content to ensure your application can provide a large font experience: based upon the Web browser's zoom setting, based upon a user-selectable UI, and based upon a user-selectable font size UI. The last option only focuses on scaling the text size vs. scaling the whole UI. Modern Web browsers have support for whole page zooming. When users zoom, the Silverlight browser plug-in receives a resize event. Silverlight provides a public Resized event that, with a little code in your application’s Loaded event, you can use to dynamically resize your Silverlight content: App.Current.Host.Content.Resized += new EventHandler(Content_Resized);

In the XAML, just put a ScaleTransform that you will use when the Silverlight plug-in resizes: <UserControl.RenderTransform> <ScaleTransform x:Name="zoomTransform" ScaleX="1" ScaleY="1"/>

As you develop a great UI with complex gradients and cool graphics, you need to ensure you have a good experience with high contrast settings. I won't go so far as to say you shouldn’t make your UI cool; just put in the extra effort to make an accessible experience. Here are a few things to think about: • Design with accessibility in mind. • Support different text sizes or browser zoom. • Honor high contrast settings.

Lastly, apply the scale as a factor of your application’s size: void Content_Resized(object sender, EventArgs e) {

Design with Accessibility in Mind

System.Windows.Interop.Content content = App.Current.Host.Content;

Using XAML and styling your application is great, but you can create some ugly UI that doesn't present well to users who need higher contrast for readability. One important thing you can do during your design is to convey information with visual cues other than color. For example, if you highlight particular list items with a red font, you could also bold them, so the user has a non-color cue to distinguish highlighted items. You can optionally add some sound as the user moves over list items to note that the list item is "active".

30

Creating Accessibility-aware Silverlight 2 Content

double scalePlayerX = (content.ActualWidth / this.Width); double scalePlayerY = (content.ActualHeight / this.Height); this.zoomTransform.ScaleX = scalePlayerX; this.zoomTransform.ScaleY = scalePlayerY; }

A second approach is based upon the same code that you use for scaling based upon the Web browser's zoom except you provide your own UI for scaling.

www.code-magazine.com

Migration Headache?

For immediate relief, visit www.VFPConversion.com today!

VFP Conversion is a migration services brand of EPS Software Corp., providing expert upgrading of VFP applications to the latest technologies available from Microsoft. To learn more about how VFP Conversion can assist your enterprise, call toll free: 1 (866) 529-3682. [email protected]

www.VFPConversion.com

The last approach is to provide an option to increase the font size. Silverlight 2 does not recognize the browser's text size settings, so you need to provide the user with a custom UI (as simple as a set of buttons for selecting text size). Adjusting the font size can be a great approach if you have a lot of text and the overall UI doesn't need scaling. Since adjusting the font size causes the text to take up more space, you need to use layout elements such as Grid and StackPanel instead of hard coding widths/heights. Silverlight elements inherit the FontSize property throughout, so setting it at the root of your application causes the value to propagate.

Bring Contrast to Your UI Supporting high contrast is an important part to making your application accessible. Silverlight 2 provides the SystemParameters.HighContrast value to indicate when you should render in a high contrast mode. A high contrast mode can be as simple as omitting non-important images, gradients, or patterns behind text to more complex styling that makes your application design a bit simpler. In addition, a high contrast mode Creating an accessible should remove or reduce transiSilverlight application means tion animations, including the use of flashing. creating a keyboard-accessible

application.

Best Accessible Silverlight™ 2 Experience Comes from a UI Automation-based Screen Reader It should make sense that you will get the best screen reader experience with Silverlight when the screen reader uses UI Automation. The technology offers much more information than predecessors. UI Automation-based screen readers can only read the rich UI Automation concepts and additional information provided.

You can take advantage of Silverlight's styling system to provide additional styles for use in high contrast scenarios. Silverlight 2 does not ship with high contrast styles for built-in controls, so you will need to create this as you style your application. Expression Blend provides support for creating custom styles for your controls. I recommend creating two styles that you can switch between based upon the SystemParameters.HighContrast value. There is one catch: you can only apply a Silverlight control style once, and it has to be before or during the control's constructor. The more holistic approach is defining custom control styles during application startup, but you can apply a custom style in your control's constructor as well: public class MyButton : Button { public MyButton() { if (SystemParameters.HighContrast) { this.Style = (Style)Application. Current.Resources["btnContrast"]; } } } }

For an example of a Silverlight control "theme" framework, see Nikhil Kothari's blog for details (http://www.nikhilk.net/Silverlight-Themes.aspx).

32

Creating Accessibility-aware Silverlight 2 Content

Custom Controls/UI So far, I've addressed application-level accessibility, but what’s good for the application developer is good for the control developer. There are a few more concepts that you need to understand first though.

Derive from Existing Controls The Silverlight team has spent a lot of time making their controls accessible; you should try to derive from the closest Silverlight control. In addition, you need to extend the control’s AutomationPeer, which encapsulates accessibility functionality. If you can't find an AutomationPeer that matches your needs, then you should at least derive from FrameworkElementAutomationPeer.

Implement AutomationPeer and Associated Provider Interface The AutomationPeer class exposes UI Automation by rolling up most of the non-pattern related UI Automation properties exposed by the AutomationElementIdentifiers class. Silverlight primarily chooses to focus on AutomationPeer concepts where it can. The best way to describe this is to say that the AutomationPeer class provides the necessary default UI Automation property values, and you identify a control’s accessibility support via functions rather than a role. UI Automation identifies control functionality by a set of interfaces called control patterns. For example, to identify that you can "invoke" a control, you implement the IInvokeProvider interface in your control's AutomationPeer. To add a control pattern, you create a new AutomationPeer class and override its GetPattern method to return the interface corresponding to the pattern requested. Now that you have a custom AutomationPeer class, you need to make your custom control return an instance of it. Override the OnCreateAutomationPeer method on your control and return a new instance of your custom AutomationPeer. Listing 2 is an example of adding the IToggleProvider to a custom AutomationPeer created for a light switch control. The important parts of the custom control are in Listing 1. To simplify technology and focus on the purpose of Silverlight, Silverlight supports only the APIs for creating provider-side UI Automation information. Silverlight does not provide the client-side UI Automation APIs.

Change is Good, But Notify Others Because accessibility properties change often, screen readers rely on change notification to be efficient. Change notification covers such things as knowing when focus changes, knowing when a control be-

www.code-magazine.com

comes disabled, and knowing when control pattern properties change.

for identifying and notifying others about changes. See Listing 1 and Listing 2 for an example of raising property changes.

Provide Focus Indication

… when you design, do it with accessibility in mind.

Silverlight provides two methods to raise a change notification: You use the AutomationPeer. RaisePropertyChangedEvent method to signal property changes, and use the AutomationPeer. RaiseAutomationEvent method to signal other types of automation changes. Luckily, Silverlight already tells screen readers about most changes for you. Silverlight provides notifications to UI Automation clients for the AutomationPeer IsEnabled, IsOffscreen, Name, and ItemStatus values. In addition, Silverlight provides structure change notifications that tell you when items are added and removed from the visual tree. By using or deriving from a Silverlight control, you also take advantage of control-specific notifications, such as when a CheckBox is checked. If you create your own control with its own AutomationPeer functionality, then you are responsible

Visually, your control can do most anything. When people talk about accessibility support, one important visual is focus. In Silverlight 2, the control template and styling model is very rich, but what doesn't exist is a common definition for visually identifying focus. If your control derives from a common Silverlight control, you should use the FocusVisualElement identification defined for the control and just restyle it as necessary. Using Silverlight's Visual State Manager (VSM), you can provide a custom control template that defines a new focus element. If your control is totally unique, you should create your custom focus visual and add code in the GotFocus and LostFocus event handlers to show and hide the focus indicator. You should also create a control template and provide a named focus element, similar to what Silverlight does.

High Contrast Lastly, instead of putting the onus on the application developers using your custom control, you

Listing 1: Example control raising UI Automation change notifications. public { // // //

public void Toggle() { changeState(!this._stateOn); }

class LightSwitch : Control Not a complete Silverlight custom control but enough to demonstrate providing accessibility support via AutomationPeer and UI Automation

protected override AutomationPeer OnCreateAutomationPeer() { return new LightSwitchAutomationPeer(this); }

private bool _stateOn; public LightSwitch(){} public bool State { get { return this._stateOn; }

public void changeState(bool newState) { // notify accessibility that the state changed LightSwitchAutomationPeer ap = FrameworkElementAutomationPeer.FromElement(this) as LightSwitchAutomationPeer;

set {

if (ap != null) { ap.RaiseStateChangedEvent(this._stateOn, newState); }

changeState(value); } } protected override void OnMouseLeftButtonDown(MouseButtonEventArgs e) { e.Handled = true; base.OnMouseLeftButtonDown(e); this.Toggle(); }

// flip light switch state this._stateOn = !this._stateOn; // update visuals this.updateVisuals(this._stateOn); } }

www.code-magazine.com

Creating Accessibility-aware Silverlight 2 Content

33

Listing 2: Custom AutomationPeer with Togglefor example control. protected override AutomationControlType GetAutomationControlTypeCore() { // Button is the closest match return AutomationControlType.Button; }

public class LightSwitchAutomationPeer : FrameworkElementAutomationPeer, IToggleProvider { public LightSwitchAutomationPeer(LightSwitch light) : base(light) { }

protected override string GetClassNameCore() { return "LightSwitch"; }

// Helper function for raising notification internal void RaiseStateChangedEvent(bool oldValue, bool newValue) { if (oldValue != newValue && AutomationPeer.ListenerExists( AutomationEvents.PropertyChanged)) { base.RaisePropertyChangedEvent( TogglePatternIdentifiers.ToggleStateProperty, oldValue == true ? ToggleState.On : ToggleState.Off, newValue == true ? ToggleState.On : ToggleState.Off); } }

#region IToggleProvider Members public void Toggle() { if (!base.IsEnabled()) { // Don't toggle if the control is not enabled throw new ElementNotEnabledException(); } ((LightSwitch)this.Owner).Toggle(); }

// Let UI Automation know that the pattern is supported public override object GetPattern( PatternInterface patternInterface) { if (patternInterface == PatternInterface.Toggle) { return this; } return base.GetPattern(patternInterface); }

public ToggleState ToggleState { get { ((LightSwitch)this.Owner).State == true ? ToggleState.On : ToggleState.Off; } } #endregion }

should add the logic necessary to customize the controls visual style based upon the high contrast option mentioned earlier in this article.

OS Limitations For Silverlight 2, the operating system and, to some extent, the browser dictate the level of accessibility that Silverlight can provide. Silverlight is a Web browser plug-in based upon either Mozilla plug-in or Microsoft ActiveX technology, depending upon the browser. The Web browser is responsible for providing operating system interaction and notifications with the plug-ins contained on a Web page. In some cases, a plug-in can work directly with the operating system and bypass the browser. Silverlight can provide accessibility information when used inside of browsers running on operating systems that support the UI Automation specification. Currently, Microsoft Windows XP and later support UI Automation; Novell has made a commitment to supporting the UI Automation interfaces on Linux. Currently, Apple’s OS X op-

34

Creating Accessibility-aware Silverlight 2 Content

erating system does not support UI Automation, and the Mozilla plug-in technology does not support interaction with the OS X accessibility framework.

Conclusion: Silverlight’s Accessible Future With Silverlight 2, you can control high contrast styles, keyboard navigation, and screen reader support. You can even take advantage of UI Automation wherever it’s implemented. Silverlight’s new features can help you build Web applications and controls that support accessibility, so all users can gain enjoyment and value from your work. And remember, when you design, do it with accessibility in mind. Mark Rideout

www.code-magazine.com

ONLINE QUICK ID 0811072

Making Custom Controls Accessible

LeAnne Fossmeyer LeAnne is a Content Publishing Lead in the Windows Experience organization at Microsoft.

While custom controls are introduced every day, not all of them are easily accessible. This article provides a quick summary of Microsoft® technologies that help make Win32-based custom controls programmatically accessible. Techniques range from implementing UI Automation, to creating or overriding properties with Dynamic Annotation, to using the new IAccessibleEx interface to close the gap between UI Automation and Microsoft Active Accessibility®. he Windows® operating system offers several technologies to help make controls accessible: Microsoft Active Accessibility, UI frameworks, and UI Automation. As Table 1 shows, the most effective technology depends on how developers introduce their new controls.

T

For controls and frameworks written from scratch, UI Automation is recommended today. While Microsoft Active Accessibility is handy for relatively simple controls, the technology doesn’t support the complexity of modern user interfaces (UI).

Quick Tips: “Do I Really Need that Control?” Before investing time in designing a new control or customizing a standard control, you should ask yourself: “Do I really need that control?”

Fast Facts To meet the accessibility requirements, a small change in standard controls can cause more code changes than the change itself. Never underestimate the implication of a small modification.

For controls based on common controls (such as COMCTL32. DLL or USER32.DLL), Dynamic Annotation enables modest customization. However, it may not be sufficient for significant customization or for sub-classed controls. For controls based on solid Microsoft Active Accessibility implementations, developers can use the IAccessibleEx Interface specification to enhance accessibility with new UI Automation interfaces. This article introduces the basics of using UI Automation, Dynamic Annotation, and the IAccessibleEx interface for developing controls.

If you’re lucky, you may find a standard control that does the job. Many modern UI libraries support accessibility and customization.

If you customize a standard control, try not to overload it with features. Unnecessary complexity can result in expensive accessibility support cost, not to mention reduced usability. Never underestimate the implication of a small modification. If you have to introduce major customizations, consider creating a control from scratch

If you have to introduce a new control, try to follow standard practices to support UI accessibility. For example, never assume that all users have access to pointing devices, and try to use system or desktop theme colors to avoid accessibility problems with custom colors. While the focus of this article is on

Type of Custom Control

Type of Customization

Suggested Approach

New control

Write new UI code from scratch

Implement UI Automation or Microsoft Active Accessibility for relatively simple controls such as a button.

COMCTRL/USER controls with minor modification

Make small changes that have no functional impact (background color)

Use Dynamic Annotation to modify (or add) Microsoft Active Accessibility/UI Automation properties.

Microsoft Active Accessibilitybased controls

Add new functionality not compatible with Microsoft Active Accessibility design

Implement the IAccessibleEx interface and address only the gap from Microsoft Active Accessibility to UI Automation.

Table 1: Suggested solutions by type of custom controls.

36

Making Custom Controls Accessible

www.code-magazine.com

programmatic access to the user interface, these fundamental accessibility criteria are very important. Many accessibility practices are available in the form of industry standards (such as Section 508 of the Federal Rehabilitation Act). Some useful resources are introduced at the Microsoft Accessibility Developer Center (http://msdn.microsoft.com/ en-us/accessibility/default.aspx).

Part I: Implementing UI Automation This article talks about implementing UI Automation for new controls in a few simplified steps. Designing a UI Automation object model for a new user interface is straightforward if you follow a few basic principles: 1. Define user scenarios clearly. Actual user scenarios are helpful. During the design process, you shouldn’t rely on a single operation device such as mice, custom color, or screen size. 2. Define the UI functionality as simply as possible. Is it clickable (such as a button or hyperlink)? Does it allow for selection (such as a list box)? 3. Isolate fundamental UI elements by function or feature. Structure a complex UI in a way consistent with user scenarios. A keyboard navigation model is a useful guide. 4. Match a UI feature from the previous step with existing controls. The UI Automation Control Type specification is a useful guideline. If you have trouble finding the correct control type, see the Control Pattern specifications and locate a control type that has the control patterns you want.

Step 1: The Foundation Note: This sample project is available for download at MSDN Online (http://msdn.microsoft.com/enus/library/ms771315(VS.85).aspx). The Windows SDK also features two additional UI Automation provider samples. The foundation of UI Automation provider objects is the IRawElementProviderSimple interface implemented by the applications or by the UI Framework that supports UI Automation. Objects that contain a more complex tree structure should also implement IRawElementProviderFragment, and the root for this structure implements IRawElementProviderFragmentRoot. Objects whose performance is very important can use IRawElementProviderAdviseEvents to know which events they should raise. Finally, you should use the IRawElementProviderHwndOverride interface when the provider needs to reorder the automation tree. Most providers do not implement this interface. In this project, you implement only IRawElementProviderSimple, which contains the following methods:

www.code-magazine.com

// IRawElementProviderSimple STDMETHODIMP GetPatternProvider( PATTERNID patternId, IUnknown** pRetVal);

MEET MEMBERS OF MICROSOFT'S ACCESSIBILITY TEAM

STDMETHODIMP GetPropertyValue( PROPERTYID propertyId, VARIANT* pRetVal); STDMETHODIMP get_HostRawElementProvider( IRawElementProviderSimple** pRetVal); STDMETHODIMP get_ProviderOptions( ProviderOptions* pRetVal);

Before investing time designing a new control or customizing a standard control, you should ask yourself: “Do I really need that control?”

For the UI Automation Framework to recognize your UI Automation provider implementation, you must respond to the WM_GETOBJECT message. The UiaReturnRawElementProvider function helps produce the key to establish a connection to the UI Automation Core:

Michael Bernstein (Lead Software Design Engineer, User Interface Platform Team) My team builds great Accessibility and Speech user experiences in Windows, as well as the developer libraries and tools that enable those experiences. This is a very rewarding area for me: I can open a new world of computer experiences to those who have never been able to access them before

case WM_GETOBJECT: { if (_pProxy == NULL) { // Create an automation element object _pProxy = CCheckbox3UIAProxy::Create(this); } return UiaReturnRawElementProvider( _hwnd, wParam, lParam, _pProxy); } break;

Now you are ready to implement the support for UI Automation properties and pattern interfaces.

Step 2: Implementing the Properties and Control Patterns UI Automation providers add support for properties with the GetPropertyValue method. For control patterns, providers support the corresponding patternId for the GetPatternProvider method. You can register the properties and control patterns with GUIDs (such as the Toggle_Pattern_GUID) that you map to the corresponding IDs with the UiaLookupId function. For existing control patterns and properties, you can find corresponding IDs in UIAutomationClient.h (such as UIA_TogglePatternId).

David Tseng (Software Design Engineer in Test, User Interface Platform Team) I'm helping to ensure that the user has a great experience no matter how he or she interacts with a computer—whether it be through speech recognition, screen reading, or screen magnification. This happens by making sure the developer is empowered with high quality mechanisms to build the best experience possible.

Making Custom Controls Accessible

37

In this example, you get the property value of a check box:

IUnknown **pRetVal) { // Clear out param *pRetVal = NULL;

HRESULT CCheckbox3UIAProxy::GetPropertyValue( PROPERTYID propertyId, IUnknown **pRetVal) { // Clear out param pRetVal->vt = VT_EMPTY; // Specify ControlType.CheckBox for the // ControlType property if (propertyId == UIA_ControlTypePropertyId) { pRetVal->vt = VT_I4; pRetVal->lVal = UIA_CheckBoxControlTypeId; } return S_OK;

Vidhya Sriram (Software Development Engineer in Test, User Interface Platform Team) My team not only works on making programmatic access to GUI applications easier but also enables application developers to expose richer information about their application to assistive technology products. I love working with this team for the commitment and passion shown towards making computers easier to use for physically challenged users and thereby bringing a positive change in their lives.

}

Many accessibility practices are available in the form of industry standards (such as Section 508 of the Federal Rehabilitation Act).

To support controls with a binary status such as the check box (checked or cleared), the Toggle Pattern is best, and the implementation is straightforward as shown in Listing 1. Here, you get the pattern provider for the toggle pattern: HRESULT CCheckbox3UIAProxy::GetPatternProvider( PATTERNID patternId,

if (patternId == UIA_TogglePatternId) { *pRetVal = static_cast(this); AddRef(); } return hr; }

Part II: Using Dynamic Annotation Now, I will talk about using Dynamic Annotation for Win32® controls with relatively minor modifications. The technique was originally introduced with Microsoft Active Accessibility version 2.0. Its purpose is to enable a custom control to expose accessibility information without having to fully implement the IAccessible interface. With Windows 7 Automation API, you can now use Dynamic Annotation to override either Microsoft Active Accessibility or UI Automation properties of the proxy objects that OLEACC creates for standard Windows controls (many of those supported by COMCTL32.dll or USER32.dll). If your custom control is a slight modification of an existing control or requires support for only a few accessibility properties, Dynamic Annotation is a good candidate for implementing your accessibility solution. Dynamic Annotation provides three different mechanisms for handling annotations: Direct Annotation, Value Map Annotation, and Server Annotation. In this article I discuss only Direct Annotation and Server Annotation.

Listing 1: The Toggle Pattern supports the binary status of a check box control. // IToggleProvider to support the Toggle control pattern. HRESULT CCheckbox3UIAProxy::Toggle() { HRESULT hr = CheckAlive(); if (SUCCEEDED(hr)) { _pControl->MoveToNextState(); } return hr; } HRESULT CCheckbox3UIAProxy::get_ToggleState(ToggleState* pRetVal) { HRESULT hr = CheckAlive(); if (SUCCEEDED(hr)) { switch(_pControl->CheckboxCheckedState()) { case CHECKED: {

38

Making Custom Controls Accessible

*pRetVal = ToggleState_On; } break; case MIXED: { *pRetVal = ToggleState_Indeterminate; } break; case UNCHEcKED: { *pRetVal = ToggleState_Off; } break; } } return hr; }

www.code-magazine.com

A sample Dynamic Annotation project is available for download at MSDN Online.

How It Works Dynamic Annotation allows you to modify some of the Microsoft Active Accessibility properties, but not every Microsoft Active Accessibility property is relevant. The supported properties are: Name (PROPID_ACC_NAME) Description (PROPID_ACC_DESCRIPTION) Role (PROPID_ACC_ROLE) State (PROPID_ACC_STATE) Help (PROPID_ACC_HELP) KeyboardShortcut (PROPID_ACC_KEYBOARDSHORTCUT) • DefaultAction (PROPID_ACC_DEFAULTACTION)

Other Considerations: WinEvents Applications and the UI Framework use WinEvents to notify accessibility applications of changes in the user interface. When one of the following happens, call NotifyWinEvent:

• An object is created, destroyed, shown, hidden, reordered, or invoked. • A selection is changed. • A Name, Value, State, Description, Location, Parent, Help, DefaultAction, or Accelerator accessibility property changes in an object.

• • • • • •

While annotating many of these Microsoft Active Accessibility properties is straightforward, be careful when you annotate the Microsoft Active Accessibility Role property. Annotation of Role won’t affect the behavior of the control or proxy object. For example, do not annotate a button as a slider because the baseline Win32 button control may not offer all of the functionality of a slider control. To annotate a control, create an AccPropServices COM object, call the appropriate SetHwndProp methods to annotate the control, and then release the AccPropServices object. Do not forget to call CoInitializeEx to support all COM practices in your WinMain routine (Listing 2). All annotations made in a Microsoft Active Accessibility property are reflected in the UI Automation translation as well as in the Microsoft Active Accessibility-to-UI Automation Proxy. If you want to override or add a UI Automation property to the control, you can specify a UI Automation Identifier GUID instead of the Microsoft Active Accessibility PropID.

If you customize a standard control, try not to overload it with features. Unnecessary complexity can result in expensive accessibility support cost, not to mention reduced usability.

These calls are needed only for objects for which you implement accessibility. The system provides accessibility for common objects and will call the appropriate events when needed. Not all annotations require an additional WinEvent.

Part III: Implementing IAccessibleEx

… you can now use Dynamic

If you already have a solid MiAnnotation to override crosoft Active Accessibility imMicrosoft Active Accessibility plementation of a control, you can use the IAccessibleEx inUI Automation properties. terface to add more accessibility functionality. The advantage of this interface is that you can reuse existing accessible object implementations, properties, object tree structures, and WinEvents. As long as the baseline Microsoft Active Accessibility implementation is clean, IAccessibleEx is an attractive solution to enhance the custom control for UI Automation.

or

To extend your Microsoft Active Accessibility implementations with the IAccessibleEx interface, do the following:

Listing 2: Annotation supports extending UI Automation properties for existing controls. HRESULT CFancyTextControl::SetAccessibleProperties() { // COM has been assumed to be initialized... IAccPropServices* pAccPropServices = NULL;

hr = pAccPropServices->SetHwndProp(_hwnd, OBJID_CLIENT, CHILDID_SELF, PROPID_ACC_ROLE, var);

HRESULT hr = CoCreateInstance(CLSID_AccPropServices, NULL, CLSCTX_SERVER, IID_IAccPropServices, (void**)&pAccPropServices); if (SUCCEEDED(hr)) { // Annotating the Role of this object be STATICTEXT VARIANT var; var.vt = VT_I4; var.lVal = ROLE_SYSTEM_STATICTEXT;

www.code-magazine.com

pAccPropServices->Release(); } return hr; }

Making Custom Controls Accessible

39

Listing 3: These methods help UI Automation map to a corresponding IAccessibleEx instance. HRESULT CListboxAccessibleObject::GetObjectForChild( long idChild, IAccessibleEx **pRetVal) { VARIANT vChild; vChild.vt = VT_I4; vChild.lVal = idChild; HRESULT hr = ValidateChildId(vChild); if (FAILED(hr)) { return E_INVALIDARG; } // List item accessible objects are stored as an array of // pointers; for the purpose of this example, I assumed // the list contents will not change. Accessible objects // are created only when needed. if (itemProviders[idChild - 1] == NULL) { // Create an object that supports UI Automation and // IAccessibleEx for the item. itemProviders[idChild - 1] = new CListItemAccessibleObject(idChild, g_pListboxControl); if (itemProviders[idChild - 1] == NULL) { return E_OUTOFMEMORY; } } IAccessibleEx* pAccEx = static_cast (itemProviders[idChild - 1]);

if (pAccEx != NULL) { pAccEx->AddRef(); } *pRetVal = pAccEx; return S_OK; } HRESULT CListItemAccessibleObject::GetIAccessiblePair( IAccessible **ppAcc, long *pidChild) { if (!ppAcc || !pidChild) { return E_INVALIDARG; } CListboxAccessibleObject* pParent = m_control->GetAccessibleObject(); HRESULT hr = QueryInterface( __uuidof(IAccessible), (void**)ppAcc); if (FAILED(hr)) { *pidChild = 0; return E_NOINTERFACE; } *pidChild = m_childID; return S_OK; }

1. Implement IAccessibleEx. Listing 3 shows the implementation of the GetObjectForChild and GetIAccessiblePair methods so UI Automation can map an IAccessible and ChildId pair to a corresponding IAccessibleEx instance. 5. Expose IAccessibleEx with IServiceProvider:: QueryService. 6. Implement IRawElementProviderSimple to provide property values and pattern providers for your control. 7. Implement control patterns and properties that describe your control. 8. Implement WinEvents for your control.

Designing a UI Automation object model for a new user interface is straightforward if you follow a few basic principles.

Conclusions and Resources While there are many ways to make custom controls accessible, choosing the right solution can be tricky. It is easy to underestimate the cost of customizing controls. Be aware of the hidden costs of a complex UI design, and make sure you have a plan for making your wonderful new controls accessible to everyone. The examples introduced in this article are available for download from the MSDN article, “Making Custom Controls Accessible.” (http://msdn.microsoft.com/en-us/accessibility/cc307845.aspx) A more thorough discussion of this topic is available on the MSDN Accessibility Development Center (http://msdn.microsoft.com/en-us/accessibility/). LeAnne Fossmeyer Michael Bernstein David Tseng Vidhya Sriram

A sample IAccessibleEx project is available for download at MSDN Online. The sample project has baseline accessibility support for a custom List Box control and the UI Automation RangeValue control pattern.

40

Making Custom Controls Accessible

www.code-magazine.com

ONLINE QUICK ID 0811082

Microsoft Accessibility Testing Tools vs. the Ten-ton Gorilla of Accessibility Guidelines Compliance Karl Bridge Karl Bridge is a Programming Writer, specializing in developer content, for the Windows Experience (WEX) organization at Microsoft. As the name implies, WEX is focused on the Windows user experience for consumers, IT professionals, and developers. Karl has written extensively for MSDN and the Windows SDK with a focus on making the Windows developer community a happy and integral part of the Windows experience. Prior contributions include the developer documentation for managed UI Automation and the Windows Vista Sidebar platform. Causing confusion, consternation, and occasional despair amongst Karl's family, friends, and co-workers are his mid-Atlantic sensibilities— a confluence of English eccentricity tempered with Canadian reserve and further jumbled by numerous strolls through Europe, the Middle East, North Africa, and Asia. Fortunately, Karl has a spellchecker and an editor to buffer readers from the inevitable mélange of mid-Atlantic spelling and grammar.

42

Close your eyes, ignore your mouse, navigate with your keyboard, and rely on your ears alone. Now try to use an application you’ve built or tested. Can you? The few informal tests described in this article can expose a plethora of usability and accessibility shortcomings, oversights, and other issues in your application. But how do you test, assess, and rectify them? o ensure the most consistent user experience across the Microsoft® Windows® product line, Microsoft has developed a number of tools to assist developers and testers to verify the accessibility implementations in their applications.

T

by exposing control properties, methods, and events such that you can use a tool to manipulate and interact with the UI manually or through automation.

Fast Facts

In this article, I describe each of these tools and explain what they are designed to test and the scenarios in which each tool, alone or in combination, may be useful. I also describe how you can use these tools throughout the development cycle and how to incorporate them into automated and nonautomated test frameworks.

An Overview of the Windows Automation Frameworks

According to a study that Microsoft commissioned and Forrester Research conducted: • 40% (51.6 million) of computer users with mild difficulties or impairments are likely to benefit from the use of accessible technology. • 17% (22.6 million) of computer users with severe difficulties or impairments are likely to benefit from the use of accessible technology.

By incorporating the relevant components of the Windows Automation API and following common accessible design practices, developers and testers can make applications running on Windows significantly more useful for people with vision, hearing, or motion disabilities. In the following sections I briefly describe the frameworks that constitute the Windows Automation API.

Microsoft Active Accessibility

The tools I discuss in this article are designed to test and report on a set of standardized accessibility issues typical of an application user interface (UI). The tests rely on programmatic access of the UI implementing the relevant API components defined in the framework known as the Windows Automation API.

Microsoft Active Accessibility is a set of COM interfaces and APIs that provide the means to expose and collect information about UI elements and controls. This allows assistive technologies, test tools, and automation frameworks programmatic access to and manipulation of the UI.

Despite the name, the Windows Automation API encompasses both accessibility and automation frameworks, including Microsoft Active Accessibility®, UI Automation, the IAccessibleEx interface, as well as other related and evolving technologies. Each of these frameworks enables programmatic access to the UI at run time

UI Automation

Microsoft Accessibility Testing Tools vs. the Ten-ton Gorilla of Accessibility Guidelines Compliance

UI Automation is similar to Microsoft Active Accessibility in that it provides a means for exposing and collecting information about UI elements and controls to support user interface accessibility and software test automation. However, UI Automation

www.code-magazine.com

is a newer technology that provides a much richer object model than Microsoft Active Accessibility, and it is compatible with both Win32® and .NET. Note: UI Automation exposes each object in the UI as an AutomationElement object to client applications. An AutomationElement object exposes common properties of the UI element it represents, such as the control type and relevant control patterns that provide properties specific to the control type.

The IAccessibleEX Interface The IAccessibleEX interface is a new interface that provides the ability for developers to add partial support for UI Automation to controls with existing IAccessible COM interface implementations. This is transparent to testers and the tools discussed in this article.

Evaluating the Accessibility of Your Application There are two primary approaches to evaluating the accessibility of an application: a manual audit based on a set of commonly accepted heuristics for an accessible UI (also known as static testing) and a partial audit to a fully automated test platform that evaluates the implementation of an accessibility framework within your application’s code (dynamic testing). Both methods can provide results based only on a set of guidelines, either from the heuristics of the audit or those built into the design of the accessibility framework.

Accessibility Auditing Typically, auditing your application for accessibility issues is a manual process that involves attempting to access and use the full functionality of the application exclusively through keyboard input and other assistive technologies (such as screen readers). As an auditor, you assess each issue discovered during the audit and log a bug for each issue as necessary. You then assign a priority and severity to each bug and, where possible and appropriate, provide recommendations for fixing or working around the issues that the audit uncovers. These recommendations are based on a combination of experience and accessibility guidelines from industry or advisory groups.

execution based on a set of predefined test cases. Depending on the sophistication of the tool, I will supply generic recommendations for specific issues uncovered during testing. In addition to compiling reports for basic unit tests, some of the tools discussed also offer support for regression and integration testing. This ability to find and track an accessibility bug through to resolution in a process you can automate makes these tools indispensible additions to a tester’s toolkit.

Despite the name, the Windows Automation API encompasses both accessibility and automation frameworks.

And in this Corner, We Have… The Tools The following Microsoft tools enable developers to test accessibility implementations and functionality in applications using Microsoft Active Accessibility or UI Automation.

UI Automation Verify UI Automation Verify (UIA Verify) is a test automation framework that facilitates automated testing for specific Microsoft UI Automation implementations. This framework provides the basis for the UI Automation Test Library command-line tool and Visual UI Automation Verify (Visual UIA Verify), a Windows-based graphical user interface (GUI) for the test framework. Most of the UIA Verify framework functionality is provided through a DLL (UIATestLibrary.dll) that enables the testing of specific UI Automation functionality and that logs the test results.

Accessibility Testing For the purposes of this article, accessibility testing is associated with automated, repeatable code

The Visual UIA Verify framework is a Windows driver for the UI Automation Test Library. This tool

www.code-magazine.com

The tools discussed in this article are available from various locations, which I’ve listed below. UI Automation Verify Available from CodePlex, the Microsoft open-source project-hosting Web site: http://www.codeplex.com/ UIAutomationVerify UI Spy UI Spy typically ships with the Windows SDK. However, since it did not ship in the Windows SDK for Windows Server 2008 and .NET Framework 3.5, released in February of 2008, you can find it in the earlier Windows SDK for Vista Update: http://www.microsoft. com/downloads/details. aspx?familyid=4377F86DC913-4B5C-B87E-EF72E5B4E065& displaylang=en

Note: The Visual UIA Verify GUI does not support event monitoring or testing and validating text-based content using the UI Automation TextPattern and TextPatternRange control pattern interfaces. The UI Automation Test Library does not share these limitations. To use the UI Automation Test Library, a driver (or client) application obtains an AutomationElement object from a control that requires verification. The driver, in turn, supplies the AutomationElement object to the UI Automation Test Library, which executes the tests required to validate the UI Automation implementation. Figure 1 illustrates the typical workflow when using the UI Automation Verify framework to test an application.

Since there is an inherent risk of testing the functionality of the tool rather than the application, auditing should be conducted by seasoned testers familiar with accessibility requirements and the broad range of assistive technologies.

Where to Get the Tools

Microsoft Accessibility Testing Tools vs. the Ten-ton Gorilla of Accessibility Guidelines Compliance

43

is designed for manual testing. It provides an interface to UI Automation Test Library functionality and eliminates the code overhead of a commandline tool. Figure 2 illustrates the five functional areas of the Visual UIA Verify framework.

UI Spy UI Spy, the precursor to the UIA Verify framework, is a Windows application designed for UI Automation testing of specific accessibility issues.

Figure1: Here you can see the UI Automation Verify framework workflow between driver, application, and UIATestLibrary.

While the lack of built-in automation support limits the effectiveness of UI Spy, it can still be an invaluable tool for viewing an application's UI hierarchical structure, property values of individual controls, and event tracking. You can also use UI Spy to interact with controls through UI Automation control patterns. This extensive functionality provides the

Figure 2: The five functional areas of the Visual UI Automation Verify framework.

44

Microsoft Accessibility Testing Tools vs. the Ten-ton Gorilla of Accessibility Guidelines Compliance

www.code-magazine.com

If you don't need the entire SDK, you can get the .NET Framework Tools with UI Spy by downloading the Windows SDK for Vista Update. This will install side by side with the new Windows SDK for Windows Server 2008. For example: C:\ Program Files\Microsoft SDKs\ Windows\v6.0\Bin (with UISpy) or C:\Program Files\Microsoft SDKs\Windows\v6.1\Bin UI Accessibility Checker AccChecker is available from CodePlex, the Microsoft opensource project hosting Web site: UI Accessibility Checker: http://www.codeplex.com/ AccCheck Inspect Objects, Accessible Explorer, and Accessible Event Watcher These tools are available as individual downloads from the Microsoft Active Accessibility 2.0 Software Development Kit (SDK) Tools Web site: http://www.microsoft. com/downloads/details. aspx?familyid=3755582AA707-460A-BF211373316E13F0& displaylang=en#filelist

Figure 3: The control view of UI Spy querying displays the UI Automation properties of an element, running with limited user access.

means for developers and testers to verify that an application UI is programmatically accessible to assistive technology devices. Figure 3 shows UI Spy querying and displaying the UI Automation properties of an element.

content using the TextPattern and TextPatternRange control pattern interfaces of UI Automation.

UI Spy is similar to Visual UIA Verify in that it does not support the testing and validation of text-based

UI Accessibility Checker (AccChecker) verifies the design and implementation of Microsoft Active

www.code-magazine.com

UI Accessibility Checker

Microsoft Accessibility Testing Tools vs. the Ten-ton Gorilla of Accessibility Guidelines Compliance

45

This ability to find and track an accessibility bug through to resolution in a process you can automate makes these tools indispensible additions to a tester’s toolkit. Accessibility in a control or application UI, regardless of the underlying UI framework. The AccChecker architecture incorporates three levels, or stages, of functionality: 1. A Windows GUI application designed to support manual testing, message logging, and suppression generation. 2. An API designed for use in automated testing frameworks. 3. A console application that supports unmanaged test automations typically unable to use the AccChecker-managed API.

Figure 4: The AccChecker UI runs with these default verifications.

All three levels of AccChecker provide similar functionality through verification routines that include Microsoft Active Accessibility programmatic access, programmatic event generation, control layout and keyboard navigation validation, and a basic screen-reader transcription service. Figure 4 illustrates the initial view of the AccChecker UI with default verifications.

Inspect Objects Inspect Objects (Inspect) is a Microsoft Active Accessibility-based tool designed for inspecting and testing IAccessible COM interface properties as well as navigating UI elements and controls. The Inspect tool tracks elements by using keyboard focus, mouse cursor movement, or its navigation tools. This built-in navigation is based on a snapshot view of the Microsoft Active Accessibility element tree hierarchy. The Inspect tool is very useful for quickly discovering UI elements that lack unique and meaningful Microsoft Active Accessibility Name property values. The Name property is required for assistive technology devices, such as screen readers, to identify controls such as buttons, text boxes, list boxes, images, and links. Figure 5 shows the Inspect tool querying the Microsoft Active Accessibility properties of an element in the Notepad menu.

Accessible Explorer Figure 5: The Inspect tool UI queries Notepad.

46

Microsoft Accessibility Testing Tools vs. the Ten-ton Gorilla of Accessibility Guidelines Compliance

Accessible Explorer (AccExplorer) is a Microsoft Active Accessibility-based tool that offers func-

www.code-magazine.com

tionality that is very similar to the Inspect tool. However, it differs in two important features: • A somewhat limited ability to track and display a live view of the Microsoft Active Accessibility element tree. • Microsoft Active Accessibility verification of specific UI elements. The AccExplorer tool provides a passed, failed, and warning system on selectable Microsoft Active Accessibility properties based on the HRESULT returned by the automated query. Figure 6 illustrates how to use the AccExplorer tool to verify specific Microsoft Active Accessibility properties for the Notepad content pane.

Accessible Event Watcher Accessible Event Watcher (AccEvent) is a command-line tool that provides developers and testers with a means of validating Microsoft Active Accessibility-based WinEvents raised by UI elements. The AccEvent tool reports events when an element is invoked or selected, when it changes state, and when there is a change in focus. You can track in-process and out-of-process events based on the options available with the Microsoft Active Accessibility SetWinEventHook function. Even though the AccEvent tool may report some UI Automation-specific WinEvents, you should use a dedicated UI Automation client for testing UI Automation events due to the differing designs of the event object models in the two frame works. Figure 7 illustrates how the AccEvent tool catches the toolbar clock events.

Figure 6: The AccExplorer tool can test specific Microsoft Active Accessibility properties in the Notepad content pane.

Conclusion No tool alone can definitively test and guarantee compliance with all accessibility guidelines. However, the tools described in this article make achieving that goal significantly easier through their comprehensive coverage of the many accessibility compliance issues that software vendors face today. Karl Bridge

Figure 7: AccEvent reports time changes for the toolbar clock.

www.code-magazine.com

Microsoft Accessibility Testing Tools vs. the Ten-ton Gorilla of Accessibility Guidelines Compliance

47

ONLINE QUICK ID 0811092

Internet Explorer 8 New Accessibility Features

JP Gonzalez-Castellan JP is the Program Manager responsible for improving the Accessibility experience in Internet Explorer. He was born in Argentina and got his Computer Science degree at Rose-Hulman Institute of Technology in Indiana. He has been involved at Microsoft with Accessibility since 2004.

Windows® Internet Explorer® 8 has a lot of cool new features that make Web page browsing more accessible. I’m going to cover Caret Browsing, Zoom Version 2, High DPI, Accessibility Rich Internet Applications (ARIA) support and User Interface Automation (UI Automation) support. Caret Browsing is particularly helpful for low-mobility users. Zoom 2 and High DPI support targets low-vision users, and the new ARIA and UI Automation support targets screen-reader users. Low-mobility users prefer to use the keyboard or devices that interact with a virtual keyboard. Some low-vision users require specialized assistive software to interact with computers while others can do well with features and tools shipped with the operating system.

hen you make something accessible you are You can hold the SHIFT key down and press the not only impacting the accessibility comarrow keys to select text. munity but the entire user base. My favorite As I mentioned before, many users prefer the keyexample is the wheelchair access ramps. After the board to the mouse because they find it faster for American Disability Act was passed, public gathercertain tasks. Users can select a word or phrase, ing places, like airports, added wheelchair ramps. as you see in Figure 2, bring up the Accelerators Airports soon noticed that mothers with baby strollthrough the context-menu key, ers and passengers with rolling select Translate with Windows suitcases were using the ramps Fast Facts Live, and see the selection’s too, since it was easier than meaning in Spanish, without picking up the strollers and According to a ever taking their hands off the suitcases over the ledge. ImHarris Poll survey, keyboard. prove the accessibility of your Americans with disabilities software and everybody wins. spend twice as much time When you provide robust keyon the Internet as those Zoom 2 board navigation to benefit users who can’t use the mouse, without disabilities. Page zoom lets you enlarge or you are also enabling users who Roughly 750,000 million reduce the view of a Web page to want to perform tasks faster, people the world over improve readability. The feature since there are now fewer tasks have some disability. is particularly useful on very large they need to use the mouse for. and very small displays, allowing for scaling of content while maintaining the intended Caret Browsing page layout. The second iteration of the zoom feature set (the first shipped in Internet Explorer 7) focuses …users can select a word Caret Browsing is a new Acceson providing a higher-quality, more predictable, and or phrase, bring up the sibility feature that allows users persistent zooming experience. Primary features in to navigate a Web page using a this release include the elimination of horizontal Accelerators through the moveable cursor on the screen scroll bars for the majority of mainstream scenarcontext menu key, select with the keyboard. Users can ios and the introduction of persistent zoom states. select and copy text as well as I personally find this feature useful when using my Translate with Windows Live, tables and images using only the TV to browse the Web. I sit far away from the TV; and see the selection’s meaning keyboard. using a wireless keyboard, I use the Zoom to make the pages more readable. The extensive keyboard in Spanish, without ever taking Moving the cursor within the shortcut’s list allows me to use the mouse as little as their hands off the keyboard. text of a Web page is similar to possible. moving the cursor within the text of a Microsoft Word docuZoom 2 works by adjusting the internal state of the ment, as shown in Figure 1. Pressing F7 toggles dots-per-inch (DPI) calculations within Internet Caret Browsing on or off. You can enable Caret Explorer 8. To understand this better, consider how Browsing for one tab or for all tabs and windows. different display settings affect a page's layout.

48

W

Internet Explorer 8 New Accessibility Feature

www.code-magazine.com

Resources and References You can learn much more about ARIA and how to use it in your Web content from the following W3C resources: Introduction to ARIA http:// www.w3.org/TR/wai-ariaprimer/ ARIA Roles, States and Properties http://www.w3.org/ TR/wai-aria/

Figure 1: Caret is visible on the screen for the first time.

ARIA Best Practices http://www.w3.org/TR/waiaria-practices/ To learn more about Windows Accessibility APIs go to http://msdn.microsoft.com and search for Microsoft Active Accessibility and UI Automation.

Figure 2: Pressing the context menu key brings up the context menu; so you can access Web Accelerators using only the keyboard. Screen Resolution: Changing from a 1200 × 1600-pixel screen resolution to an 800 × 600-pixel resolution means changing the spacing between pixels. In other words, pixels are effectively bigger. A screen element of 100 × 100 pixels is now twice

www.code-magazine.com

as big on the 800 × 600-pixel screen as it was on the 1200 × 1600-pixel screen. System DPI: Changing from 96 DPI to 192 DPI, but keeping all other factors the same, implies more

Internet Explorer 8 New Accessibility Feature

49

pixels in an inch. The spacing between pixels has not changed. However, more pixels are required to form a logical inch. Zoom Factor: Changing the zoom factor from 1 to 2 means that everything should be twice as big. Figure 3 shows how to calculate the Zoom Factor. HTML Layout Measurements: This value is always 96 DPI, as explained in the CSS 2.1 specification (http://www.w3.org/TR/CSS21/syndata.

Figure 3: Zoom factor = System DPI/Layout DPI. Listing 1: Sample Web page to understand better the new Zoom.

html#percentage-units). The Internet Explorer page zoom feature cannot change any of the above settings except for the zoom factor. However, it can lay out or render a page at a DPI setting that is different from the configured system DPI, thus achieving zoom results.

You can see the importance of carefully selecting the anchor for positioned elements and using relative dimensions instead of absolute or device-dependent dimensions (such as pixels). The sample code in Listing 1 divides the screen into quadrants of color, using absolute positioning, to show the difference between the Internet Explorer 8 adaptive page zoom and the Internet Explorer 7 optical page zoom The boxes in Figure 4 are absolutely positioned, and they are anchored to the corners of the viewport. Each is sized using relative dimensions (%). The box in the middle is anchored to the center of the body. In Figure 5, you can see how Internet Explorer 7 displays a 175% zoom while Figure 6 shows how Internet Explorer 8 handles a 175% zoom.

<style> div { width: 50%; height: 50%; position: absolute; } #center { width: 20%; height: 20%; left: 40%; top: 40%; border: 5px black solid; }

You can see the importance of carefully selecting the anchor for positioned screen elements and using relative dimensions instead of absolute or device-dependent dimensions (such as pixels). If you had positioned the center DIV using pixels, the DIV would have moved from the center when zoomed.

High DPI

Red

Blue

Yellow

Green

50

Internet Explorer 8 New Accessibility Feature

You can use the Windows DPI Scaling (https://windowshelp.microsoft.com/Windows/en-US/help/454f5078-2b514cda-b4c0-6391e870c41d1033.mspx) feature to scale up Windows fonts and user interface (UI) elements (such as buttons, icons, input fields, and so on) by a given percentage. This is different from the scaling that occurs when you lower the display resolution because in the DPI Scaling case, Windows is providing fonts and UI elements that are drawn with more pixels, resulting in a larger, higher fidelity, and sharper Windows experience. Like Windows Vista, the Internet Explorer 8 user interface is entirely DPI Scaling-aware, and you will notice that all UI elements and fonts are scaled accordingly. Additionally, larger and higher fidelity icons are used. Notice the difference in the Internet Explorer chrome when Windows DPI Scaling is set to 96 DPI (Figure 7) and 120 DPI (Figure 8).

www.code-magazine.com

Figure 4: Notice that at 100% the square is centered in the viewport. By default, Internet Explorer 8 will zoom the content of the Web to match your Windows DPI Scaling settings. (Please note that this is a change from Internet Explorer 7, which did not zoom to match Windows DPI Scaling settings.) For example, if you set your DPI Scaling to 120 DPI, Internet Explorer 8 will zoom the content of Web pages by 125%.

ARIA syntax is a great mechanism to unlock your dynamic, rich Web applications for everyone. For more information, especially for developers who want to take advantage of High DPI in their Web pages and WebOCs, please see: Making the Web Bigger (http://go.microsoft.com/fwlink/?LinkID=12 5391&clcid=0x409)

www.code-magazine.com

Figure 5: 175% zoom in IE 7 magnifies the text and divs in the Web page and forces the horizontal and vertical scrollbars to show up.

Internet Explorer 8 New Accessibility Feature

51

Figure 6: 175% zoom in IE 8 magnifies the Web page’s text but not the divs. So the layout of the page is maintained and hence no horizontal and vertical scrollbars are needed. fied Web pages for accessibility, you can use ARIA to mark up your rich Web applications with roles, states, and properties. For example, to match the behavior you create through script, you can define a DIV element as a button, check box, or another ARIA role. Figure 7: Internet Explorer Chrome at 96 DPI.

Figure 8: Internet Explorer Chrome at 120 DPI.

ARIA and UI Automation Support In the latest release of Internet Explorer 8, Microsoft has made a big investment in Accessible Rich Internet Applications (ARIA). The W3C defines ARIA as a syntax for making dynamic Web content and custom UI accessible. Windows Internet Explorer 8 recognizes the ARIA role, state, and property information from controls and exposes it to assistive technologies. Assistive technologies can then use Microsoft Active Accessibility® and/or UI Automation Accessibility APIs, which are supported for the first time in Internet Explorer, to retrieve the information. Instead of building separate simpli-

52

Internet Explorer 8 New Accessibility Feature

ARIA syntax is a great mechanism to unlock your dynamic, rich Web applications for everyone. Today Web pages with dynamic content and custom UI controls (such as TreeView controls) have to reuse existing HTML controls. For example, you can make custom TreeView controls accessible by defining each item as an HTML list element. This approach can add complexity to the code, make it more difficult to implement, and prevent all users from getting the same rich behavior.

Code Sample: Create an Interactive TreeView Control Using ARIA The code sample in Listing 2 uses ARIA to create an interactive TreeView control using HTML and

www.code-magazine.com

Listing 2: Creating an interactive TreeView control.

} }

<style type="text/css"> li { list-style-type: none; } <script type="text/javascript"> // Function to expand and collapse tree items by referencing the ariaExpanded attribute. function Toggle(root, itemId) { var el = document.getElementById(itemId) var treeitem = document.getElementById(root) if (el) { if (treeitem.ariaExpanded == "true") { treeitem.ariaExpanded = "false"; el.style.display = "none"; treeitem.childNodes[0].innerText="+ "; } else { treeitem.ariaExpanded = "true"; el.style.display = "block"; treeitem.childNodes[0].innerText="- ";

JavaScript. Users of Internet Explorer 8 using an assistive technology (such as a screen reader) can interact with this control and experience the same rich behaviors users without vision impairments do. If you don’t have access to a screen reader you can use the Inspect32 testing tool available at http://www.microsoft.com/downloads/. You can experience the benefits of ARIA first hand by hovering with the mouse or tabbing with the keyboard through the TreeView. The role displayed in Inspect 32 is “treeview item” instead of “outline item”. Similarly a screen reader will speak out “treeview item” instead of “outline item”.

}

• + Transportation
  • Car
  • Boat
  • Bike

Conclusion With Internet Explorer 8 and its support for ARIA and UI Automation, developers have a number of useful tools for developing accessible, rich Web pages. With Caret Browsing, the new Zoom functionality, and High DPI support, Internet Explorer 8 helps make content more accessible to your users. JP Gonzalez-Castellan

Users of Internet Explorer 8 using an assistive technology (such as a screen reader) can interact with a custom the TreeView control and experience the same rich behaviors users without vision impairments do.

www.code-magazine.com

Internet Explorer 8 New Accessibility Feature

53

Are You Ready To Start?

EPS Can Help! EPS Software provides:

Application Development • • • • •

.NET 3.0 (WPF, WCF) Windows & Smart Client ASP.NET & AJAX Web Tablet PC & Mobile PC Windows Mobile

Prototyping Services Situation Assessment Project Management Mentoring Training

Contact us at: [email protected] 866-529-3682

www.eps-software.com

ONLINE QUICK ID 0811102

A Pragmatic Approach to WPF Accessibility

Alvin Bruney Alvin Bruney is a Technology Specialist working for Royal Bank of Canada in the .NET Centre of Excellence program. He is a Microsoft Press author and a long-time ASP.NET MVP.

As is often the case, applications are not typically designed with accessibility in mind. Usually, applications are designed to satisfy business requirements. If those business requirements do not include accessibility, more likely than not, the application as a whole will be inaccessible to important segments of users. There are, however, steps you can take to mitigate this common lack of foresight in requirements analysis. o start off a project right, address accessibility at the design level, even when overlooked in the business requirements. Treat accessibility as you would usability: neither is explicitly a business requirement, but both are required parts of a successful user interface. Think about how users can interact with your user interfaces and plan for alternate accessibility with screen readers, magnifiers, visual cues, keyboard usage, and so on.

T

in WPF is due to the fact that the new UI Automation handles accessibility support behind the scenes. You can take advantage of this framework with new projects and use the framework’s bridge and proxy technology to extend existing projects’ feature sets.

UI Automation is more robust and exposes more programmatic information to assistive technologies, allowing for an improved experience. You can read more about UI Automation elsewhere in this magFast Facts azine as well as at the MSDN® According to the U.S. Accessibility Developer CenPresident's Committee on ter. For now, I want to focus Employment of People with on testing accessibility.

Another way to ensure accessibility is to use the right development tools. The good news is that all versions of Visual Studio® address accessibility; the bad news is that this supDisabilities, the majority of port doesn’t guarantee perfect accommodations cost less Testing WPF Applications coverage, and you should inthan US$500, and many clude test and validation tools have little or no cost at When you build WPF applicain your development arsenal. all. However, the average tions, you can test accessibilOnce you’ve got your design recruiting cost equals ity support using tools such as in place and you’ve built what more than 13 percent of an Jaws, UI Spy, Windows-Eyes, you believe to be an accessible and ZoomText to name a few. application, control, Web page, employee's annual salary. There’s even an accessibilor service, you can use these ity checker, shown in Figure 1, tools to test your deliverables on the Visual Studio tools menu for applications for compliance with accessibility requirements. based on Web site project templates. This tool analyzes Web pages against Web Content Accessibility In this article, I’ll talk about Guidelines (WCAG) and Section 508 of the U.S. these issues with respect to WinTreat accessibility Rehabilitation Act. dows® Presentation Foundation as you would usability: (WPF). I’ll also touch on some A strategy that I recommend is to use a combiareas where current offerings neither is explicitly a nation of tools so that you can achieve greatfrom adaptive technology venbusiness requirement, er accessibility testing coverage. For instance, dors present challenges to both the Accessibility Checker in Visual Studio tarusers and developers. but both are gets Web pages; UI Spy targets UI Automarequired parts of a tion compliance; AccChecker targets Microsoft® WPF Has Accessibility Active Accessibility® compliance and general accessuccessful user interface. sibility testing. You can decide which tool is a better Built In fit based on your needs and the underlying accessibility framework you’re using. You might use UI When you build a WPF application, you still need Spy and UIA Verify for testing WPF applications to design for accessibility; however, it’s an easier based on UI Automation and Accessibility Checker road to travel today because of the new accessible for those based on Microsoft Active Accessibility. technology framework that you can build on. Bear in mind that none of these tools help with designing accessibility into your applications. They One reason for the increased accessibility support

56

A Pragmatic Approach to WPF Accessibility

www.code-magazine.com

can only validate that what you did design and build complies with known, testable requirements.

Challenges with the New Technology For the most part, supporting accessibility with the UI Automation framework works as intended. However, I want to walk through some cases where potentiality may not align with reality. UI Automation is new, and some assistive technology vendors (ATVs) do not yet support it, which presents several challenges. Here are some specifics you may run into, as well as a few workarounds.

JAWS Screen Reader The screen reader JAWS has a “JAWS Cursor” mode that allows users to read text and UI information line by line on the screen. When in this mode, the reader does not read the screen information for applications built with WPF native controls. The root cause is the current lack of support for UI Automation in JAWS. Fortunately, Jaws Cursor mode is not the primary interface of the screen reader, and the workaround is simply to avoid this mode where possible. As of this writing, JAWS plans to fully support UI Automation in 2009.

a price for supporting this backward compatibility: bitmaps and images that use the legacy raster technology begin to blur and distort under increased DPI. Users with certain types of visual impairments may However, there’s a price for have difficulty working with such supporting this backward an application.

compatibility: bitmaps

Even users without visual impairand images that use the legacy ments who work with these applications at non-native DPIs can raster technology begin to blur and suffer eye strain after prolonged distort under increased DPI. exposure. To put this in perspective, consider a desk clerk working eight hours a day for several months with a software application where some of the images are blurred. This situation can easily lead to a health issue, which can quickly turn into a liability issue. To make working with your applications easier on everyone’s eyes, you should create vector-based images and bitmaps using tools such as Microsoft Blend, or convert existing raster-based images to vector-based images. Vector-based images can respond automatically to changes in DPI settings while retaining visual clarity. There are already a number of products that offer this conversion for free or for a fee.

Win32 Controls

ZoomText

WPF supports legacy Win32 controls with the HwndHost control, providing an easier migration path from Win32 to XAML-based applications. However, there’s

Users with visibility impairments use programs like ZoomText to read content on screen. ZoomText, created by Ai Squared, is a screen reader-magnifier for Microsoft

Figure 1: The accessibility checker in Visual Studio 2008 analyzes Web pages and displays results.

www.code-magazine.com

A Pragmatic Approach to WPF Accessibility

57

Figure 2: Parts of the user interface are cut off on this magnified screen.

Expanding Law On October 2, 2007, National Federation of the Blind (NFB) v. Target Corporation was certified as a class action lawsuit. The plaintiff, NFB, alleges that the Target.com Web site violates the rights of the disabled because some parts of the site are inaccessible to the visually impaired. The case raises eyebrows as the first accessibility-based class action lawsuit to target a public facing Web site owned by a non-federal entity. These lawsuits have tended to target government institutions.

Windows. In some instances, the part of the magnified screen on a WPF native application may completely disappear or become unreadable as in Figure 2. This happens because the hardware-accelerated WPF rendering bypasses the software rendering layer. While you can avoid this in general either by turning off hardware acceleration or by using the Magnification API, the latter option is not available with the ZoomText application. It’s important to note that while this is not strictly a WPF issue, the end result is that WPF-based applications may be inaccessible to ZoomText. Unfortunately, the workarounds are not particularly pleasant. One option is to use another screen magni-

If NFB prevails, the mandate for and scope of accessibility requirements may apply to all Web pages. But, as I have shown you in this article, there are proactive steps that IT organizations can take to comply with these requirements.

fier that supports UI Automation. Today, the choices available are pretty close to nil. The situation is expected to improve as ATVs start supporting WPF, UI Automation, and other standard accessibility technologies. A second option is to disable hardware acceleration for Microsoft Windows in the dialog shown in Figure 3. Hardware acceleration uses DirectX® to improve video and animation quality. When you disable hardware acceleration, DirectX is not used.

Hardware Refreshes Another accessibility issue may arise particularly in the workplace with hardware refreshes. Periodically, the computers or workstations that employees use are upgraded in a hardware refresh cycle, including new monitors. While applications built on WPF technology will automatically scale correctly to fit the increased screen resolution, legacy-based controls may not scale as you expect. Some common controls such as the WPF dialog box are not WPF native. Instead, they are simply wrappers around legacy Win32 technology. Usually, issues with these controls manifest themselves in any number of ways, from controls with chopped or unreadable text to controls that do not maintain their layout integrity under different screen conditions, such as the DPI changes and high contrast mode in Figure 4.

Migration from Legacy Technologies Figure 3: The hardware acceleration slider lets you disable hardware acceleration in Windows XP.

58

A Pragmatic Approach to WPF Accessibility

Even legacy Visual Basic® 6 (VB6) migration can inadvertently introduce accessibility issues in WPF. Let’s examine one common scenario. Most VB6

www.code-magazine.com

common controls have an autosize property that allows the control to grow or shrink as the form or parent container is resized. By default, the autosize property is set to false. When these applications are migrated to .NET, the default setting for the autosize property remains set to false, and the result is that the contents of the control cannot grow as screen conditions change. Contrast this to a new application developed using native WPF controls where the autosize property isn’t even present. Here, the control automatically grows as required to meet users’ needs. Figure 4: The effects of legacy technology can manifest as unreadable text in high contrast mode.

The visually impaired user couldn’t care less which technology is to blame; the application is inaccessible! You need to plan for this progression to avoid introducing accessibility issues into your applications.

ble! You need to plan for this progression to avoid introducing accessibility issues into your applications.

In my experience, WPF has made the most progress on the accessibility front by far. However, you cannot rely on a platform and tools alone to cover accessibility mandates. The onus is on you to design for accessibility so that you can deliver a better product to your clients.

Another accessibility issue arises when these legacy VB6 controls are migrated first to .NET, where they retain their “inaccessible” settings, and then to WPF inside a WindowsFormsHost control. It’s important to note that this not a WPF issue at all. In fact, its roots are buried deep in VB6 legacy settings. However, WPFbased applications can inherit the accessibility burden.

As the WPF technology gains traction, assistive technology vendors will come on board in increasing numbers with support for the new UI Automation framework. At that point, you can expect WPF-based applications to have full accessibility support. Until that time, you should plan for accessibility and use the available tools described earlier to help test for accessibility coverage.

The visually impaired user couldn’t care less which technology is to blame; the application is inaccessi-

Alvin Bruney

Why Bother Designing and Building Accessible Applications? For one, an accessible application caters to the largest possible audience, and that’s always good for business. The services you offer won’t exclude the estimated 20 million Americans with some sort of disability (source: American Census Bureau, 2006).

ADVERTISING INDEX

Conclusion

Advertisers Index Active Software Professionals www.aspalliance.com

6

VFP Conversion www.VFPConversion.com

31

Applied Innovations www.appliedi.net

76

WestWind www.west-wind.com

35

Aspose www.aspose.com

75

Xiine www.xiine.com

69

Code Magazine www.code-magazine.com

7

Developer Express www.devexpress.com

2

DevMavens www.devmavens.com EPS Software Corp. www.eps-software.com Microsoft www.microsoft.com/enable/ Tech Conferences www.devconnections.com

www.code-magazine.com

Advertising Sales: Lake Quincy Media LLC 330-677-2440 [email protected] [email protected]

15 54, 60 5 41

This listing is provided as a courtesy to our readers and advertisers. The publisher assumes no responsibility for errors or omissions.

Sales Managers Erna Egger +43 (664) 151 0861 [email protected] Tammy Ferguson 832-717-4445 ext 26 [email protected]

A Pragmatic Approach to WPF Accessibility

59

ONLINE QUICK ID 0811112

Writing a UI Automation Provider for a Win32-based Custom Control Brendan McKeon Brendan is a senior software development engineer in the Windows division. He’s worked at Microsoft for 10 years and was a key senior contributor to UI Automation.

Do you have a complex custom control that you want to make programmatically accessible, but you aren’t sure how? Custom controls, by their nature, tend to be very diverse: each is typically written for a specific purpose, making it difficult to generalize implementation details. How do you know what to implement? You should consider supporting accessibility for any custom control that performs its own rendering and input management—routing mouse and keyboard input—within the HWND that it owns. n this article, I walk through the steps you need to providers expose information about the UI as a tree of follow to implement a server-side provider for a IRawElementProviderSimple objects. The IRawEleWin32-based custom control mentProviderSimple interface using Microsoft® UI Automais the base interface representing Fast Facts tion. The example I use assumes and exposing information about the custom control has its own a single UI element. In general, UI Automation Handle to a Window (HWND) uses client-side providers to The MyCustomControlProvider and Window Procedure. While support built-in controls such class only exposes information the emphasis is on unmanaged as Win32 buttons, for the overall HWND. Here, I C++ implementations, the techcomplete the implementations niques are equally applicable to while server-side providers of these IRawElementProvidmanaged code. expose information about erSimple methods. In this excustom controls to ample, I expose a custom name UI Automation. What is a Provider? and specify a control type:

I

UI Automation presents the UI on the desktop to a client application in the form of a tree of IUIAutomationElement objects, each of which can expose properties describing the UI. The UI itself must supply these properties using APIs and messages. The work of calling the UI-specific APIs and returning the information back to UI Automation is done by a component called a provider. UI Automation comes with providers for the various standard OS controls, such as Win32 button and list controls, already built in. To make a custom control accessible through UI Automation, you need to supply a UI Automation provider for that control.

Exposing Built-in Properties of a Single Element The first step to making your custom control accessible is to create a class that implements the UI Automation provider interface, IRawElementProviderSimple, as shown in Listing 1. While UI Automation clients see the entire user interface (UI) as a tree of IUIAutomationElement objects,

62

Writing a UI Automation Provider for a Win32-based Custom Control

IFACEMETHODIMP MyCustomControlProvider :: get_ProviderOptions(ProviderOptions * pRetVal) { *pRetVal = ProviderOptions_ServerSideProvider | ProviderOptions_UseComThreading; return S_OK; }

This method tells UI Automation that the class is a server-side provider rather than a client-side provider. In general, UI Automation uses clientside providers to support built-in controls such as Win32 buttons, while server-side providers expose information about custom controls to UI Automation. The second provider options flag tells UI Automation to use COM’s threading rules when it makes calls using the interface: incoming calls must come in on the same thread on which you handed out this object, which saves you the trouble of having to do synchronization ourselves. This assumes that CoInitialize was called on the main UI thread during application startup so that the main UI thread is Apartment-threaded.

www.code-magazine.com

Listing 1: This example is an initial implementation of a provider class, MyCustomControlProvider. #include

// Standard implementations of AddRef, Release, // QueryInterface go here

// New class that exposes a custom control to UI Automation. // This implements the provider-side interfaces. class MyCustomControlProvider: public IRawElementProviderSimple { HWND _hwnd;

// IRawElementProviderSimple methods IFACEMETHODIMP get_ProviderOptions(ProviderOptions * pRetVal); IFACEMETHODIMP GetPatternProvider(PATTERNID idPattern, IUnknown ** pRetVal ); IFACEMETHODIMP GetPropertyValue(PROPERTYID idProp, VARIANT * pRetVal ); IFACEMETHODIMP get_HostRawElementProvider( IRawElementProviderSimple ** pRetVal );

// Reference to control data or object model goes here public: }; MyCustomControlProvider(HWND hwnd): _hwnd(hwnd) { }

Listing 2: The WM_GETOBJECT message ensures UI Automation can access an instance of your provider. LRESULT MyCustomControlWndProc(HWND hwnd, UINT uMsg, WPARAM wParam, LPARAM lParam) { switch(uMsg) { case WM_GETOBJECT: { MyCustomControlProvider * pProvider = new MyCustomControlProvider(); LRESULT lres = UiaReturnRawElement(hwnd, wParam,

Patterns are interfaces that UI Automation uses to expose control-specific functionality such as selection or command invocation. I’ll revisit them later, so I’ll set the return value to NULL here: IFACEMETHODIMP MyCustomControlProvider:: GetPatternProvider(PATTERNID idPattern, IUnknown ** pRetVal ) { *pRetVal = NULL; return S_OK; }

The GetPropertyValue method handles element properties. UI Automation divides properties into two categories: those related to a specific area of functionality—for example, those dealing with selection, which only apply to controls that manage selection—and those that broadly apply to any element, including Name, Enabled, and Control Type. This latter category of property is handled in the GetPropertyValue method. For now, I’m supplying suitable values for the element’s Name and Control Type properties. These are two of the more important properties, as screen readers typically read the name and type of a control as focus changes, but I can support any number of properties later:

www.code-magazine.com

lParam, pProvider); pProvider->Release(); return lres; } } return DefWindowProc(hwnd, uMsg, wParam, lParam); }

IFACEMETHODIMP MyCustomControlProvider:: GetPropertyValue(PROPERTYID idProp, VARIANT * pRetVal ) { pRetVal->vt = VT_EMPTY; if(idProp == UIA_NamePropertyId) { pRetVal->bstrVal = SysAllocString(L“MyControlName“); pRetVal->vt = VT_BSTR; } else if(idProp == UIA_ControlTypePropertyId) { pRetVal->lVal = UIA_ButtonControlTypeId; pRetVal->vt = VT_I4; } return S_OK; }

Finally, get_HostRawElementProvider tells UI Automation which HWND this control is associated with: IFACEMETHODIMP get_HostRawElementProvider( IRawElementProviderSimple ** pRetVal ) { return UiaHostProviderFromHwnd(_hwnd, pRetVal); }

Once you’ve implemented the class, you have to ensure that UI Automation can actually access an

Writing a UI Automation Provider for a Win32-based Custom Control

63

Listing 3: One technique for handling a provider’s lifetime is to create and update a state flag. class MyCustomControlProvider: public IRawElementProviderSimple { HWND _hwnd; // Create a pointer to internal control state, set to // non-NULL in the constructor. // OnUIDisposed sets it to NULL to indicate that UI is no // longer present. Using NULL to indicate this avoids having // a separate flag. // Incoming UI Automation interface methods must check this // before use. // IMyInternalControlInterface _pControl;

IFACEMETHODIMP MyCustomControlProvider::GetPropertyValue( PROPERTYID idProp, VARIANT * pRetVal ) { // Always clear out parameter before anything else pRetVal->vt = VT_EMPTY; // Now I check to see if the UI is still present if(_pControl == NULL) { return UIA_E_ELEMENTNOTAVAILABLE; } // UI is still present, so code continues as before if(idProp == UIA_NamePropertyId) { ... // get name from _pControl and return via pRetVal. }

... // Called when corresponding UI is deleted void OnUIDisposed(); };

...

void MyCustomControlProvider::OnUIDisposed() { _pControl = NULL; }

instance of it. This is done by handling the WM_ GETOBJECT message in the window’s Window Procedure, as shown in Listing 2.

JECT message to see if the HWND supports UI Automation, and uses the returned provider to obtain more properties.

In response to WM_GETOBJECT, I create and return an instance of the provider to UI Automation via the UiaReturnRawElement API from UIAutomationCore.dll. Standard COM reference counting rules apply: UiaReturnRawElement calls AddRef() internally to claim its own reference; so I call Release() when I’m done with the WndProc’s reference.

All this happens only on demand: it’s only when a client requests an object for this specific HWND that the WM_GETOBJECT message is sent. GetPropertyValue is called only when a client requests a property.

Also, the code here creates a new instance of the provider each time: this is allowable, as is creating a single instance the first time and caching that value for later reuse. Just remember to use the Release() method on that reference in the WM_DESTROY handler. Note: Throughout this article, I refer to the Inspect Objects tool, which is available in the Active Accessibility 2.0 SDK Tools download on the Microsoft Download Center (http://www.microsoft.com/ downloads/details.aspx?familyid=3755582a-a707460a-bf21-1373316e13f0&displaylang=en). This tool helps you see what UI Automation sees. You can use it to test and verify your provider implementations. If I point the Inspect Objects tool at the custom control, it should report the Name and Control Type that I’ve specified above. What’s happening here is that Inspect Objects asks UI Automation for the object at a specified point on the screen. UI Automation determines the HWND, sends a WM_GETOB-

64

Writing a UI Automation Provider for a Win32-based Custom Control

Connecting a Provider to Real Control Data To make this example more realistic, I’m going to modify the provider code to contain a reference to the custom control’s own data so that it can return real data from GetPropertyValue. Depending on how your control works, this might involve a public or internal COM object, window messages, or direct access to the control’s data. An important issue to address here is that of provider lifetime. UI Automation holds onto a reference to the control on behalf of clients, and clients are not required to release their references at any particular point in time. This means that a provider can outlive its corresponding UI. One way to deal with this situation is to add an “alive” flag to the provider and set it to a “dead” state when the UI is no longer present. Further incoming calls to the IRawElementProvider can then check this flag and return the defined value UIA_E_ELEMENTNOTAVAILABLE.

www.code-magazine.com

Listing 3 shows the new code that takes provider lifetime into account. Now if you point the Inspect Objects tool at the control, you should see the control’s actual name instead of the string.

Adding Structure by Adding IRawElementProviderFragment So far, I’ve simply exposed a single element. However, most custom controls have some internal structure, whether items in a ListBox or a tree of elements in a HTML-like control. The next step is to expose that structure to UI Automation by adding the IRawElementProviderFragment interface. You need to consider what sort of class structure to use to expose your control’s internal structure. If you have a two-level homogenous container— for example, a list that contains only one type of list item—then it may make sense to have one class for the overall parent list and another class for the items. On the other hand, if the control has a more generalized object tree—as is the case with HTML— then it may make more sense to have a hierarchy of provider objects that mirrors the UI’s own class structure. It could have a base class that implements common functionality and derived classes that handle element-specific behavior. Whatever structure you choose, you need to implement both IRawElementProviderSimple and IRawElementProviderFragment interfaces for each class that represents an element in the structure. As you’ll see later, the root element also needs to implement IRawElementProviderFragmentRoot. You only need to access the tree structure when a client requests it. Furthermore, you don’t expose a tree structure all at once; instead, you access or traverse it element by element as needed. Because internal structure is very control-specific, I won’t provide concrete examples in this walkthrough. I’ll just outline what each method does. Note that, as with IRawElementProviderSimple, all the methods should check that the corresponding UI element is “alive” first before they go on to do the real work. The Navigate method is perhaps the most obvious method in this interface. It’s called with a specified direction (Parent, Next or Previous sibling, or First or Last child), and returns an instance of IRawElementProviderFragment that represents the element in the specified direction. This object must also implement IRawElementProviderSimple or else UI Automation would be able to navigate to it but not get any information about it: IFACEMETHODIMP MyCustomControlProvider:: Navigate(NavigateDirection direction, IRawElementProviderFragment ** pRetVal );

www.code-magazine.com

The returned IRawElementProviderFragment instance can be a new instance of a provider or a cached instance, so long as it obeys COM’s reference counting rules. Note that the object that this is called on continues to point to the same UI element that it was previously pointing to. The tree structure that is returned by this method should be consistent: if you can navigate to a child, then when you navigate back to the parent, you should end up at a provider instance that represents the same UI element. If you cache providers, this may be the same provider object instance. The root provider of your structure, the one that occupies the entire HWND, must return NULL when asked for its parent or siblings. You should provide just the subtree structure that corresponds to the UI that your control is providing; UI Automation will do the work of integrating this into the larger treeof-peer-HWNDs. Next, for the GetRuntimeId method, UI Automation requires that you return an integer array that uniquely identifies this specific element within the scope of the owning HWND: IFACEMETHODIMP MyCustomControlProvider:: GetRuntimeId (SAFEARRAY ** pRetVal);

If you already have a unique integer assigned to each UI element, using that would work perfectly here as a single element array. Be cautious about using a pointer value because you have to take into account that pointers on 64-bit systems require two integers. Once you have your array, prepend to it the value UiaAppendRuntimeId and use the API’s SafeArrayCreateVector and SafeArrayPutElement functions to convert the C-style array to a SAFEARRAY.

Note on Microsoft® Active Accessibility Compatibility UI Automation automatically makes some of the structure, properties and events of your custom control available to Microsoft Active Accessibility clients. The information UI Automation translates is the subset of data that Microsoft Active Accessibility supports. WM_GETOBJECT requests both Microsoft Active Accessibility support and UI Automation support from a HWND, with the lParam indicating which type the sender is looking for. When you pass the provider to UiaReturnRawElement, it checks the lParam and wraps the provider in a UIA-to-MSAA adaptor, which it then hands back to the client. This UIA-to-MSAA translation wrapper also implements the IAccessibleEx interface, which allows Microsoft Active Accessibility clients to query for the UI Automation properties and patterns that are outside the scope of the Microsoft Active Accessibility interfaces.

The root element of your control can simply return NULL here: UI Automation automatically constructs Runtime IDs for the host HWND, and from UI Automation’s point of view, the root element of the control is essentially the same element as that HWND. The get_BoundingRectangle method returns the bounding rectangle coordinates of the control. For this, return the location of the UI element in screen coordinates. If the element is not visible, return a rectangle consisting of all 0s. You may need to use MapWindowPoints to convert from HWND client-relative coordinates to screen coordinates: IFACEMETHODIMP MyCustomControlProvider:: get_BoundingRectangle(UiaRect * pRetVal);

As with GetRuntimeId, the root element doesn’t have to do anything here, since its location is the same as the host HWND, and UI Automation can already get that information from the HWND.

Writing a UI Automation Provider for a Win32-based Custom Control

65

Note on Client-side Providers UI Automation also supports client-side providers, which are loaded by the client application and often provide the default UI Automation support for many of the standard Win32 controls. Server-side and client-side providers are very similar in terms of general class structure: they implement the same set of IRawElement* interfaces and expose properties and patterns in the same way. But there are differences. While server-side providers live in the same process as the control itself; client-side providers live in the client application’s address space. Server-side controls can obtain information from the underlying control by accessing the control’s internal data if necessary. Client-side controls, however, must rely on APIs or window messages.

You only use the GetEmbeddedFragmentRoots method when your control hosts other HWNDs within it. Since I’m not covering this issue yet, this method returns NULL here. Again, the root element doesn’t have to do anything because its location is the same as the host HWND: IFACEMETHODIMP MyCustomControlProvider:: GetEmbeddedFragmentRoots(SAFEARRAY** pRetVal);

UI Automation calls the SetFocus method when it wants you to set your control’s internal focus state to a particular element. UI Automation automatically focuses the parent HWND first, so you only need to update your control’s internal state: IFACEMETHODIMP MyCustomControlProvider:: SetFocus();

The get_FragmentRoot method provides UI Automation with a convenient way to get to the root element of your control without having to call Navigate(Parent) repeatedly: IFACEMETHODIMP MyCustomControlProvider:: get_FragmentRoot( IRawElementProviderFragmentRoot ** pRetVal);

Next, you need to implement two structure-related methods only on the root node. These are on the IRawElementProviderFragmentRoot and allow UI Automation to request the focused element or the element at a specific screen location:

Listing 4: This method returns a pattern provider based on the patterns the control implements. IFACEMETHODIMP MyCustomControlProvider::GetPatternProvider( PATTERNID idPattern, IUnknown ** pRetVal ) { // Always start by setting output parameter to NULL *pRetVal = NULL; // Check that control is still alive if(_pControl == NULL) { return UIA_E_ELEMENTNOTAVAILABLE; } // Now check if it‘s a pattern that this element supports if(idPattern == UIA_ExpandCollapsePattern && _pControl->CanExpand()) { // Assuming here that this class also implements // IExpandCollapseProvider *pRetVal = this; AddRef(); } return S_OK; }

66

Writing a UI Automation Provider for a Win32-based Custom Control

IFACEMETHODIMP MyCustomControlProvider:: ElementProviderFromPoint(double x, double y, IRawElementProviderFragment ** pRetVal); IFACEMETHODIMP MyCustomControlProvider:: GetFocus( IRawElementProviderFragment ** pRetVal);

These methods should return a provider instance representing the focused element or the element at the specified point (in screen coordinates), or NULL if the point or focus is on the root element itself or if the point or focus is not on the element at all. For both these methods, the provider should return the deepest element possible. Finally, I’ll return to the implementation of IRawElementProviderSimple::get_HostRawElementProvider. You should modify this such that it only returns the HWND for the root. For all child elements within the control, it should return NULL. You can take a piecemeal strategy for implementing all of these methods. First, start with implementing the Navigate and BoundingRectangle methods. This enables you to get to the root element by hovering the cursor over any part of the control, and then walking through the tree using the Navigation functionality of the Inspect Objects tool. Use the highlight rectangle feature to quickly verify that the locations are being exposed correctly as you navigate. Then you can implement ElementProviderFromPoint. When you hover the mouse over an element within the control, Inspect Objects should go straight to that element instead of stopping at the HWND. Finally, implement the GetRuntimeId and GetFocus methods.

Deciding What Part of the Tree Structure to Expose When deciding what parts of the tree structure to expose, remember that UI Automation is only interested in elements that are important to an end user. This includes any element that an end user would perceive as being a separate control in its own right. For example, a push-button and its text are typically not considered distinct elements from an end-user’s point of view, even if they are represented as such in a control’s internal structures. In such a case, you should expose just a single element representing the entire button.

Exposing Control-specific Functionality Using Patterns The next step in implementing a provider is to customize the provider with information specific to the corresponding UI. I’ve already covered properties (such as Name and Control Type), so it’s time to turn to control patterns. A pattern is just an interface that contains both properties and meth-

www.code-magazine.com

ods related to a common theme. For example, the SelectionItem pattern interface has properties like get_IsSelected that return the current selected state, and methods like Select that change the selection. Implementing a pattern is a matter of figuring out which interfaces to implement, and then implementing the interface in terms of the underlying control data. The mechanics of exposing the interface are similar to using QueryInterface, but instead using the method IRawElementProviderSimple::GetPat ternProvider. This method takes a parameter indicating the pattern that is requested and returns an IUnknown object that implements the corresponding interface. While it is often most convenient to implement the pattern interface on the same provider object, this is not required. You can implement the interface on a separate object if desired, though this separate object will need to adhere to the same warnings regarding lifetime that the original provider does. Listing 4 shows a simple implementation of GetPatternProvider which implements an interface on the same object as the provider. It isn’t necessary to cast the returned object to the interface corresponding to the requested pattern: UI Automation calls QueryInterface on the returned object as a separate step after this to get the correct interface. Ensure that manipulating the control using these interfaces is consistent with manipulating the control using the mouse or keyboard. For example, if selecting some element with a mouse or keyboard causes some other element to change state, that same change should occur when the element is selected using the ISelectionItemProvider::Select method.

[Name and Control Type] are two of the more important properties, as screen readers typically read the name and type of a control as focus changes…

Notifying UI Automation of UI Events The last piece of UI Automation functionality that you need to address is events. When control state changes, you need to tell UI Automation so that it can inform any interested clients. For example, if the focus changes from one element to another, UI Automation can inform a screen reader, which can in turn query for the name of the element with focus.

www.code-magazine.com

There are three main categories of events in UI Automation: • Events associated with a property, such as changing the Name or IsEnabled values. • Events associated with an action, such as the pressing of a button. • Events associated with the creation of deletion of elements, called structure change events. The general technique for notifying UI Automation of an event is the same in all three cases: the control creates an instance of a provider for the UI element that corresponds to the event, and then calls one of the UI Automation “Raise” APIs, passing in the ID of the event or property as appropriate. For example, to signal focus change, a control would use code similar to the following, which assumes that pFocusedItem is a reference to an item that has just received focus: // Use UiaClientsAreListening to avoid creating a // provider if no clients are listening if(UiaClientsAreListening()) { IRawElementProviderSimple pProvider = new MyCustomControlProvider(hwnd, pFocusedItem); UiaRaiseAutomationEvent(pProvider, UIA_FocusChangedEventId); pProvider->Release(); }

If clients are listening, the UiaRaiseAutomationEvent call will create a reference to the provider that remains in place until the client is done with the element. Before the Inspect Objects tool can track focus in the control, I need to implement the HasKeyboardFocus property to return a VT_BOOL variant with value VARIANT_TRUE when the element has focus. When that’s done, Inspect Objects should be able to track focus as you move keyboard focus around within your control using tab or arrow keys. NOTE: The UiaClientAreListening method returns TRUE if a client is listening to any event from this process. You can get more fine-grained information by implementing the optional IRawElementProviderAdviseEvents interface on the root element of your control. UI Automation will then call the AdviseEventAdded and AdviseEventRemoved methods on this interface as clients register and unregister for specific events.

Deciding which Properties, Patterns, and Events to Implement So far, I’ve discussed the mechanics of exposing the structure of a control and of exposing the prop-

Writing a UI Automation Provider for a Win32-based Custom Control

67

Property

Description

Name

Identifies the control. This is often the text on the control or the label. The name shouldn’t contain the type of the control. For example, “OK” rather than “OK button.”

ControlType

Specifies one of the defined UIA_XxxxControlTypeId values from UIAutomationClient.h, like UIA_ButtonControlTypeId. While ControlType is effectively an integer enumeration value, LocalizedControlType is a human-readable string. UI Automation automatically supplies the LocalizedControlType based on the ControlType that you supply, so you don’t need to implement the LocalizedControlType property.

IsEnabled

Identifies whether the element is enabled or disabled.

HasKeyboardFocus

Identifies whether the element has keyboard focus.

IsKeyboardFocusable

Identifies whether the element can have keyboard focus.

AcceleratorKey and AccessKey

Identifies the accelerator key or access key for the control.

Table 1: Your custom control should support these properties for each of its elements.

erties, patterns, and events for elements of the control. But how do you decide what to expose? Much of that depends on the purpose and functionality of the control. However, there are a few properties that almost all elements should expose, as listed in Table 1. UI Automation provides some other properties automatically, such as ProcessId and NativeWindowHandle. Even though these are defined in the header file, you don’t need to implement support for them.

When deciding what parts of the tree structure to expose, remember that UI Automation is only interested in elements that are important to an end user.

In determining which other properties and patterns to support, a useful guide is the control type: many control types have a set of associated properties and patterns that are expected for controls of that type. For example, buttons are expected to implement the Invoke pattern, and combos and lists are expected to implement the Selection pattern. You can find further information on these properties and patterns, as well as the entire Windows Automation 3.0 API, in the upcoming Windows® Software Development Kit for Windows 7. Brendan McKeon

68

Writing a UI Automation Provider for a Win32-based Custom Control

www.code-magazine.com

Have You Xiine It?

All You Can Read. Digitally. Experience the next generation in digital reading FREE. Go to www.xiine.com to read CoDe Magazine and other great publications.

ONLINE QUICK ID 0811122

Creating UI Automation Client Applications

Matthew Karr Software Development Engineer

Sometimes an application needs to interact with the user interface (UI) of a second application. The first application might be a test application that drives the UI of the target to run through some automated tests. It might describe the UI out loud, as an aid to users that are blind. It might be a speech application that allows users to give vocal commands. In each of these cases, the application needs a way to inspect and interact with the UI of the system and other running applications.

Matthew Karr has been a developer at Microsoft for 5 years, working on Automation and Accessibility. He’s helped ship UI Automation V1 and the Magnification API with Vista, and is currently working on the COM UI Automation API for Windows 7.

icrosoft® UI Automation is a tool for doing just that. It provides an abstracted model of the UI, and allows a client application to both investigate and manipulate the UI of all running applications on the system. In this article, I explain how you can use UI Automation for automated testing, and I walk through code samples that demonstrate the basic features of UI Automation.

In his free time he enjoys juggling, snowboarding, science fiction, and video games.

Using UI Automation for Automated Testing

M

The goal of many automated test tools and scenarios is the consistent and repeatable manipulation of the user interface. This manipulation can involve simple unit testing of specific controls, or complex recording and playback of test scripts that iterate through a series of generic actions on a group of controls. One complication that arises from automated applications is the difficulty of synchronizing a test with a dynamic target. As an example, consider a list box control—such as one contained in the Windows Task Manager—that displays a list of currently running applications. Since the items in the list box are dynamically updated outside the control of the test application, consistently repeating the selection of a specific item in the list box is impossible. Similar problems arise when attempting to repeat simple focus changes in a UI that is outside the control of the test application.

Creating UI Automation Client Applications

• The UI Automation tree facilitates navigation through the structure of the UI. The tree is built from the collection of window handles (HWNDs). • Automation elements are individual components in the UI. These can often be more granular than a window handle. • Automation properties provide specific information about UI elements. • Control patterns define a particular aspect of a control’s functionality. They can consist of information about properties, methods, events, and structures. • Automation events provide event notifications and information.

Basics of a UI Automation Client Application

Fast Facts Microsoft UI Automation was available for both managed and unmanaged applications from the beginning; however, the framework interface was limited for managed API for its client applications. Now with new introduction of COM based unmanaged client API, the UI Automation clients can also be managed or unmanaged.

Programmatic access provides the ability to imitate, through code, any interaction and experience exposed by traditional mouse and keyboard input. UI

70

Automation enables programmatic access through five components:

For the Windows® 7 operating system, UI Automation has a new COM client API for both unmanaged and managed clients. Unmanaged applications can now use UI Automation without requiring a change in languages or loading the common language runtime (CLR). Perhaps the best way to understand UI Automation in general, and the COM client specifically, is to see them in action. Listing 1 demonstrates a complete program that gets the UI element at the current cursor position, and then prints its name and control type.

Next I’ll walk through the various pieces that make up this example.

www.code-magazine.com

Listing 1: The program gets the UI element at the current cursor position and prints its name and control type. int _tmain(int argc, _TCHAR* argv[]) { // Initialize COM and create the main Automation object IUIAutomation *g_pAutomation; CoInitialize(NULL); HRESULT hr = CoCreateInstance(__uuidof(CUIAutomation), NULL, CLSCTX_INPROC_SERVER, __uuidof(IUIAutomation), (void**)&g_pAutomation); if(FAILED(hr)) return (hr);

if(SUCCEEDED(hr)) { wprintf(L“Element‘s Name: %s \n“, name); SysFreeString(name); } // Get the element‘s Control Type (in the current languange) // and print it BSTR controlType; hr = pAtMouse->get_CurrentLocalizedControlType(&controlType); if(SUCCEEDED(hr)) { wprintf(L“Element‘s Control Type: %s \n“, controlType); SysFreeString(controlType); }

// Get the element under the cursor // Use GetPhysicalCursorPos to interact properly with // High DPI POINT pt; GetPhysicalCursorPos(&pt);

// Clean up our COM pointers pAtMouse->Release(); g_pAutomation->Release(); CoUninitialize(); return 0;

IUIAutomationElement *pAtMouse; hr = g_pAutomation->ElementFromPoint(pt, &pAtMouse); if(FAILED(hr)) return hr; } // Get the element‘s name and print it BSTR name; hr = pAtMouse->get_CurrentName(&name);

Creating the CUIAutomation Object

Directly Obtaining UI Automation Elements

The heart of the new COM API is the IUIAutomation interface, which lets clients get automation elements, register event handlers, create various helper objects, and access other helper methods.

Once you have the UI Automation object, you can discover the entire UI. The UI is modeled as a tree of automation elements (IUIAutomationElement interface objects), where each element represents a single piece of UI. The IUIAutomationElement interface has methods relevant to all controls, such as checking properties or setting focus.

To get started using UI Automation in your application, do the following: 1. Include Uiautomation.h in your project headers. This header will bring in the other headers that define the API. 2. Declare a global pointer to an IUIAutomation interface. 3. Initialize COM. 4. Create an instance of the CUIAutomation class and retrieve the IUIAutomation interface in your global pointer. The following example function creates the object instance and stores the retrieved interface address in the global pointer g_pAutomation: HRESULT InitializeUIAutomation() { CoInitialize(NULL); HRESULT hr = CoCreateInstance(__uuidof(CUIAutomation), NULL, CLSCTX_INPROC_SERVER, __uuidof(IUIAutomation), (void**)&g_pAutomation); return (hr); }

www.code-magazine.com

The root element of the UI Automation tree is the desktop. You can obtain this element by calling either the IUIAutomation::GetRootElement or IUIAutom ation::GetRootElementBuildCache method. Each method retrieves an IUIAutomationElement interface pointer, from which you can search or navigate the rest of tree, as described later in this article. If you have screen coordinates—such as the cursor position in this example—you can retrieve an IUIAutomationElement interface by calling the IUIAutomation::ElementFromPoint method. To retrieve an IUIAutomationElement interface from a window handle (HWND), call the IUIAuto mation::ElementFromHandle method. You can retrieve an IUIAutomationElement interface that represents the focused control by calling the IUIAutomation::GetFocusedElement method.

UI Automation Properties for Clients Properties of IUIAutomationElement objects contain information about UI elements, usually controls.

UI Automation and Threading Because of the way UI Automation uses Windows messages, conflicts can occur when a client application attempts to interact with its own UI on the UI thread. These conflicts can lead to very slow performance or even cause the application to stop responding. If you intend your client application to interact with all elements on the desktop— including its own UI—you should make all UI Automation calls on a separate thread. This recommendation also applies when your application locates elements and uses control patterns. It is safe to make UI Automation calls within a UI Automation event handler because the event handler is never called on a UI thread. However, when subscribing to events that may originate from your client application’s UI, you must make the call on a non-UI thread. Remove event handlers on the same thread.

Creating UI Automation Client Applications

71

Listing 2: This method uses the RangeValue pattern to set a control to its maximum value. HRESULT TurnItUp(IUIAutomationElement *pElement) { IUIAutomationRangeValuePattern * pRangeVal;

double max; hr = pRangeVal->get_CurrentMaximum(&max);

HRESULT hr = pElement->GetCurrentPatternAs(UIA_RangeValuePatternId, __uuidof(IUIAutomationRangeValuePattern), (void **)&pRangeVal);

if(SUCCEEDED(hr)) { hr = pRangeVal->SetValue(max); }

if(FAILED(hr) || pRangeVal == NULL) { return hr; }

pRangeVal->Release(); return hr; }

The properties of an element are generic to all elements and not specific to a control type. Control patterns, discussed later, expose control-specific properties. UI Automation properties are read-only. To set properties of a control, you must use the methods of the appropriate control pattern; for example, use the IScrollProvider::Scroll method to change the position values of a scrolling window.

Programmatic access provides the ability to imitate, through code, any interaction and experience exposed by traditional mouse and keyboard input.

To improve performance, you can cache property values of controls and control patterns when you retrieve elements. Some generic properties, and all control pattern properties, are available as properties on the IUIAutomationElement interface or control pattern interface, and you can retrieve them with accessor methods.

When using the generic accessors, you must specify properties with the property identifiers defined in UIAutomationClient.h. You use them to specify properties when retrieving property values, constructing property conditions, and subscribing to property-changed events. If a UI Automation provider does not implement a property, UI Automation is able to supply a default value for that property. For example, if the provider does not support the property identified by UIA_IsDockPatternAvailablePropertyId, UI Automation returns FALSE.

Control Patterns Control patterns complement properties. Control patterns are collections of associated properties, events, and methods that describe an element. More than one pattern can apply to a single element. A control pattern represents a collected group of capabilities. The pattern might be something

72

Creating UI Automation Client Applications

simple like the Invoke pattern, which lets clients invoke a control. In contrast, the more complicated Value pattern supports getting and setting a control’s value, and then checking whether the value is read-only. You can obtain a control pattern by calling the IUIAutomationElement::GetCurrentPattern or IUIAutomationElement::GetCurrentPatternAs method, or their cached versions. Once you get a control pattern interface, use it as you would the element itself, either by directly calling control pattern methods or by accessing control pattern properties. The TurnItUp method in Listing 2 uses the RangeValue pattern to set a control (such as a volume slider) to its maximum value. In addition to checking for errors when getting the current pattern, this example also checks for NULL because NULL is a valid return value when a control does not support the pattern requested.

Control Types While a control type is mechanically a simple enumeration property, on a conceptual level it is much more important. An element’s control type is a broad classification of the control, such as button, window, or list box. Each control type has certain expected control patterns. For example, a button control should support either the Invoke pattern or the Toggle pattern, and a hyperlink should support the Invoke Pattern and possibly the Value pattern. Some controls have conditional support for several control patterns. For example, the menu item control has conditional support for the Invoke, Expand Collapse, Toggle, and SelectionItem control patterns, depending on the menu item control’s function within the menu control.

Searching and Navigation There are two major ways of getting to elements. The first is to use a Find method, which lets the UI

www.code-magazine.com

Listing 3: The FindFirst method here finds a particular application window. IUIAutomationElement* GetTopLevelWindowByName(LPWSTR windowName) { if (!windowName) { return NULL; } IUIAutomationElement* pRoot; IUIAutomationElement* pFound; VARIANT varProp; varProp.vt = VT_BSTR; varProp.bstrVal = SysAllocString(windowName); // Get the desktop element HRESULT hr = g_pAutomation->GetRootElement(&pRoot);

Automation core minimize cross-process calls and improve searching time. The second is to use a tree walker to walk a UI tree.

Using Find Methods The Find methods require three things: a parent element, a scope to search on, and (most importantly) a condition to search for. You create conditions by calling various APIs on the CUIAutomation object. You can create simple Property conditions, or combine them with And, Or, and Not operators to produce more complex conditions. Once you create a condition, you need to specify the element to start searching from and the search scope. The element you want to start from is the element on which you call the FindFirst or FindAll method. The search scope is typically one of three choices: the element itself, its children, or its descendants.

If a UI Automation provider does not implement a property, UI Automation is able to supply a default value for that property.

// Get a top-level element by name, such as „Program Manager“ if (pRoot) { IUIAutomationCondition* pCondition; g_pAutomation->CreatePropertyCondition(UIA_ NamePropertyId, varProp, &pCondition); pRoot->FindFirst(TreeScope_Children, pCondition, &pFound); pRoot->Release(); pCondition->Release(); } VariantClear(&varProp); return pFound; }

ticular name) is quite simple. However, conditions can be considerably more complex.

Using Tree Walkers Another method for navigating through the tree is the use of tree walkers. Tree walkers allow use of directnavigation methods such as moving between parent, child, and sibling to walk through a filtered view of the tree. There are several built-in filtered views: • Raw or unfiltered, which shows all elements. • Control view (the default), which filters out elements that either are redundant or are just used for layout. • Content view, which filters controls even more selectively than Control view does. Users can also create their own custom views with Conditions. The following simple example shows how to walk to the first child of the control referenced by pElement: // Get the control view walker IUIAutomationTreeWalker * pWalk; g_pAutomation->get_ControlViewWalker(&pWalk); // Go to the element‘s first child IUIAutomationElement * pFirst; pWalk->GetFirstChildElement(pElement, &pFirst);

UI Automation Events for Clients You usually want to limit search scope as much as possible to improve performance and to reduce the chance of finding an element you weren’t looking for. Listing 3 is a simple example of searching for a named application window using FindFirst methods. In the example, the search condition (a par-

www.code-magazine.com

UI Automation lets clients subscribe to events of interest. This capability improves performance by eliminating the need to continually poll all UI elements in the system to see if any information, structure, or state has changed. Efficiency is also improved by the ability to listen for events only within a defined scope. For example,

Creating UI Automation Client Applications

73

a client can listen for focus change events on all UI Automation elements in the tree, or on just one element and its descendants. Client applications subscribe to events of a particular kind by using methods such as AddAutomationEventHandler or AddFocusChangedEventHandler to register an event handler These methods take a callback interface, which should be implemented in an object created by the client. A method such as IUIAutomationEventHandler::HandleEvent will then be called on the appropriate thread whenever the event occurs. On shutdown, or when UI Automation events are no longer of interest to the application, UI Automation clients should remove the events. They do so either by calling a removal method for specific events such as RemoveAutomationEventHandler or RemoveFocusChangedEventHandler, or by calling the RemoveAllEventHandlers method to remove all events.

UI Automation and Screen Scaling Starting with Windows Vista®, users can change the screen resolution (in dots per inch or dpi) so that most UI elements appear larger. Previously, the scaling had to be implemented by applications. Now, the Desktop Window Manager performs default scaling for all applications that do not handle their own scaling. UI Automation client applications must take this feature into account. The UI Automation API does not use logical coordinates. Methods and properties either return physical coordinates or take them as parameters. By default, UI Automation applications that run at a screen resolution other than 96 dpi will get incorrect results from these methods and properties. For example, because the cursor position is in logical coordinates, the client cannot simply pass these coordinates to the IUIAutomation::ElementFromPoint method to get the element that is under the cursor. In addition, the application will not be able to correctly place windows outside its client area. The solution to this situation has two parts: • Make your client application aware of screen resolution by calling the Win32 function SetProcessDPIAware at startup. This function makes the entire process aware of screen resolution, so that no windows belonging to the process are scaled. • Call GetPhysicalCursorPos to get the current cursor coordinates.

The properties of an element are generic to all elements and not specific to a control type. Control patterns, discussed later, expose control-specific properties. an administrative application the operating system prompts the user for consent to run the application with elevated user rights. Similarly, when a user attempts to perform a task that requires administrative user rights, Windows presents a dialog box that asks the user for consent to continue. This dialog box is protected from cross-process communication, so that malicious software cannot simulate user input. Only signed client applications are trusted to communicate with applications running at a higher user-rights level. To gain access to the protected system UI, applications must be built with a manifest file that includes a special attribute. This UIAccess attribute is included in the <requestedExecutionLevel> tag where the value of the level attribute is an example only, as follows: <trustInfo xmlns= „urn:0073chemas-microsoft-com:asm.v3“> <security> <requestedPrivileges> <requestedExecutionLevel level=“highestAvailable“ UIAccess=“true“ />

The UIAccess attribute value is “false” by default; that is, if the attribute is omitted, or if there is no manifest for the assembly, the application will not be able to gain access to protected UI. With access to the system and target application’s UI, your client applications can navigate the UI, retrieve properties and control patterns for UI elements, and listen to relevant UI events. Matthew Karr

User Account Control and User Rights Finally, you need to consider User Account Control and user rights to ensure your client can access the information it needs. Before a user launches

74

Creating UI Automation Client Applications

www.code-magazine.com