Evaluation Protocols and Testing Tools
From ATHEN
The ATHEN "AT Initiative" sub-group has been working to identify, refine, or create a set of protocols for evaluating and testing information technology products for accessibility. Currently, this page serves as the coffee table upon which all our ideas, examples, and resources are spread. Ultimately our hope is that all this will evolve into a final product.
Contents |
The Goal
Develop a product (tools/best practices/guidelines/protocol) that
- Helps vendors integrate accessibility in the design process.
- Helps purchasers/users evaluate accessibility of products under consideration.
The product should be:
- generic enough that it can be applied to software applications and individual tools used for communication, collaboration, assessment, authoring and delivery of instruction.
- easy to understand, and not so dense nor technical as to make it difficult to use and appply.
- Informative for audience members who are not familiar with the user experience of individuals with disabilities (i.e., individuals with disabilities typically interact with information technology, with or without assistive technology)
Target Audience
The target audience for this product includes the following three groups:
- Product developers
- Product evaluators (purchasing agents)
- AT professionals/users
Next Steps
The first next step is to organize and annotate existing resources. This is a work in progress. Currently, items from the "Resources" section below are being moved to the pages linked from the "Existing Tools" section. A goal is ultimately to compare these resources and identify common themes and/or components.
Existing Tools
Central Repository/Accessibility Certification
In the UK, TechDIS is spearheading the effort to develop an Accessibility Passport, "a way of encouraging people who commission or design learning objects or software to take accessibility into account, and to give them feedback on the effectiveness and inclusivity of their materials."
In discussions regarding the feasibility of this approach, the following questions and ideas have emerged:
- Who is going to do the "certification" and based on what objective criteria?
- How do you do a comparative analysis across categories of products?
- How do you determine if the source of an accessibility problem is AT or IT?
The California State University is thinking about creating such a repository as a way to:
- assist campuses in understanding the accessibility and usability implications of adopting a particular academic technology resource
- allow the community to comment on the usefulness of the resource
Glossary
- Accessibility - The ability to perceive and interact with UI.
- Usability - The measure of how easily and quickly someone can learn and use the applications for the intended purpose.
Resources
- Top 100 Tools for Learning (in PDF, requires Adobe Acrobat Reader)
- Research on Web Accessibility in Higher Education - See Method section for a procedure for manually measuring "functional accessibility" of web pages
- DRAFT Web Accessibility Rubric - imperfect, but has been shown to have high inter-rater reliability.
- Objective evaluation of likely usability hazards â preliminaries for user testing by Jasper Sprengers
- Irish National Disability Authority IT Accessibility Guidelines Web and IT software guidelines.
Descriptions of high level accessibility goals and the difficulties faced by users Prioritised guidelines for each technology Motivation and justification for each guideline Guidance on design techniques and testing methods
- Guidelines to Standardisers of ICT products and services in the CEN ICT domain This document (dated Feb 2003) has a series of checklists at the end. They are in the form of a matrix listed by the impact on types of disabilities and the degree and relevance to each one as 'not significant', 'minor problems', 'moderate problems', 'severe problems'.
- Guidance on accessibility for human-computer interfaces ISO/TS 16071 "The purpose of this Technical Specification is to provide guidance to developers on designing human-system interactions to increase accessibility, promote increased effectiveness,efficiency and satisfaction for people who have a wide variety of capabilities and preferences. Accessibility is therefore strongly related to the concept of usabilitity (see ISO 9241-11)."
- PAS78 free to download A collection of links to guidelines and the British Publicly Available Specification (PAS 78:2006) 'Guide To Good Practice In Commissioning Accessible Website,' Go to the bottom of the page for the free download.
- Report on International ICT Accessibility Standards Proposed, Being Developed and Recently Published researched by Richard Hodgkinson FISTC, Tiresias.org 2nd October 2007. This report includes many of the international standards across all aspects of ICT.
Videos
Videos may be helpful in communicating to developers a better sense of individuals with disabilities' "user experiences". The following are examples, both from University of Illinois at Urbana/Champaign:
DISCLAIMER: Some of the videos show problems that have been resolved, so they should not necessarily be viewed as accurate of the current product, especially EBSCO.
