Downloadable Versions

NC State University Moodle 2.1.1 Accessibility Evaluation

Greg Kraus

greg_kraus@ncsu.edu

@gdkraus

November 29, 2011

Table of Contents

Executive Summary

NC State University currently uses Moodle 1.9.13 for course delivery and is assessing Moodle 2.1.1 to determine the best upgrade path. Part of this assessment is an accessibility evaluation. In the past NC State has made modifications to Moodle that were not part of the core code in order to make the environment more accessible. Moodle 2.0 and later versions introduce many changes that impact accessibility, some positively, some negatively. Some of the accessibility problems NC State had to address previously have now been resolved. These include the following.

The major problems in 2.1.1 include the following.

While there are many other problems outlined later in this report, these five problems represent some of the most critical issues that must be addressed before Moodle can be deployed to campus. Appendix B details and rates the importance of all of the accessibility problems in Moodle 2.1.1. The “High” priority items must be addressed before deploying Moodle and there must be a commitment to addressing all of the “Medium” and “Low” priority items as well.

Back to top

Overview

In the Fall of 2011 Greg Kraus, University IT Accessibility Coordinator, evaluated the accessibility of Moodle 2.1.1 in order to determine any modifications that would need to be made in order to deploy Moodle to NC State University. While the original scope of the project was to completely evaluate Moodle from both a student’s a teacher’s perspective, because of time constraints, only the student perspective was fully evaluated. However, many of the problems revealed from the student’s perspective also apply to the teacher’s interactions with the system.

The evaluation includes both functional and technical evaluations. A technical evaluation examines the code used in an application and the interactions between the application and the end user to ensure it is implemented according to standards like the Web Content Accessibility Guidelines. It determines if the application is behaving in ways that allow assistive technologies to effectively interact with the system. An example of this type of evaluation is, “Does a particular button in the user interface present itself in a way so that it can be understood by, and interacted using both visual and non-visual means?” The limitation of a technical evaluation is that it does not take a holistic view of asking if the individual pieces of an application work together correctly and if the end user can actually perform the necessary functions with it.

Alternatively, functional evaluations ask if the end user can accomplish the necessary tasks to effectively use an application, regardless of how a particular feature is coded. An example of this type of evaluation is, “Can the user post a discussion to the forum?” This question is then tested using various modes of interacting with the application, such as using a mouse, a keyboard, or an assistive technology. If the user can accomplish the task through all of those methods, the application is considered functionally accessible for that task.

Functional and technical evaluations are not mutually exclusive. In fact, they help inform each other and both aspects are necessary in developing an accessible application. The individual components of an application have to be implemented to meet a technical standard in order for the functional tasks to even be possible for users of assistive technologies. As the functional tasks are performed and evaluated, a skilled evaluator will be able to readily identify the technical issues behind problems that are discovered. The results that follow detail the problems that were uncovered by trying to accomplish certain functional tasks and also technical problems revealed in individual components of the system.

Throughout this report various versioning numbers will be referenced. They include

In addition to the evaluating 2.1.1, 2.1.2 (the latest release from Moodle) and 2.2 (which is still in development) were also referenced to determine if any significant change had been made.

The report details individual components of the system, which are divided into three areas: system-wide features, course-level features, and general comments. For features new to Moodle 2.x, any notable accessibility features are also reported in addition to accessibility problems.

Appendix B lists all of the noted problems, the importance of each problems, any suggested fixes, and a high-level assessment of how difficult each fix would be.

Back to top

System-Wide Features

The items in this section detail features found throughout the system. They can be present within courses and in other areas of the system.

Blocks

Blocks serve the same functions in 2.1 as they do in previous versions. The biggest difference is that blocks are now dockable on the side of the page in order to minimize them. Additionally there is a new Navigation block that allows users to navigate to any point within any of their courses, no matter where they are in the system. The Navigation block is detailed in the next section.

Accessibility Features

Accessibility Problems

Navigation Blocks

The Navigation block is a new feature in 2.x that allows users to jump to any point in any of their courses regardless of where they are in the system.

Accessibility Features

Accessibility Problems

Back to top

File Picker

The File Picker window allows users to upload files to Moodle. It is a critical component for instructors and students in order to work with Moodle. This system is completely redesigned from 1.9.13 and introduces a number of accessibility problems.

Accessibility Problems

Back to top

Forms

Forms are used throughout Moodle for interacting with the system. Most of the form elements and interactions are accessible, but there are some notable exceptions.

Accessibility Problems

Back to top

WYSISYG Editor

Moodle provides a WYSIWYG editor for entering text in the system. This is an optional choice for entering and editing text. Alternatively, through a setting in the user profile, users can choose to use plain text boxes to enter and edit text. Moodle uses TinyMCEas the editor which has a number of accessibility features. There is one problem that needs to be noted.

Accessibility Problems

Back to top

Help Windows

The help windows are implemented differently in 2.1 than they were in 1.9.x. They are now pop-up areas within the page rather than a new pop-up window.

Accessibility Problems

Back to top

“My Home” Page

This is an alternative main page for users where they can get a more detailed view of their courses and upcoming tasks.

Accessibility Problems

Back to top

View Profile Page

Accessibility Problems

Back to top

Calendar Page and Calendar Block

Accessibility Problems

Back to top

Forgotten Password Page

Accessibility Problems

Back to top

Recent Activity

This page is accessed through the “Full report” link in the Recent Activity block.

Accessibility Problems

Back to top

Language Change Jump Menu

Accessibility Problems

Back to top

Course Pages

The items in this section detail features present only at the course level.

Completion Tracking

Completion tracking is a new feature of 2.x. It allows users to see which activities they have completed and which they have not. Some items are marked as completed by fulfilling a set of criteria, and other items are marked as completed by the user.

Accessibility Problems

Back to top

Forum

The Forum activity is largely unchanged from 1.9.x to 2.1. Complex discussions, such as multi-threaded discussions where there are replies to replies, thus creating a whole “tree” of discussions can be quite difficult for screen reader users to navigate.

Accessibility Problems

Back to top

Quiz

The Quiz activity was changed extensively from 1.9.x to 2.x, including many accessibility enhancements, however, there are still several areas that need improvement.

Accessibility Problems

Back to top

Choice

The Choice activity was inaccessible in 1.9.x and is still inaccessible in 2.1.

Accessibility Problems

Back to top

Folder

The folder was redesigned from 1.9.x to 2.x. It used to be called “Display a directory.” The version in 2.1 is much less accessible than the 1.9.x version.

Accessibility Problems

Back to top

Chat

The standard Chat activity has numerous accessibility problems, which is acknowledged by Moodle by the fact that there is a more accessible version of the chat tool available to users. This assessment only details the more accessible version.

Accessibility Problems

Back to top

Lesson

Accessibility Problems

Back to top

Grades

Accessibility Problems

Back to top

Page (User Created Content)

Any accessibility problems with the Page would arise from use created content. Those issues can be read in the WYSIWYG section.

Back to top

Book

The Book is not an official part of the Moodle core but is a highly used tool at NC State so it is considered in this evaluation too.

Accessibility Problems

Back to top

Overview - Activity Report

These reports are viewable through the Navigation block, under the My Profile section. For some reason this link disappeared during testing, but the direct URL to the page would still work. It is not known why this changed, whether it was a bug or if some other event caused it.

Back to top

General Accessibility Problems

Back to top

Conclusion

While Moodle 2.1.1 has made some noteworthy accessibility improvements over 1.9.x, there are still significant problems that have persisted over numerous versions of Moodle and there are new problems introduced as well. The problems listed in Appendix B are rated in terms of their importance in fixing. The “High” priority items must be addressed before deploying Moodle and there must be a commitment to addressing all off the “Medium” and “Low” priority items as well. The same list of problems is also in the attached spreadsheet.

Back to top

Appendix A - Moodle Accessibility Links

Moodle Accessibility Forum
http://moodle.org/mod/forum/view.php?id=5752

Moodle 2.1 Demo Site
http://school.demo.moodle.net

Moodle 2.2 Demo Site (QA Testing)
http://qa.moodle.net

Moodle Accessibility Issue Tracker
http://tracker.moodle.org/browse/MDL/component/10083

Open Moodle Accessibility Issues
http://tracker.moodle.org/secure/IssueNavigator.jspa?mode=hide&requestId=12610

(Alternative Link for Open Moodle Accessibility Issues)
http://tracker.moodle.org/secure/IssueNavigator.jspa?reset=true&jqlQuery=project+%3D+mdl+and+component+%3D+Accessibility+and+status%3DOpen

Accessibility Issues for Moodle 2.2
http://tracker.moodle.org/browse/MDL-27843

ARIA Bug Report
http://tracker.moodle.org/browse/MDL-29623

Back to top

Appendix B

This section lists all of the problems noted in the report. Additionally, potential solutions and notes are included. The solutions do not go into the technical details of implementation, but often give a high level summary of actions that need to be taken. The last two columns of the table – Importance and Difficulty – are assessments of how important of a bug each item is, and an estimate on how difficult it will be to correct the problem. The importance levels are subjective and are based on the estimated frequency of use of a function and how much of a hurdle the problem creates. This same information is in the attached spreadsheet.

Component Problem Potential Solution Importance Difficulty
Blocks When a block becomes docked, its skip link is still present in the main page. The link still functions, but adds unnecessary confusion because it implies the block is still there. Remove the skip links. Medium Low
Blocks When a block is docked, it is taken out of the page headings because the heading text is rendered as an SVG text transform so the heading can be displayed as vertical text. Render the heading as plain text off the screen or use a CSS3 transform. SVG support in assistive technologies is not good enough yet. Medium Low
Blocks When a block is docked, screen reader users cannot access the contents of the block. Technically, it is possible for a screen reader user to access the contents of a docked block if the user knows exactly how the page is laid out, but Moodle does not give any indications or clues as to how to access the block’s contents. Blocks must be undocked for screen reader users to effectively access them. Use ARIA to denote that it is a menu with an expandable submenu. High Medium
Blocks The “hide block” function is an image, rather than a button, like the adjacent move to dock button. Therefore it is rendered and accessed differently by the screen reader user and leads to unnecessary confusion. This issue is fixed in 2.1.2 n/a n/a
Navigation Block There is no indication to screen reader users that a node is expandable or collapsible. Use ARIA to define a tree with expandable nodes. Medium Medium
Navigation Block When a node is expanded or collapsed, there is no indication that something has happened. Use ARIA to define a tree with expandable nodes. Medium Medium
Navigation Block In high contrast mode in Windows the custom bullets do not appear because they are rendered as background images Images that convey important information should be coded in the HTML, not as CSS background images. Medium Low
Navigation Block Dragon Naturally Speaking cannot easily access the collapsible menus and must use the mouse grid to click on them. Make all of the expandable nodes into links Medium Medium
Navigation Block The custom bullets in the tree have the alt text of “Moodle” which is unnecessary and confusing. The alt text should be a null string. Medium Low
File Picker For screen reader users it is not clear that a new window has opened up when the File Picker appears on the screen. Use ARIA to define the window as a modal window and automatically set the focus to an item in the modal window. High High
File Picker For both screen reader users and keyboard-only users, the focus is initially put on the last item in the File Picker, so to use the File Picker the user must start by reverse tabbing though the window. Put the focus on the same item within the File Picker each time it is opened. High High
File Picker The file picker is very difficult to navigate for screen reader users because of the way the entire window is laid out. There is no logical structure or sections of the page. The file picker needs to grab and trap the focus. It also needs to provide some logical structure for navigating from the repository list and the file selection and properties area. High High
File Picker It is very easy to escape the modal window and get lost in the page. This is a difficult problem to fully solve because assistive technologies don't implement this newer technique of modal windows very well. At a minimum, the correct ARIA coding should be used and the keyboard should be trapped within the modal window. Also, putting the modal window as the last item in the page with a h1 heading will help screen reader users who do escape out of the modal window. High High
File Picker For screen reader users, it is not apparent that something has changed in the right-hand panel of the file picker once a repository in the left panel is selected. The contents in the right-hand panel need to be coded with the correct ARIA attributes. High Low
Forms Form validation is not clear throughout the system. When there is an error in submitting a form it takes a lot of hunting to figure out what is wrong. Form errors should be noted with the correct ARIA mark up and, if the errors appear after a new page reload, the error message should be automatically focused on. If the error  message displays because of JavaScript validation, the error message should be coded with the correct ARIA attributes. High Medium
Forms Form elements coded incorrectly: many text input elements are missing labels make sure all form elements have labels High Medium (because of volume)
Forms Form elements coded incorrectly: read-only input elements being rendered as plain text but still have a corresponding label element if a form element is read-only, it should be rendered as an actual form input element with the readonly attribute set Medium Low
Forms Form elements coded incorrectly: many select input elements are missing labels make sure all form elements have labels High Medium (because of volume)
Forms Frequently there are orphaned form labels at the bottom of pages with large forms. remove the excess labels Low Low
WYSIWYG Editor The code for nested lists is incorrect. It reads fine in screen readers, but is not technically correct and does not validate. CKEditor implements this correctly and could be used in place of TinyMCE, although this is not necessary. Low High
Language Change Jump Menu The combo box to select your language is a jump menu and is present at the top of the main system page and each course’s main page. Either use a "Go" button next to the language change menu to execute the action, or only put this setting in the user profile Medium Low
Help Menu When opening the help window, there is no notification that something has opened. It just says there is a link there and focuses on to the close graphic, which has no alt text. The proper ARIA attributes need to be added, plus autofocusing the user to the window High Medium
View Profile Page The View Profile page uses a table for layout, but the way it is used there should probably be headings for the left column. add table headers Medium Low
Calendar The table for the calendar display should use table headers, like the mini-calendars already do. fixed in 2.1.2 n/a n/a
Calendar The main content needs a heading at the beginning. fixed in 2.1.2 n/a n/a
Calendar The “skip to main content” link doesn’t take you to the beginning of the main content. fixed in 2.1.2 n/a n/a
Calendar The “click to hide” link text is repeated often and should be unique for each type of event to show or hide use unique text for each element Medium Low
Calendar For the small calendars on the side of the page, either when viewing the main monthly calendar or in the calendar block, events can be seen by hovering over a date with a mouse, but this is not keyboard accessible and is not read by screen readers.  An onfocus event needs to be added so keyboard events will trigger the popup box. Also the popup will need to have the appropriate ARIA attributes High Medium
Calendar For the small calendars on the side of the page, either when viewing the main monthly calendar or in the calendar block, the table summaries do not need to include the text “data table”. Edit the table summary Low Low
Calendar The main monthly view of the calendar page does not have a table summary. Add a table summary Medium Low
Forgot Password Page The buttons do not need labels. Remove the labels Low Low
Forgot Password Page The same ID is used for multiple buttons. Be sure to use unique IDs Low Low
My Home Page The “My home” page needs some type of semantic structure to list out all of the upcoming activities, like homework assignments that are due, instead of making it all plain text. Add semantic structure to the page, like lists Medium Medium
Recent Activity Page There are empty heading tags. remove the empty heading tags Low Low
Recent Activity Page The headings are out of order. correct the heading order Low Low
Recent Activity Page There are form elements without labels. assign labels correctly Medium Low
Completion Tracking The alt text for the check box needs to indicate which particular item is being marked complete. Currently the relationship between the checkbox and the item can only be inferred by proximity. edit the alt text to include something like "Mark homework number 1 complete" High Low
Completion Tracking There needs to be some type of visual indicator linking the item and the completion checkmark so it is easier to visually associate the checkmark with the correct item. use CSS to draw a line between the two items, or else use :hover and :focus to indicate which item is being checked Medium Medium
Forums Screen reader users may have a hard time finding the actual box to type their reply in after clicking the “Reply” link because it is so far down on the page. This problem should be considered in the context of redesigning the entire forum. One easy solution is to put a heading at the reply entry location. High Medium
Forums There is no semantic structure to the discussion so it is very difficult, if not impossible at times, to accurately follow a discussion thread. This is a more challenging problem because it will involve redesigning how the forum is rendered. There needs to be some discussion on this issue. High High
Quiz For the heading for the question text, it would be better if it said “Question 2 text, heading level 3”. Currently it says “Question text, heading level 3”. change the alt text Medium Low
Quiz Flagging a question needs to say “Unflagged. Click to flag question 3” instead of “Click to flag this question.” Part of the problem is that some of the information is stored in the alt attribute and some is stored in the title attribute. All of the information should be stored in one location, or duplicated entirely in both locations. change the alt text Low Low
Quiz When flagging a question, nothing is spoken by screen readers. use an ARIA live region to voice the change Medium Low
Quiz The timer needs to be an ARIA live region. assign an ARIA Live region of timer High Low
Quiz Matching questions - no labels assign labels correctly High Low
Quiz Short Answer questions - no label assign labels correctly High Low
Quiz In the question status, questions tell users they are “not yet answered” even though I have given an answer. What it actually means is I haven’t saved my answer yet. Once it is saved, by moving from page to page, then it will say it is answered. There should be some way to explicitly save a page or have pages automatically save or else make the message clearer in the question status area. This is more of a usability issue for everyone, but is probably a bit more pronounced for screen reader users since they will be unsure if there was a "save" button somewhere on the screen that they missed Medium Medium
Quiz After selecting an answer, the text next to the question still say “Not yet answered.” What it actually means is the question is not yet saved. This is more of a usability issue for everyone, but is probably a bit more pronounced for screen reader users since they will be unsure if there was a "save" button somewhere on the screen that they missed Medium Medium
Quiz In the quiz summary, the question numbers should be table cell headings. assign table headers correctly Medium Low
Quiz The Quiz Navigation section uses a background color to indicate an answered question, which does not show up in high contrast mode the status of an answered vs an unanswered question needs to be indicated with a different cue as well Medium Medium
Choice The form elements are unlabeled assign labels correctly High Low
Choice The final response tally is cumbersome to read for screen reader users This should be greatly simplified into a single simple data table with headings. Medium Medium
Folder The file list is not keyboard accessible. Keyboard users cannot download the file because they cannot click on the link. (Note: screen reader users can download the file) The whole folder module needs to be redesigned, probably as a list with nested lists with the correct ARIA markup for trees High Medium
Folder There is no indication to screen reader users that the file tree has expandable and collapsible nodes. code the file list as a tree with ARIA attributes High Medium
Folder The relationship in the file hierarchy is only displayed graphically - there is no explicit relationship between files in different folders or subfolders. code the files as a list and use ARIA attributes to denote it as a tree High High
Folder The images used to visually display the tree hierarchy is read as a link with meaningless text. remove the link from the graphic High Low
Lesson The progress bar is not read by screen reader users This needs alternative text or the progress indicator needs to be displayed as text as well High Low
Lesson The title of the individual content page should be an h3, so it is less than the lesson title which is an h2. change the heading structure Medium Low
Grades The “user report” grade view is a complex table and needs headers explicitly assigned assign table headers correctly High Low
Grades The report view type is an unlabeled jump menu add an appropriate label to the form element and make a "Go" button to execute the selection in the menu Low Low
Book The chapter title should be a heading. add a heading Medium Low
Book The print view of the book should have headings. add headings Medium Low
Book Each chapter within the book should have a unique HTML title tag. change the page title Medium Low
Overview - Activity Report There are list coding errors where there are unordered lists with no list items and paragraph tags are used in place of list items.
(https://moodle-access.delta.ncsu.edu/course/user.php?id=2&user=25&mode=complete)
correct the coding errors Medium Medium
Chat The standard Chat activity has numerous accessibility problems, which is acknowledged by the fact that there is a more accessible version of the chat tool. Ideally there should be one version of chat because the technology exists to make a fully accessible chat client Medium High
Chat While the accessible version of the Chat activity is accessible, it could be more usable. The way to navigate the discussion is a bit cumbersome to listen to for screen reader users because of the use of definition lists. It might be better to store the chat messages in a table with three columns. Column one would be a header column and include the poster's name. Column 2 would be the message. Column 3 would be the time stamp. This would allow screen reader users to simply navigate up and down the second column and easily hear the most pertinent information - the person's name and their message. Medium Medium
Chat Most chat rooms have the place to post the message at the bottom of the screen and the chat history runs in reverse order. This implementation works just fine, but it might be confusing to users who are used to the more traditional method. consider changing the orientation of the messages and the location of the message box Low Medium
General Multipage items (e.g. Book, Activity Reports, Calendar) should have their page titles change to indicate to the user which page of an item they are viewing. change the page title appropriately Medium Low
General Background images should not be used for key visual cues because they will not be rendered in high contrast mode in Windows. move important background images to foreground images with alt text Medium Medium
General Screen reader applications tend to make the page visually jump around unexpectedly. It does not impact the functionality, but it will cause problems for users who might use a screen reader while needing to also view the screen. This is a problem I have never seen before so I'm not sure what is causing it. It does not happen all of the time, but when it does happen it occurs in both JAWS and NVDA. Low Hard
General ARIA landmarks need to be used to denote, at a minimum, the navigation area and the main content. Other areas could easily be added too. add the appropriate ARIA landmarks and roles High Low
General There are CSS rules defined for .blink (blinking text) and .justifytext (fully justified text). It is not apparent that either of these rules are being used, but they should not be used in most cases as they both have accessibility problems. remove these rules Low Low
General There are CSS rules that make text bolder. When appropriate, text should be coded with the strong tag instead of simply made bolder with CSS. just make sure the rule is being applied appropriately Medium Low
General Modal windows need to be implemented consistently to help ensure assistive technology users can interact effectively with the window. Modal windows are a tough problem because assistive technologies don't fully support the ARIA attributes that help define modal windows. Every effort should be made to (1) add ARIA attributes for modal windows, (2) properly set the focus to the modal window, (3) trap the keyboard so users cannot escape the modal window, and (4) put the modal window in a place in the DOM that is easy for screen readers users to find and put a heading 1 as the first item in the modal window. High High
General There needs to be consistency across all of the pages within Moodle to denote the main content area. The skip to main content link works, but there needs to be a consistent heading structure so users always know the main content starts, like always putting the same heading level at the start of the main content. Different activities in Moodle define the beginning of the "main content" differently. It would be nice if there was always a heading, probably an h1, at the beginning of the main content. Medium Medium
General The “screen reader” profile setting has very little impact in the UI. From an examination of the source code it only appears to impact the Chat activity and a grade book column for student names. It should be dropped in favor of making all aspects of the UI accessible. If this setting is used, it should only be used in cases where a particular implementation cannot be made accessible to screen reader users and a different implementation needs to be rendered. This should only be done as a last resort since most aspects of Web pages can be made accessible if implemented correctly. Low Medium

Back to top