Overview of the website

Eesti.ee is a one of a kind website. In their introduction it is described as following: "The State Portal eesti.ee provides safe Internet environment for communication with the state – offering reliable information and e-solutions for citizen, entrepreneur and official."

This sort of task defintily puts a lot of high expectations to one's functionality and security. One of the field of this high expectations would definetility be usability and accesibility.

As Eesti.ee is meant to be an online version of the State, catering its citizens needs in various fields (social support petitions, applying for several state-organized documents, - aids etc.), then one would expect this peticular site to cater those needs as the State is supposed to cater them in real life. This means multilanguage (aside estonian russian, english), support for disabled persons - as the "brick and mortar" facilities had to cater the needs of mobility impairment, then the internet removed the need for this type of support. But this research team fears, that with the absence of this major impairment to cater, the state has forgotten other - newly arisen impairment to serve. For example towards hearing and seeing impairment etc.

Navigation (structure)

The navigational principles of Eesti.ee are strictly mouse based. There are no apparent ways of keyboard-browseability. Although there is one noticeble feature, which many visually impaired (and others) will benefit from. There is a text magnifier input in the website, a content map and also the ability to browse the textual contents of the website (non-graphical version of the website).

Content of the website

Eesti.ee doesn't feature many modern multimedia solutions (flash, images), as the website's aim is not to so much entertain the user but rather to be a useful tool for one. As mentioned before, the content of the website is browseable by simply textual outtake (link to that is offered in the upper navigational bar), therefore one can say that the graphical interface is minimalistic.

What makes the usability more difficult (in terms of browseability and navigation), is the massive content provided by the website. As Eesti.ee is growing every month in terms of services offered to the user, the menubars consisting of functions have gotten quite massive. Although the management of the website has tried to simplify things by generating 3 main categories for functions, there is still too much information on the website for the user to comprehend.

User inputs used

There is a lot of Javascript and a multitude of anchors used in the site. This makes it hard for the possible visually impaired user to consume the content of the website.

Screenshots of the website

[1] [2] [3]

Accessibility testing

Accessibility assessment strategy

Clarify the scope of work
- Reviewed and identify important points
- Understand the scope of work for this phase
- Understand roles of involved parties
Visit site
Prepare for kickoff
Start scope of work list
Set up filing systems
- Initial filing system for electronic documents and reference materials set up
Gather background information
Verify accuracy of existing work
On-line work review
Finalize scope of work list
Create/update required documents
Prepare required reports

Overview of the automatic accessibility testing

WAVE Toolbar is a Firefox toolbar which provides a mechanism for running WAVE reports directly within Firefox. The toolbar reports run entirely within your web browser. The toolbar can check intranet, password-protected, dynamically generated, or sensitive web pages. Also, because the WAVE toolbar evaluates the rendered version of your page, locally displayed styles and dynamically-generated content from scripts or AJAX can be evaluated. The WAVE toolbar features four types of reports: Errors, Features and Alerts, Structure/Order View, Text-only View, Outline View.

Errors, Features and Alerts - This report displays the web page with the embedded accessibility icons and indicators.
Structure/Order View - This report presents the web page with a different set of icons and indicators that show the overall structure of the page,
Text-only View - This report provides the underlying text of the page,
Outline View - This report displays the headers in the page.

WAVE shows the original web page with embedded icons and indicators that reveal the accessibility information within the web page. Over 80 possible icons show missing alternative text for images, missing form labels, table structure, script elements and event handlers, document structure and reading order, and much more.

Developer's web page: http://wave.webaim.org

The Markup Validation Service by the World Wide Web Consortium (W3C) allows Internet users to check HTML documents for conformance to HTML or XHTML standards. It also provides a quick method for web page authors to check their posted pages for mark-up errors. HTML validators operate by comparing the mark-up on a web page to the W3C standards. Once the validator has read the page and determined the applicable standards it looks for such things as missing opening or closing tags, missing quotation marks and other hand-coding errors. The validator then provides a report indicating that the coding is correct or not. Errors are noted in a list. One error, such as neglecting to close a tag, can cause a cascade of errors through the page, producing dozens or even hundreds of noted errors. However when the page author addresses the first error listed it will also eliminate the "cascade errors". This validator is also “An HTML validating system conforming to International Standard ISO/IEC 15445—HyperText Markup Language, and International Standard ISO 8879—Standard Generalized Markup Language (SGML)” – which basically means that in addition to W3C recommendations, it can validate according to these ISO standards.

Eesti.ee accessibility analysis (according to Section 508 and WCAG guidelines, automated testing)

Truwex Online (http://checkwebsite.erigami.com/accessibility.html), Web Accessibility Testing Tool (Section 508 compliance) was used to conduct the testing. The document that was elected for testing purposes was the front page of Eesti.ee (http://www.eesti.ee/est/), in Estonian language.

Let's look closer at the manually ordered report (by Section 508 and WCAG standards).

Accessibility testing results

The parser found 8 accessibility issues. Some of them have to be checked manually.

Issues

[508 (N)] [WCAG 12.4 (2)] Form control without explicit label is found

Description. Form control should have non-empty explicit label, or at least non-empty title. Explicit label is a label tied with input by 'for'/'id' attributes.

[WCAG 2.2 (2)] Low-contrast text is found

Description. Text color and its background color are too close: color difference should be no less than 400 and brightness difference should be no less than 125. Note: Contrast calculations here are based on colors specified via HTML markup. If colors you see by your eyes are in good contrast but Truwex says they are not, try to open your page in a browser with switched off images (set "Don't load images" option in FireFox or in Opera). It is possible that you use background images that make good contrast but without them the contrast is not good.

[WCAG 3.4 (2)] Fixed fonts are found

Description. Use relative rather than absolute units in markup language attribute values and style sheet property values.

[WCAG 3.4 (2)] Fixed sizes are found

Description. Fixed sizes other than font-size property are found. This includes absolute units of measure in frame sizes and in inline styles of HTML buttons and other text containers.

[WCAG 3.5, 12.3 (2)] Hardly reachable text for voice reader is found

Description. Percent of text, reachable by voice reader within a specified time limit, is less than the defined threshold. I.e. our estimation shows that less than 75% of page's text is reachable in 90 sec (these are the default values). In other words, it may take a long time for a user with a screen reader software to reach certain parts of a web page. Consider using hierarchical headers H1, H2 etc. and skip links to create shortcuts for users of screen readers. Note: the Map shows only 10 the most hardly reachable text fragments.

[WCAG 6.4, 9.3 (2)] Device-dependent event handler is found

Description. Elements with specified mouse event handlers should also have paired keyboard event handlers. E.g. 'OnClick' and 'OnKeyPress', 'OnMouseOver' and 'OnFocus' etc.

[WCAG 13.1 (2)] Link name/destination clash

Description. Two links with the same text should go to the same page, otherwise their 'title' attributes should differ.

[WCAG-2 1.4 (2)] Low luminosity contrast text is found

Description. Luminosity contrast ratio between text and background behind the text should be equal to or greater than 5:1, except if the text is pure decoration. Larger-scale text (at least 18 point or 14 point bold) can have a contrast ratio of 3:1. This is the WCAG 2.0 priority 2 (AA) requirement and it often gives the result different from WCAG 1.0 color contrast checks. Note: Contrast calculations here are based on colors specified via HTML markup. If colors you see by your eyes are in good contrast but Truwex says they are not, try to open your page in a browser with switched off images (set "Don't load images" option in FireFox or in Opera). It is possible that you use background images that make good contrast but without them the contrast is not good.

Warnings

The parser found 20 accessibility warnings. Some of them have to be checked manually.

[508 (C)] [WCAG 2.1 (1)] The use of color must not be a single method for indicating important information on a web page

Description. Manual check. Ensure that all information conveyed with color is also available without color, for example from context or markup.

[508 (D)] [WCAG 6.1 (1)] Documents must be organized so they are readable without requiring user agent support for style sheets

Description. Manual check. Organize documents so they may be read without style sheets. For example, when an HTML document is rendered without associated style sheets, it must still be possible to read the document.

[508 (J)] Flicker rate must be in the specified limits

Description. Manual check. Page shall be designed to avoid causing the screen to flicker with a frequency greater than 2 Hz and lower than 55 Hz. Check gif images and objects, applets, embed elements.

[508 (O)] [WCAG 13.6 (3)] Page without skip link

Description. A 'skip link' is one of the methods in providing a mechanism for users of assistive technologies to skip repetitive navigational links. Keyboard users also fully take advantages of this method. E.g. assume a page with navigation links at its top, and the links are coded in HTML before the page's main content. And if before the navigation bar a 'jump to main content' link will be provided, it will be very simple for users to access page content, in browser using TAB key or in screen reader.

[508 (P)] Page requiring timed response must allow user to get more time

Description. Manual check. When a timed response is required, the user shall be alerted and given sufficient time to indicate more time is required.

[WCAG 2.2 (2)] Images must have sufficient contrast

Description. Manual check. Ensure that foreground and background color combinations on images provide sufficient contrast when viewed by someone having color deficits or when viewed on a black and white screen.

[WCAG 3.1 (2)] Use markup instead of images where possible (it should be looked over manually)

Description. Manual check. When an appropriate markup language exists, use markup rather than images to convey information. For example, use MathML to mark up mathematical equations, and style sheets to format text and control layout. Also, avoid using images to represent text -- use text and style sheets instead.

[WCAG 3.2 (2)] Document must be valid

This document uses HTML 4.01 Transitional doctype. This document was successfully checked as HTML 4.01 Transitional! Passed, 2 warnings.

[WCAG 3.3, 3.5 (2)] Do not use headers and other structural markup only for visual effects

Description. Manual check. Use header elements according to specification. Do not use them only for presentational effects, e.g., for bold text. Do not use BLOCKQUOTE element for indents only. Do not use emphasis elements such as EM and STRONG for font effects only.

[WCAG 3.6 (2)] Mark up lists

Description. Manual check. Mark up lists and list items properly. If your page contains logical lists, mark up them with UL for unordered list, OL for ordered list, DL for definition list and LI for list item. Do not use tables with bullet images to imitate unordered lists. Do not use numbers in text for ordered lists: e.g., a line contains '1.1 Chapter' text and the next line contains '1.2 Next chapter' text.

[WCAG 3.7 (2)] Mark up quotations

Description. Manual check. Mark up quotations with Q and BLOCKQUOTE elements.

[WCAG 4.1 (1)] Changes in the natural language must be clearly identified

Description. Manual check. Clearly identify changes in the natural language of a document's text and any text equivalents (e.g., captions)

[WCAG 5.3 (2)] Do not use tables for layout if they can not be linearized properly

Description. Manual check. Do not use tables for layout unless the table makes sense when linearized. Otherwise, if the table does not make sense, provide an alternative equivalent (which may be a linearized version).

[WCAG 6.2 (1)] Equivalents for dynamic content must be updated when the dynamic content changes

Description. Manual check. Ensure that equivalents for dynamic content are updated when the dynamic content changes.

[WCAG 7.1 (1), 7.2, 7.3 (2)] Avoid flickering, blinking and movement in pages

Description. Manual check. Until user agents allow users to control blinking, avoid causing content to blink (i.e., change presentation at a regular rate, such as turning on and off). Until user agents allow users to freeze moving content, avoid movement in pages. Check objects, applets and scripts.

[WCAG 8.1 (1)] Programmatic elements must be directly accessible

Description. Manual check. Make programmatic elements such as scripts and applets directly accessible or compatible with assistive technologies [Priority 1 if functionality is important and not presented elsewhere, otherwise Priority 2.]

[WCAG 10.2 (2)] Implicit labels must be properly positioned

Description. Manual check. For all form controls with implicitly associated labels, ensure that the label is properly positioned. The label must immediately precedes its control on the same line (allowing more than one control/label per line) or be in the line preceding the control (with only one label and one control per line).

[WCAG 11.1 (2)] Use the latest W3C technologies available whenever possible

Description. Manual check. Use W3C technologies when they are available and appropriate for a task and use the latest versions when supported.

[WCAG 13.3, 13.4 (2)] Provide clear and consistent navigation

Description. Manual check. Provide information about the general layout of a site (e.g., a site map or table of contents). In describing site layout, highlight and explain available accessibility features. Use navigation mechanisms in a consistent manner.

[WCAG 14.1 (1)] The clearest and simplest language must be used

Description. Manual check. Use the clearest and simplest language appropriate for a site's content.

Tools used for examining the accessibility of the eesti.ee

Technical assesment – in order to examine the technical accessibility solutions applied effectively we used automated testing tool Truwex Online http://checkwebsite.erigami.com/accessibility.html

Involving Users in Evaluating Web accessibility – in order to explore usability issues that were not discovered by automated tools we: a)examined pages by using graphical browers; b)conducted a test by using specialized browser - Mac OS X VoiceOver accessibility feature

Overview of the real-time accessibility testing

The recording of the accessibility testing session can be found here: http://www.youtube.com/watch?v=O12hJyg5rHc

The testing session was conducted using Mac OS X VoiceOver accessibility feature, Safari web browser and recorded using the Silverback application. VoiceOver allows a visually impaired user to move around the screen using the keyboard and the computer reads the titles of elements on screen as well as gives hints about what type of information the user can submit in different input fields.

The test assignment was to log in to the State Portal eesti.ee using Mobile-ID and send an email using the official @eesti.ee email address. The task was simplified as much as possible since from the very beginning it became apparent that the State Portal’s architecture makes it very difficult to navigate using a screen reader.

The testing session emphasized some of the major shortcomings in the way the State Portal is structured. First of all it is hard to find the Enter button to get to the login page since the HTML is poorly structured and does not correspond to the visual representation. Some of the input fields, such as the search box for example, are missing labels so there is no way for the screen reader to identify it as a search box.

For the user to get to the main content of the page he must first navigate through the left menu, which features dozens of items and sub-menus.

Overall the results of this simple accessibility testing session illustrate the shortcoming of the portal’s structure very nicely. This kind of testing is a great way for developers and designers to think of their products from a very different perspective then they normally would.

Usability session

Usability assessment strategy

Our aim was that testing session per user should not last more than 40minutes. Users were asked to indicate when they are finished with a task or if they want to stop. Users give answers for all tasks in various ways - verbally or via questionnaire.

Session logic

Welcome
Profile interview (age, occupation, internet usage level, has or hasn't used eesti.ee,first impressions and emotions towards e-government services)
Task description ( 4 tasks in total)
Post-task questionnaires (how difficult was it to complete the given task)

Usability testing strategy

The tasks which the subject has to complete are:

Enter self-service environement
Register your official e-mail address.
Check your information in the Population Registry.
Register your business.

Note: users were not asked to complete the task but to find an appropriate link. Test session is conducted in full version,

Performance and satisfaction metrics

How easy or difficult was it to complete the task?

1-Very easy; 2-Easy; 3-Pretty easy; 4-Neither; 5-Pretty difficult; 6-Difficult; 7-Very difficult;

Time - to measure how long it took a user to complete a specific task, we started the clock after the person read the task
Errors - how many times the user selected the wrong feature in the interface

Valid face to face and online usability assessment

users consider welome text of the homepage a feature that is not nesseccary, some read it but only 2% click on the

provided quick link "Services for citizens";

some users start to look at information they need first from the right side of the page,

as this consists news, they find the homepage unuseful or not well organized;

users concider that important annaouncements aren't relavant, they are impersonal;
10% of users started to search for needed information through "Search box"
log in button is too hidden therefore users in majority started to look for asked link description;
90% of users started to look at useful information at the centre of the page but did not

find informatsion needed, therefore were in trouble or confused, only 10% scrolled down;

users found the link "Official e-mail address" not descriptive enough

suggestion: better description could be - "Register your personal e-mail account";

some started to look at e-mail registration in the section "Government e-services"
users epext the links to be in alphabetical order, therefore users spended too much time for searching the needed link

Usability testing results

Here are the main user requirements, which can be applied to various aspects of the State portal:

Users are not expecting to jump between different modules while completing a single task. This contributes to confusion.
The user always needs a concrete way of understanding his whereabouts in the portal.
Important functions should not be positioned at the end of the page, since users tend not to discover them.
Important functions (such as save or submit) should be represented as buttons instead of links.

Here are the test results - http://spreadsheets0.google.com/ccc?key=tp3vWamJE8a2CbwenxnUVLw&hl=en - (it opens in Google Docs environment; notice the two tabs).

Overview of the hands-on usability testing

In other to check for errors that automatic tools missed we conducted

a) manual test accoring to guidelines of WAI

b) involved a user in evaluating Web accessibility

A brief manual test

The goal was to examine the frontpage and it's accessibility. The structure of the test was based on the tasks described on WAI guidelines.

1. Turn off images, and check whether appropriate alternative text for the images is available. The search box disappears and is not accessible; The icon "Sisene" (Enter) does not have alt text, but is accessible;

2. Turn off the sound, and check whether audio content is still available through text equivalents. No audio content, therefore there was nothing to explore;

3. Use browser controls to vary font-size: verify that the font size changes on the screen accordingly and that the page is still usable at larger font sizes. Font size changes accordingly;

4. Test with different screen resolutions, and/or by resizing the application window to less than maximum, to verify that horizontal scrolling is not required (caution: test with different browsers, or examine code for absolute sizing, to ensure that it is a content problem not a browser problem).

IE – 1280x1024 – no horizontal scrolling needed;

1024x768 - - no horizontal scrolling needed;

800x600 – horizontal scrolling needed;

5. Change the display color to gray scale (or print out page in gray scale or black and white) and observe whether the color contrast is adequate. The boldest information areas are the tab „Ettevõtjale“ (Entrepreneur) and the line under it as well as the logo – as these sections are either on the top or in the center position of the page, they also stand out (together with the welcome text) from the rest of the information. In general all information is readable.

6. Without using the mouse, use the keyboard to navigate through the links and form controls on a page for example, using the "Tab" key, making sure that all links and form controls are accessible and that links clearly indicate where they lead to.

Findings:

Structure of the tab movement: top navigation, left sidebar, middle part of the page, right column of the page, header.

Top navigation:

Logo –ok
? – do not understand where the tab is located
Option to change text size–ok
Not possible to choose the graphical version
?
?
Choice of „Enter“ is missing or hidden;

Search form – not possible to choose/change the radio buttons;

The recording of the usability testing session can be found here: http://www.youtube.com/watch?v=c6rgHOzTT7c

Recommendations for usability improvement

Services

Services should respect the navigation logic and the design of the portal.
The names of services should correspond to the goals of the services' users.
A user should be able to achieve his goal by using one service instead of multiple ones in a row. A service which is used in multiple steps could be also divided into multiple pages.
If the user has already logged in the portal should not ask for the users' information (name, age, etc.) again. Also the portal should ask for data located in other registries but should fill the corresponding fields automatically.

Content

The users do not read texts and labels thoroughly. Instead the scan and try decide if the paragraph represents something he is looking for. A well partitioned text where paragraphs begin with words which the user was looking for gives the best overview. People try to find relevant information as fast as possible. Users often decide if the text is useful to them by just reading the first line or the first paragraph. It is better to create shorter pages but not insert to many links to relevant information because it makes the process of getting an overview of the text slower. The user is also looking for facts.
Not all people are familiar with search options, but most of them are. Sometimes users are afraid of clicking on links. So it is important to give the important information without additional navigation. The information should be discoverable with minimal trouble.
Requirements to writing
- Start writing with answering the user's question.
- The rest can be written later.
- Numbers should be written as numbers.
- Write summaries.
- A short text is better than a long one.
- Do not use too many big images.
- Organize content in a logical way.
- One paragraph should only contain a single idea.
- Do not use juridical or technical language.
- Do not use bullets or lists in the introduction part.

Conclusion

The State portal eesti.ee is a complex website which needs to fulfil certain expectations. One of the main requirements is that the portal has to accommodate different needs of very different types of users.

One of the main shortcomings of the website is that it is not optimized for being used by people with disabilities and is clearly lacking in the accessibility department. There is a reason for that, since during the years the portal has been developing and growing very little attention has been put to the goal of making the portal complaint with accessibility standards. Another major problem of the portal is the structure of the HTML itself. The portal is table based and that makes it very hard to navigate for people with screen readers. There are lots of information and articles which are poorly structured and hard for users to understand.

Solving all of these problems requires a systematic approach. A new version of the portal is currently is the works and the project was started with conducting of a comprehensive usability and accessibility analysis. The redesign should transform the State portal to a product, which is easier and more pleasing to use.