Using Scenery and the Parallel DOM to support accessible web applications

Jesse Greenberg (jesse.greenberg@colorado.edu)
Michael Kauzmann (michael.kauzmann@colorado.edu)
Taliesin Smith (taliesin.smith@colorado.edu)
Emily Moore (emily.moore@colorado.edu)

Presentation Overview

Scenery: A library developed by PhET for interactive web graphics.
Capabilities of Scenery for graphics and accessibility
Demonstrate Scenery usage in PhET simulations
Demonstrate using Scenery for other graphics

PhET Interactive Simulations

Over 170 Physics and Science simulations
Free and openly licensed
Over 100 million runs per year, in 87 languages
Simulations are interactive and exploratory
Example: Friction

PhET Implementation Overview

Uses model view controller paradigm
Model code for physics, state, and physical calculations
View code for presenting model information to the user
Entire UI uses HTML5 renderers (Canvas, SVG, WebGL)
- Planets, circuits, balloons, particles
- Buttons, links, sliders, menus

Why custom render entire interface?

More control over visual rendering
Improved platform consistency
Better performance
Better for developing a project of scaling that uses custom graphics

The View: Scenery

Open source graphics library for HTML5, by PhET
Scenery is a scene graph implementation
Scene graph is a "tree" like data structure for what is displayed on screen
Items in the tree are called "Nodes" and have parent-child relations with other nodes in the graph

The View: Scenery

"Nodes" have graphical information
- Image data
- Shape with fill and stroke
- Transformation (translation, rotation, scale)
Scenery converts scene graph into visual display

A diagram representing a scene graph data structure. The diagram
is a tree-like hierarchy. At the root of the tree is a box labelled 'Play Area'. Out of this node are two arrows: one pointing to a child node
labelled 'Physics Book' and the other pointing to a node labelled 'Chemistry Book'. From each of these books are a number of arrows each pointing to
child nodes labelled 'Atom'. Beneath this diagram is an arrow pointing to the rendered image, which shows a physics book and a chemistry book made of
atoms in a play area.

The View: Scenery

Scenery supports HTML Canvas, SVG, and WebGL
All three of these renderers can be used in a single application
Example: Scenery Playground
(This example can be run locally after pulling Scenery)

Accessibility in traditional Web Graphics

Canvas:
- Sub DOM inside canvas
- addHitRegion, removeHitRegion, drawFocusIfNeeded
WebGL: Sub DOM inside Canvas
SVG: Native markup (title, desc), ARIA enhanced SVG

Issues with these approaches

Recommendation for each renderer is different
Supporting each traditional approach triples cost
Experimental technologies have limited support and cannot be used in production code

The Parallel DOM

Parallel HTML tree in addition to graphics
Accessible representation for what is displayed
Independent of other scenery graphics rendering
Visually hidden, available for assistive technologies
Leverages HTML and ARIA for best of web accessibility

The Parallel DOM: How it works

Add accessibility information to nodes in scene graph
- Tag Name
- Accessible name
- Role
- Descriptive Content
Scenery renders HTML tree from scene graph information

A diagram representing a scene graph data structure. The diagram
is a tree-like hierarchy. At the root of the tree is a box labelled 'Play Area'. Out of this node are two arrows: one pointing to a child node
labelled 'Physics Book' and the other pointing to a node labelled 'Chemistry Book'. From each of these books are a number of arrows each pointing to
child nodes labelled 'Atom'. Beneath this diagram are two arrows. The first pionts to a rendered image, which shows a physics book and a chemistry book made of
atoms in a play area. The second arrow points to a graphic showing the rendered PDOM, containing the HTML in the following block.

Play Area

In zoomed in view of where books meet, surface temperature thermometer is cool and atoms jiggle a tiny bit.

Zoomed in Chemistry Book

Resting lightly on top of Physics Book. Atoms jiggle a tiny bit.

Zoomed in Physics Book

Below Chemistry Book. Atoms jiggle a tiny bit.

Parallel DOM Features

Design accessible HTML for any web graphic
Highly dynamic descriptions with graphics
All benefits of accessible HTML
- Users are comfortable navigating semantic HTML
- Interact with content at their own pace
- Many navigation strategies to read DOM content

Parallel DOM Features

Leverage built in DOM interaction from native HTML
Keyboard navigation
Alternative input (when possible)
Example: Scenery Playground

(This example can be run locally after pulling Scenery)

Demonstration of PDOM

Friction (with visualization of accessible content)
Friction (with embedded content only)

Using Scenery for other web graphics

Scenery is free and open source!
Not recommended as a replacement for native HTML apps
It is a powerful tool for graphics interactives, and games where accessibility is built into the framework

Other Scenery examples

Conclusions

Web content is becoming increasingly graphical and interactive
PhET developed Scenery, a free and open source library that can produce state of the art web graphics
Scenery supports state of the art web accessibility
Scenery and its accessibility support has been used extensively by PhET
We encourage anyone to use this free, open resource in development of interactives and graphics!

Resources and Funders

Resources

Funders

The National Science Foundation (DRL-1503439, DRL-1621363, DRL-1814220)
The Hewlett Foundation
University of Colorado

Contact Us

Get Involved!

Twitter: @PhETSims
github: @phetsims
Facebook: @PhETSims
phet.colorado.edu/blog