assistive documentation data¤
hey folx, my name is tonyfast and i'm excited to discuss a concept for assistive, accessible and labor-reducing, documentation.
our plan for today is to explore a more accessible future of scientific literature. consider this presentation a possible sensory future for computational notebooks. together we'll experience a screen reader while discussing the following items:
- give some history about why i am here
- share a brief demonstration about the concept of documentation as data
- listen to accessible notebook experiences with y'all
- consider how accessibility could guide sustainable open science programs
> react in zoom if that was your first time experiencing a screen reader
disclaimers¤
- there are two threads in this presentation
- disability advocacy for assistive documentation
- immersing folx in the screen reader experience
- this talk assumes awareness of pandas dataframes and jupyter notebooks.
- this talk demonstrates two active works in progress.
nbconvert-a11yis the more mature project containing accessible reference templates for computational notebooksnobookthinks that books and publications are terrible, non-reusable databases.nobookteaches data literacies learners explore, transform, and expand their documentation worlds
- this is a performance piece developed to be read primarily by a screen reader
%%
## notebooks for all is example of leading with accessibility
i'm here because of the [notebooks for all] project.
more specifically, i'm here because Jenn Kotler does remarkable work connecting communities.
<details><summary>a presentation with a short history of the notebooks for all project</summary>
<iframe height="600" loading="lazy" src="https://tonyfast.github.io/tonyfast/draft/tonyfast/tonyfast/tonyfast/xxiv/2024-03-11-NOA.html#astronomy-notebooks-for-all" title="notebooks for all history and references" width="100%"></iframe>
</details>
> __screen reader fact__:
screen readers have dynamic tables of contents that allow folx to navigate quickly through headings
[notebooks for all]: https://iota-school.github.io/notebooks-for-all/
notebooks for all is example of leading with accessibility¤
i'm here because of the notebooks for all project. more specifically, i'm here because Jenn Kotler does remarkable work connecting communities.
a presentation with a short history of the notebooks for all project
> screen reader fact: screen readers have dynamic tables of contents that allow folx to navigate quickly through headings
%%
### notebooks for all research timeline
* in 2021, jenn and erik visited [jupyter accessibility meeting](https://jupyter-accessibility.readthedocs.io/en/latest/)
* in 2022
* we meet patrick at the accessibility meeting
* fall we had DDRF funding an ambitious statement of work
* we learned boatloads from jenn and isabela testing the rendered ~~static~~ notebook experience with affected users from the blind visually impaired community. this is a first of its kind research brings the disabled experience to open source practices.
* in 2023
* we came together to experience __disabled joy__ together at the Day of Accessibility at the Space Telescope Science Institute
<iframe allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" allowfullscreen="" frameborder="0" height="315" referrerpolicy="strict-origin-when-cross-origin" src="https://www.youtube.com/embed/videoseries?si=Uo1rOXfXaX42CebX&list=PLCPZgcYzVpj_WHHCTUpec8THYEMzXZnR1" title="YouTube video player" width="560"></iframe>
* we realized that the work we've done building accessible notebooks applies to [large language model interfaces](https://huggingface.co/spaces/huggingchat/chat-ui/discussions/263)
* in 2024
* another first, patrick smyth and sarah kane provided access to [__nonvisual data science education__](https://www.youtube.com/watch?v=TenQgvGmrlE&list=PLoOee19aArh0Lvu8O2Mgbg56sJBdkAClw) for blind low vision learners with a screen reader focus
* development shifts to codifying the design results into the [`nbconvert-a11y` project](https://github.com/deathbeds/nbconvert-a11y) that we'll discuss today. [`nbconvert-a11y` is published on pypi](https://pypi.org/project/nbconvert-a11y/), but not recently.
* notebooks for all continues as a volunteer open source project
notebooks for all research timeline¤
- in 2021, jenn and erik visited jupyter accessibility meeting
- in 2022
- we meet patrick at the accessibility meeting
- fall we had DDRF funding an ambitious statement of work
- we learned boatloads from jenn and isabela testing the rendered
staticnotebook experience with affected users from the blind visually impaired community. this is a first of its kind research brings the disabled experience to open source practices.
-
in 2023
- we came together to experience disabled joy together at the Day of Accessibility at the Space Telescope Science Institute
- we realized that the work we've done building accessible notebooks applies to large language model interfaces
- in 2024
- another first, patrick smyth and sarah kane provided access to nonvisual data science education for blind low vision learners with a screen reader focus
- development shifts to codifying the design results into the
nbconvert-a11yproject that we'll discuss today.nbconvert-a11yis published on pypi, but not recently. - notebooks for all continues as a volunteer open source project
help me meet you where you are¤
- ACTION - share
what are notebooks to you?
by typing in the chat or turning on your mic. -
tonyfast's list
> this work is personal to me because notebooks are an assistive technology for me. with a career in data and modeling, notebooks have given me access to language and science to share with other people. they are where i continue to learn how to program.
- notebooks are literacy machines for reading and writing computational thought
- they can make computational thought percievable, operable, understandable, and robust. further, they can compromising, assistive, and flexible for better data-driven documents.
- notebooks provide access to compute
- notebooks are place to test things
- notebooks are data
- notebooks are hypermedia collage
- notebooks are computational skateparks to do rad tricks with text
- notebooks are a medium, message, and model
- notebooks are a means and an ends
%%
### lets think about collections of notebooks
building documentation from notebooks is difficult because we treat them as text files, not data files.
data-driven documentation respects the structure of notebooks to build comprehensive data structures
containing the information about cells, outputs, mimebundles, and headings across all the document in a project.
import nobook.builder
docs = nobook.builder.Contents.Config.new(
root="mast_notebooks/", toc="_toc.yml",
kind=nobook.builder.ConfigKind.jb)
docs = (await docs.discover()).head()
docs = await (await docs.expand()).compact()
docs.head(10).assign(html=None) # html is a whole page so we dont put that in the document
lets think about collections of notebooks¤
building documentation from notebooks is difficult because we treat them as text files, not data files. data-driven documentation respects the structure of notebooks to build comprehensive data structures containing the information about cells, outputs, mimebundles, and headings across all the document in a project.
import nobook.builder
docs = nobook.builder.Contents.Config.new(
root="mast_notebooks/", toc="_toc.yml",
kind=nobook.builder.ConfigKind.jb)
docs = (await docs.discover()).head()
docs = await (await docs.expand()).compact()
docs.head(10).assign(html=None) # html is a whole page so we dont put that in the document
%%
### documentation as data allows us to interactively explore our works
when we have our [documentation as data](https://tonyfast.github.io/tonyfast/draft/tonyfast/tonyfast/tonyfast/xxiv/2024-03-22-documentation-as-data.html)
we can do interesting things we might not have considered before. the most interactive documentation can be is
in an interactive computing context.
docs.iloc[2:4].html.display.iframe(height=600).display()
documentation as data allows us to interactively explore our works¤
when we have our documentation as data we can do interesting things we might not have considered before. the most interactive documentation can be is in an interactive computing context.
docs.iloc[2:4].html.display.iframe(height=600).display()
%%
our `docs` are structured information about the cells containing the source and metadata
> __screen reader fact__: screen readers can navigate tables with arrow keys
docs.cells.head(2).pipe(display)
our docs are structured information about the cells containing the source and metadata
> screen reader fact: screen readers can navigate tables with arrow keys
docs.cells.head(2).pipe(display)
%%
we have structured information about the outputs containing errors and mimebindles
docs.outputs.head(2).pipe(display)
we have structured information about the outputs containing errors and mimebindles
docs.outputs.head(2).pipe(display)
%%
we have an enormous collection of hypermedia
docs.bundles.head(2)
we have an enormous collection of hypermedia
docs.bundles.head(2)
beyond accessibility - what makes an assistive, labor-reducing notebook?¤
-
POUR are principles for web content accessibility guidelines, but they are insufficient for interative computing in education. regulatory compliance of interactive computing is neglectful of the needs of disabled scientists.
- we use extended POUR-CAF chartability guidelines
> Chartability is a set of heuristics (testable questions) for ensuring that data visualizations, systems, and interfaces are accessible. Chartability is organized into principles with testable criteria and focused on creating an outcome that is an inclusive data experience for people with disabilities.
- when we have nowhere else to turn we turn to game accessibility guidelines
- the document sounds and navigates naturally with assistive technology
- the DOM and ARIA generate a well structured accessibility object model
- the document has been tested with a screen reader and with screen reader navigation
- design choices consider the audible, visual, and tactile representation of information units
- the design choices are configurable
- the design is robust, it passes automated tests
- the designs and content have been tested with, hopefully paid, affected users
> this is last because organizations should do consider the prior before testing with affected users, especially, unpaid ones. crip time can be considerably costly for some disabled people
%%
## demonstrate the newest `nbconvert-a11y` template
* we've been experiencing a notebook produced by `nbconvert-a11y` template this whole time!
* __how has the screen reader experience been for folx?__
* we'll compare the current annotation object model with [a typical version](https://spacetelescope.github.io/mast_notebooks/notebooks/astroquery/beginner_search/beginner_search.html)
> * html and aria shape the accessibility object model tree
> * no aria is better than bad aria
> * first rule of aria: use native elements if you can
> * second rule of aria: don't overload native semantics unless necessary.
* we'll explore flexible and compromising [aria live](https://developer.mozilla.org/en-US/docs/Web/Accessibility/ARIA/ARIA_Live_Regions) settings for screen readers and visual access needs
* in notebooks, we can't ensure that content or package authors have made their content accessible
* we developed better exception types using exception groups that allow us to measure accessibility and [inaccessibility using axe accessibility testing](https://github.com/deathbeds/nbconvert-a11y/blob/main/tests/test_third.py)
* we developed a pytest fixture for [the w3c validator to test our templates](https://github.com/deathbeds/nbconvert-a11y/blob/main/tests/test_w3c.py)
* in notebook accessibility tests with [sa11y](https://github.com/ryersondmp/sa11y/)
demonstrate the newest nbconvert-a11y template¤
- we've been experiencing a notebook produced by
nbconvert-a11ytemplate this whole time! -
how has the screen reader experience been for folx?
-
we'll compare the current annotation object model with a typical version
> * html and aria shape the accessibility object model tree > * no aria is better than bad aria > * first rule of aria: use native elements if you can > * second rule of aria: don't overload native semantics unless necessary.
-
we'll explore flexible and compromising aria live settings for screen readers and visual access needs
-
in notebooks, we can't ensure that content or package authors have made their content accessible
- we developed better exception types using exception groups that allow us to measure accessibility and inaccessibility using axe accessibility testing
- we developed a pytest fixture for the w3c validator to test our templates
- in notebook accessibility tests with sa11y
what makes accessibility hard and what to do?¤
a list of challenges in accessibility in no particular order. the list items can be expanded for actions to improve these challenges.
describing the disabled experience in issues using written language is insufficient
> encourage audio and video feedback for software projects. meet people where they are
* low accessibility and screen reader literacies amongst developers
> every person experiencing this talk has a screen reader on their device. spend 10 minutes every couple days trying it out. become familiar with your accessibility settings so you may discover your own access needs
* complex builds increase the distance between source code and html
> slow feedback loops kill productivity and desire. reducing complexity between source code and the outcome requires maintaince and refactoring. constant growth mindsets increase technical debt.
</details>
hyrum's law, technical debt, lack of history increase the complexity with retrofitting inaccessibility
> kill it with fire - Manage Aging Computer Systems (and Future Proof Modern Ones)
* lack of practice guides or reference implementations for components
> make sure your framework is robust and aligns with standards
* capitalism
[capitalism & disability](https://www.haymarketbooks.org/books/1289-capitalism-and-disability)
accessibility and open science¤
accessibility - the technical side of disability - provides foundational principles for sustainable, enduring, archival data-driven scientific literature.
- accessibility demands asking "what does this mean? is it percievable? operable? understandable? robust?" semantically meaningful design improves all of these qualities.
- without reference implementations that include disabled developers and affected communities, we'll continue to make inaccessible data experiences.
- some of y'all are early in your career - if you are lucky, you'll be seeing the fruits of your labors 50 years from now. future you will be a different person with different access needs - how do you want to experience science when you are old?
- accessibility creates better data and metadata for archiving documents
- accessibility principles recover the sense data experiences that formed early philosophies of science, mathematics, reason, and stargazing.
connecting in a more accessible future¤
please reach out to me if you're interested in collaborating or finding ways to support this work
extra content¤
%%
### turning on your screen reader
* [Windows Narrator](https://dequeuniversity.com/screenreaders/narrator-keyboard-shortcuts)
* [Google TalkBack](https://dequeuniversity.com/screenreaders/talkback-shortcuts)
* [GNOME Orca](https://help.gnome.org/users/orca/stable/commands_controlling_orca.html.en)
* [Mac VoiceOver](https://dequeuniversity.com/screenreaders/voiceover-keyboard-shortcuts)
* [IOs VoiceOver](https://dequeuniversity.com/screenreaders/voiceover-ios-shortcuts)
turning on your screen reader¤
%%
<h3>start at the end: <a href="https://science.nasa.gov/mission/hubble/multimedia/sonifications/">data sonification - westerlund 2</a></h3>
<blockquote>
This is a cluster of young stars – about one to two million years old – located about 20,000 light-years from Earth. In its visual image form, data from Hubble (green and blue) reveals thick clouds where stars are forming, while X-rays seen by Chandra (purple) penetrate through that haze. In the sonified version of this data, sounds sweep from left to right across the field of view with brighter light producing louder sound. The pitch of the notes indicates the vertical position of the sources in the image with the higher pitches towards the top of the image. The Hubble data is played by strings, either plucked for individual stars or bowed for diffuse clouds. Chandra’s X-ray data is represented by bells, and the more diffuse X-ray light is played by more sustained tones.
</blockquote>
<iframe allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" allowfullscreen="" frameborder="0" height="473" src="https://www.youtube.com/embed/ESz8Cvirh00" title="Data Sonification: Westerlund 2 (Multiwavelength)" width="840"></iframe>
start at the end: data sonification - westerlund 2
This is a cluster of young stars – about one to two million years old – located about 20,000 light-years from Earth. In its visual image form, data from Hubble (green and blue) reveals thick clouds where stars are forming, while X-rays seen by Chandra (purple) penetrate through that haze. In the sonified version of this data, sounds sweep from left to right across the field of view with brighter light producing louder sound. The pitch of the notes indicates the vertical position of the sources in the image with the higher pitches towards the top of the image. The Hubble data is played by strings, either plucked for individual stars or bowed for diffuse clouds. Chandra’s X-ray data is represented by bells, and the more diffuse X-ray light is played by more sustained tones.