Skip to content

generating filters and visibility widgets for tablesยค

we can infer a lot about the table interactions from the inherit type information in a dataframe. we demonstrate the steady state html that would form the basis for an accessible experience.

demo

%%
```css
a[href="#gist"] {
    display: inline-block;
    height: 200px;
    width: 100%;
    background: -moz-element(#gist-region);
    background-size: contain;
}
```

a[href="#gist"] {
    display: inline-block;
    height: 200px;
    width: 100%;
    background: -moz-element(#gist-region);
    background-size: contain;
}



%%
    gists = (await (
        "https://api.github.com/users/tonyfast/gists?page=" + pandas.RangeIndex(1, 5).astype(str)
    ).http.get()).explode().series().set_index("id")
    gists.index.name = "gist_id"
we're going to quickly make a table of my gist from github.

    files = gists.join(gists.files.apply(dict.values).apply(list).explode().series())
unravel all the gists in each gist payload.

    time = files.columns[files.columns.str.endswith("_at")].tolist()
    files[time] = files[time].apply(pandas.to_datetime, axis=1)
    files = files.assign(timespan=files.updated_at - files.created_at)

convert times to their proper types and substract them so out dataframe 
contains integers, datetimes, and timedeltas, types with numerical representations.

    files[cat] = files[cat := "language type".split()].astype("category")
    df = files[time + "timespan filename description language type size".split()]

now are going to add a few lines of `css` that uses the `--ratio` variables in each numerical cell
to produce visual representation of the cell value in the larger distribution.
it is really interesting how we start to create the cell border form when we do this.
at a glance we can visually observe relative positions of values in a column.
further, scrolling creates really interesting movement that provides user controlled animation.

    notebooks = (
        await files[files.language.eq("Jupyter Notebook")].raw_url.http.get()
    ).apply(json.loads).series()
read the `notebooks` and separate out the cells, outputs, and display objects.

    cells = notebooks.cells.enumerate("cell").series()
    outputs = cells.outputs[cells.outputs.fillna("").astype(bool)].enumerate("display").series()
    display_data = outputs['data'].dropna().series()

    df = (df.join(cells.groupby("gist_id").id.count().rename("cells"))
          .join(outputs.groupby("gist_id")["data"].count().rename("outputs")))
    from nbconvert_a11y.table import new; import nbconvert_a11y
    table = df[
        "filename description language type created_at updated_at timespan size cells outputs".split()
    ].sort_values("created_at", ascending=False).table(id="gist").style(
```css
dialog > details >  form > button[formmethod=dialog] {
    display: none;
}
dialog[open] > details > form {
    & > button[formmethod=dialog] {display: unset;}
    & > button[formmethod=dialog] + button {display: none;}
}
```
    )

gists = (await (
    "https://api.github.com/users/tonyfast/gists?page=" + pandas.RangeIndex(1, 5).astype(str)
).http.get()).explode().series().set_index("id")
gists.index.name = "gist_id"

we're going to quickly make a table of my gist from github.

files = gists.join(gists.files.apply(dict.values).apply(list).explode().series())

unravel all the gists in each gist payload.

time = files.columns[files.columns.str.endswith("_at")].tolist()
files[time] = files[time].apply(pandas.to_datetime, axis=1)
files = files.assign(timespan=files.updated_at - files.created_at)

convert times to their proper types and substract them so out dataframe contains integers, datetimes, and timedeltas, types with numerical representations.

files[cat] = files[cat := "language type".split()].astype("category")
df = files[time + "timespan filename description language type size".split()]

now are going to add a few lines of css that uses the --ratio variables in each numerical cell to produce visual representation of the cell value in the larger distribution. it is really interesting how we start to create the cell border form when we do this. at a glance we can visually observe relative positions of values in a column. further, scrolling creates really interesting movement that provides user controlled animation.

notebooks = (
    await files[files.language.eq("Jupyter Notebook")].raw_url.http.get()
).apply(json.loads).series()

read the notebooks and separate out the cells, outputs, and display objects.

cells = notebooks.cells.enumerate("cell").series()
outputs = cells.outputs[cells.outputs.fillna("").astype(bool)].enumerate("display").series()
display_data = outputs['data'].dropna().series()

df = (df.join(cells.groupby("gist_id").id.count().rename("cells"))
      .join(outputs.groupby("gist_id")["data"].count().rename("outputs")))
from nbconvert_a11y.table import new; import nbconvert_a11y
table = df[
    "filename description language type created_at updated_at timespan size cells outputs".split()
].sort_values("created_at", ascending=False).table(id="gist").style(

dialog > details >  form > button[formmethod=dialog] {
    display: none;
}
dialog[open] > details > form {
    & > button[formmethod=dialog] {display: unset;}
    & > button[formmethod=dialog] + button {display: none;}
}



)
%%    
    table.style(
```css
dialog > form > button[formmethod=dialog] {
    display: none;
}
dialog[open] > form {
    & > button[formmethod=dialog] {display: unset;}
    & > button[formmethod=dialog] + button {display: none;}
}
td {
    --marker-width: .2ch;
    background: linear-gradient(90deg, 
                                rgba(255, 0, 0, 0) 0% calc(var(--ratio) * 100% - var(--marker-width)),
                                rgba(255, 0, 0, 1) calc(var(--ratio) * 100% - var(--marker-width)) calc(var(--ratio) * 100% + .5 * var(--marker-width)),
                                rgba(255, 0, 0, 0) calc(var(--ratio) * 100% + var(--marker-width)) 100%
                               );
    background-size: 100% 50%;
    background-position-y: 25%;
    background-repeat: no-repeat;
    &:empty::before {
        content: "nan";
    }
}
&, table, tr 
    background: var(--bg, white);
    color: var(--fg, black);
}
```
    )
    table.container.details.attrs.update(open="")
    table.container.form.append(
        table_filters := new(
            "details", new("summary", "filters"), open=""
        )
    )
    table.container.form.append(
        table_visibility := new(
            "details", new("summary", "visibility"), open=""
        )
    )

table.style(

dialog > form > button[formmethod=dialog] {
    display: none;
}
dialog[open] > form {
    & > button[formmethod=dialog] {display: unset;}
    & > button[formmethod=dialog] + button {display: none;}
}
td {
    --marker-width: .2ch;
    background: linear-gradient(90deg, 
                                rgba(255, 0, 0, 0) 0% calc(var(--ratio) * 100% - var(--marker-width)),
                                rgba(255, 0, 0, 1) calc(var(--ratio) * 100% - var(--marker-width)) calc(var(--ratio) * 100% + .5 * var(--marker-width)),
                                rgba(255, 0, 0, 0) calc(var(--ratio) * 100% + var(--marker-width)) 100%
                               );
    background-size: 100% 50%;
    background-position-y: 25%;
    background-repeat: no-repeat;
    &:empty::before {
        content: "nan";
    }
}
&, table, tr 
    background: var(--bg, white);
    color: var(--fg, black);
}



)
table.container.details.attrs.update(open="")
table.container.form.append(
    table_filters := new(
        "details", new("summary", "filters"), open=""
    )
)
table.container.form.append(
    table_visibility := new(
        "details", new("summary", "visibility"), open=""
    )
)
%%
## creating the column filters

any numeric column, integers, floats, time can be used as a filter for the table.
we build those for


        bounds = []
1. separate the numeric values and make them `input[type=number]`

        numeric = table.object.dtypes.apply(nbconvert_a11y.outputs.is_number)
        stats = table.stats[table.stats.columns[numeric]].T.set_index(table.object.columns[numeric])
        ranges = stats.bs4.input(type="number")
        ranges.bs4.attrs(stats).pipe(bounds.append)

1. separate the timedeltas values and make them `input[type=number]`. preprocessing is different than numbers

        # missing units
        timedeltas = table.object.dtypes.apply(isinstance, args=(numpy.dtypes.TimeDelta64DType,))
        stats = table.stats[table.stats.columns[timedeltas]].T.set_index(table.object.columns[timedeltas]).map(pandas.Timedelta.total_seconds)
        ranges = stats.bs4.input(type="number")
        ranges.bs4.attrs(stats).pipe(bounds.append)

1. separate the times and make them `input[type=datetime-local]`


        times = table.object.dtypes.apply(isinstance, args=(pandas.core.dtypes.dtypes.DatetimeTZDtype,))
        stats = table.stats[table.stats.columns[times]].T.set_index(table.object.columns[times]).bs4.to_attribute()
        ranges = stats.bs4.input(type="datetime-local")
        ranges.bs4.attrs(stats).pipe(bounds.append)

1. make the classes filterable

        classes = table.object.dtypes.apply(
            isinstance, args=(pandas.core.dtypes.dtypes.CategoricalDtype,)
        )
        classes = table.object.dtypes[classes].apply(
            operator.attrgetter("categories")
        ).explode().apply(
            lambda s: new("option", s, value=s, selected="")
        ).groupby(pandas.Grouper(level=0)).apply(
            lambda s: new("select", *s, multiple="")
        ).to_frame("classes")
        bounds.append(classes)

1. rejoin them back with the original container

        table_filters.append(
            (ranges := pandas.concat(bounds).reindex(table.object.columns)
             .rename_axis(index="column")
             .dropna(how="all", axis=0).fillna("").table()
            ).table
        )

creating the column filters

any numeric column, integers, floats, time can be used as a filter for the table. we build those for

    bounds = []

  1. separate the numeric values and make them input[type=number]

     numeric = table.object.dtypes.apply(nbconvert_a11y.outputs.is_number)
 stats = table.stats[table.stats.columns[numeric]].T.set_index(table.object.columns[numeric])
 ranges = stats.bs4.input(type="number")
 ranges.bs4.attrs(stats).pipe(bounds.append)

  2. separate the timedeltas values and make them input[type=number]. preprocessing is different than numbers

     # missing units
 timedeltas = table.object.dtypes.apply(isinstance, args=(numpy.dtypes.TimeDelta64DType,))
 stats = table.stats[table.stats.columns[timedeltas]].T.set_index(table.object.columns[timedeltas]).map(pandas.Timedelta.total_seconds)
 ranges = stats.bs4.input(type="number")
 ranges.bs4.attrs(stats).pipe(bounds.append)

  3. separate the times and make them input[type=datetime-local]

     times = table.object.dtypes.apply(isinstance, args=(pandas.core.dtypes.dtypes.DatetimeTZDtype,))
 stats = table.stats[table.stats.columns[times]].T.set_index(table.object.columns[times]).bs4.to_attribute()
 ranges = stats.bs4.input(type="datetime-local")
 ranges.bs4.attrs(stats).pipe(bounds.append)

  4. make the classes filterable

     classes = table.object.dtypes.apply(
     isinstance, args=(pandas.core.dtypes.dtypes.CategoricalDtype,)
 )
 classes = table.object.dtypes[classes].apply(
     operator.attrgetter("categories")
 ).explode().apply(
     lambda s: new("option", s, value=s, selected="")
 ).groupby(pandas.Grouper(level=0)).apply(
     lambda s: new("select", *s, multiple="")
 ).to_frame("classes")
 bounds.append(classes)

  5. rejoin them back with the original container

     table_filters.append(
     (ranges := pandas.concat(bounds).reindex(table.object.columns)
      .rename_axis(index="column")
      .dropna(how="all", axis=0).fillna("").table()
     ).table
 )
%%
## create visibility toggles

1. make an dataframe of `input[type=radio]` from an enum and the current columns in the table we are building.

        visibility = DataFrame(
            None,
            Index(
                Series(nbconvert_a11y.options.Presentation.__members__.keys()).bs4.th(), name="visibility"
            ),
            Series(table.table.thead.tr.select("th")).bs4.pop("aria-sort").apply(copy.copy)
        ).bs4.input(type="radio")

1. `aria-label` for now. need to add ids to the column headers to make `aria-labelledby` work

        visibility.bs4.attrs({
            "aria-label": 
            visibility.index.to_series().apply(getattr, args=("string",)).values[:, None] + " " + 
                  visibility.columns.to_series().apply(getattr, args=("string",)).values[None, :]

        })


1. give the same name to radio button row so we get arrow key support

        for i, (k, v) in enumerate(visibility.items()):
            v.bs4.attrs(name=F"visibility-{i}")
1. check the first ones to initialize

        visibility.iloc[0, :].bs4.attrs(checked="")
1.  the columns as the row index makes the preferred keyboard pattern for navigation with native html this is a better reference.

        visibility = visibility.T.rename_axis(index="column").table()




1. rejoin the visibility grid with the original container

        table_visibility.append(visibility.table)

create visibility toggles

  1. make an dataframe of input[type=radio] from an enum and the current columns in the table we are building.

     visibility = DataFrame(
     None,
     Index(
         Series(nbconvert_a11y.options.Presentation.__members__.keys()).bs4.th(), name="visibility"
     ),
     Series(table.table.thead.tr.select("th")).bs4.pop("aria-sort").apply(copy.copy)
 ).bs4.input(type="radio")

  2. aria-label for now. need to add ids to the column headers to make aria-labelledby work

     visibility.bs4.attrs({
     "aria-label": 
     visibility.index.to_series().apply(getattr, args=("string",)).values[:, None] + " " + 
           visibility.columns.to_series().apply(getattr, args=("string",)).values[None, :]

 })

  3. give the same name to radio button row so we get arrow key support

     for i, (k, v) in enumerate(visibility.items()):
     v.bs4.attrs(name=F"visibility-{i}")

  4. check the first ones to initialize

     visibility.iloc[0, :].bs4.attrs(checked="")

  5. the columns as the row index makes the preferred keyboard pattern for navigation with native html this is a better reference.

    visibility = visibility.T.rename_axis(index="column").table()

  6. rejoin the visibility grid with the original container

     table_visibility.append(visibility.table)
    table
gist_id filename description language type created_at updated_at timespan size cells outputs
90c41d4994f75c594db804aeba56fc26 first_and_second_laws_of_thermodynamics.ipynb a notebook with revised literate pidgy cells for the first part of thermodyamics course Jupyter Notebook text/plain 20469 16.0 7.0
aa3b16c5a284150e3d727a843b6cefec axe_types.py Python application/x-python 8442
713ae6c57602c0f85d011421b20d5ea0 BinaryExamples.ipynb a revision of a pycalphad documentation page as literate program. Jupyter Notebook text/plain 352154 0.0 6.0
c004044b4fe641735031ecf2069cf595 aom.json an exported accessibility tree from https://iota-school.github.io/notebooks-for-all/branch/extent/exports/html/lorenz-executed-smol.html#14 JSON application/json 52330
e17946facd998a931527467d646cc822 README.md extensible notebook schemas Markdown text/markdown 3044
99303420edc6021bbeeaa96c522469a1 ummm.py Python application/x-python 266
c8e19ffb2b85f5bf6f5a4b6e76c327cd abc.py abc of python modules Python application/x-python 469
4e33098a9f3a50805ad99d08e96825a6 graph shit.py Python application/x-python 4213
243d7758d659c406f6e12cbd7b045384 hide_empty_outputs.html a selection to hide empty jupyterlab output cells HTML text/html 90
42612252e0ed0df02f811932becd8241 2022-12-01-1.ipynb Jupyter Notebook text/plain 4956 8.0 0.0
abc7748fcc84177b59156d156132aabb lua.py Python application/x-python 408
d9842e69957895a883203101d32053c5 2022-09-18-static-notebook-tags.ipynb Jupyter Notebook text/plain 21687 16.0 13.0
d3aa6592100eb30d7ec9efcb3b3efb06 details-summary.ipynb using two html standards to hide notebook content Jupyter Notebook text/plain 5841 8.0 1.0
146153f37825c9a0d078027c141823c5 2022-08-05-iris-demo.ipynb a notebook demo'ing pandas/sklearn Jupyter Notebook text/plain 103473 14.0 12.0
ef00eeb40ce268ff68cc7d606437ac6c nbval triggers text/plain 84
1172d78b9a9a9738c8be759d304220e4 2022-07-17-shebang-magic.ipynb make shebangs a cell magic Jupyter Notebook text/plain 9822 14.0 11.0
3006d326442a9f4dd2ba61370cc2d37e 2022-07-16-larknb.ipynb a json grammar to parse notebooks into source code Jupyter Notebook text/plain 19876 15.0 8.0
3006d326442a9f4dd2ba61370cc2d37e _snippet.py a json grammar to parse notebooks into source code Python application/x-python 1844 15.0 8.0
d8ed2e33c5f70ad3b8340d4cf6cace21 2022-07-16-list-of-urls-py-md.ipynb an essay on the interfaces between md & py using a list of urls Jupyter Notebook text/plain 16111 9.0 8.0
6c236af7dcaa87fc012f31b720575dd7 2022-06-28-.ipynb Jupyter Notebook text/plain 22890 11.0 9.0
1ddcdce4eab431a8227be352d4c5a57b 2021-12-15-a11y-highlights.md.ipynb 2021 a11y highlights Jupyter Notebook text/plain 446103 34.0 17.0
df99b18e240d43c8b966aeb0a6e5a0d8 2021-12-15-a11y-highlights.md.ipynb 2021 a11y highlights Jupyter Notebook text/plain 436488 34.0 3.0
4cec811eb02f0107dc77f7da564657a2 2021-12-15-a11y-highlights.md.ipynb 2021 a11y highlights Jupyter Notebook text/plain 446103 34.0 17.0
bec93ba06cb420d58db3b1a4656e05b7 2021-12-15-a11y-highlights.md.ipynb 2021 a11y highlights Jupyter Notebook text/plain 444956 32.0 16.0
121f22a609d142992bf081bda7a875f7 2021-10-13-gannt.ipynb making gannt charts with pandas Jupyter Notebook text/plain 111977 19.0 8.0
e2cae138e4bb043cd277709ff45ba8a3 2021-10-11-colorizing.ipynb learning about dataframes using color Jupyter Notebook text/plain 54416 25.0 7.0
f34f1fc4454de5e61b6ed55a6736071c 2021-10-08-distributions-dataframe.ipynb create a dataframe from your python distributions Jupyter Notebook text/plain 22674 21.0 7.0
0d5cbee1f3b4aac205e9ddd407d0f4dc requirements.txt making calendars in pandas Text text/plain 15 43.0 11.0
0d5cbee1f3b4aac205e9ddd407d0f4dc 2021-10-03-pd-calendar.ipynb making calendars in pandas Jupyter Notebook text/plain 89667 43.0 11.0
cf3a246202f177500d47146303bdc235 2021-10-01-drum-machine.ipynb the faker - sample data generator - drum machine Jupyter Notebook text/plain 3755 9.0 2.0
2075d26303eb9e556f44a9f11c2a9f71 forge-assets.ipynb Jupyter Notebook text/plain 145602 0.0 42.0
745132fbaeddde27a882716d04f71d42 miniforge.ipynb Jupyter Notebook text/plain 5396 0.0 4.0
7c6f96d65cf4a9159a293159b138c2a2 requirements-df.ipynb Jupyter Notebook text/plain 9061 0.0 2.0
3d335e1da911f31930070a107c927b1d nono.ipynb Jupyter Notebook text/plain 11832 0.0 0.0
2169468ba38cb5cc033d0ef11afb5a84 discover-tests.ipynb Jupyter Notebook text/plain 2837 0.0 1.0
2169468ba38cb5cc033d0ef11afb5a84 better.py Python application/x-python 265 0.0 1.0
bacfb0c2d81b1ceba7794385a7882049 nox-in-the-notebook.ipynb using nox in an interactive jupyter session. Jupyter Notebook text/plain 3480 0.0 1.0
044ded6a7acadc56430e8d36c1f77cb6 _.py ensureconda logic Python application/x-python 335
56fc6cab7e7ce145c963d9312da295df typer-chain.ipynb Jupyter Notebook text/plain 3736 0.0 1.0
56fc6cab7e7ce145c963d9312da295df typer-doit.ipynb Jupyter Notebook text/plain 6072 0.0 1.0
f74eb42f2a998d8e428a752ceb0cb1d1 readme.md Markdown text/markdown 1697
7c6fbc36df87ef7c90bdfdca038e8f2e 2020-11-09-dgaf.ipynb Jupyter Notebook text/plain 3449 0.0
520e34998e3040d1fcda740543d7d6e0 postBuild https://gke.mybinder.org/v2/gist/tonyfast/520e34998e3040d1fcda740543d7d6e0/HEAD text/plain 9 0.0 25.0
520e34998e3040d1fcda740543d7d6e0 requirements.txt https://gke.mybinder.org/v2/gist/tonyfast/520e34998e3040d1fcda740543d7d6e0/HEAD Text text/plain 101 0.0 25.0
520e34998e3040d1fcda740543d7d6e0 dodo.py https://gke.mybinder.org/v2/gist/tonyfast/520e34998e3040d1fcda740543d7d6e0/HEAD Python application/x-python 211 0.0 25.0
520e34998e3040d1fcda740543d7d6e0 gcxs_blogpost.ipynb https://gke.mybinder.org/v2/gist/tonyfast/520e34998e3040d1fcda740543d7d6e0/HEAD Jupyter Notebook text/plain 370084 0.0 25.0
56db68b91e395c14a9fd8e0ea8cb024e gists_sqlite.ipynb https://mybinder.org/v2/gist/tonyfast/56db68b91e395c14a9fd8e0ea8cb024e/HEAD Jupyter Notebook text/plain 14877 0.0 11.0
56db68b91e395c14a9fd8e0ea8cb024e postBuild https://mybinder.org/v2/gist/tonyfast/56db68b91e395c14a9fd8e0ea8cb024e/HEAD text/plain 68 0.0 11.0
f5e6d43ba1704482e13c0209c9987691 readme.md Markdown text/markdown 133 0.0 4.0
f5e6d43ba1704482e13c0209c9987691 requirements.txt Text text/plain 90 0.0 4.0
f5e6d43ba1704482e13c0209c9987691 rich-template.ipynb Jupyter Notebook text/plain 119962 0.0 4.0
193202c1a41c57bb98a617e34486ac53 Untitled6.ipynb Jupyter Notebook text/plain 109382 0.0 26.0
ccb1134bbbcf63a59620dfaa716a1136 ics_example.ipynb Jupyter Notebook text/plain 3734 0.0 1.0
a699b67a4c6625a6c0b4eb7eb06d3bff states.py Python application/x-python 1440 0.0 1.0
a699b67a4c6625a6c0b4eb7eb06d3bff test_states.ipynb Jupyter Notebook text/plain 3294 0.0 1.0
cfb55f41f5452ef33ec6fbb4e0bda991 doctest-myst.ipynb Jupyter Notebook text/plain 5731 0.0 1.0
357b09758b5fb0f31fd2c0e7f7cf3967 readme.ipynb Jupyter Notebook text/plain 1117725 0.0 2.0
e0468f0decb0a454d8fae4de8511d794 df-df-api.ipynb Jupyter Notebook text/plain 22500 0.0 10.0
292377ff60690bff0bf866e93a36eaf2 reprish.ipynb Jupyter Notebook text/plain 6115 0.0 9.0
c72d948631f3d38702d2876942ceb57e custom_repr.ipynb Jupyter Notebook text/plain 62577 0.0 21.0
min 6 0.000 0.000
max 17544796 56.000 43.000
controls

filters
column min max classes
language
type
created_at
updated_at
timespan
size
cells
outputs
visibility
column visual nonvisual none
gist_id
filename
description
language
type
created_at
updated_at
timespan
size
cells
outputs 
%%
```css
th[aria-sort]::after {
    content: attr(aria-sort);
    font-weight: 100;
}
th:empty::before, td:empty::before {
    content: "nan";
}
[data-jp-theme-name="JupyterLab Dark"] section,
[data-jp-theme-name="JupyterLab Dark"] table,
[data-jp-theme-name="JupyterLab Dark"] tr {
    --fg: white; --bg: black;
}
a[href="#demo"] {
    height: 200px;
    background: -moz-element(#gist);
}
```

th[aria-sort]::after {
    content: attr(aria-sort);
    font-weight: 100;
}
th:empty::before, td:empty::before {
    content: "nan";
}
[data-jp-theme-name="JupyterLab Dark"] section,
[data-jp-theme-name="JupyterLab Dark"] table,
[data-jp-theme-name="JupyterLab Dark"] tr {
    --fg: white; --bg: black;
}
a[href="#demo"] {
    height: 200px;
    background: -moz-element(#gist);
}