documentation as software and data¤
computational literature like documentation is presented with antithetical distribution practices like static sites and pdf. the idea of static computational literature pushes against its real potential to give visitors agency over their content. we see developments towards more interactive computational literature with projects like jupyterlite, pyodide, and pyscript. these approaches providing slightly more agency to interact with code, but this is only part of the interaction we could imagine.
documentation is valuable when people spend time with it. different folks might find different parts of documentation valuable as different places in their career. documentation has less to do with aesthetics than with spending quality time learning and practicing. a significant part of the documentation experience is how much time your audience is willing to put into participating.
the most interactive way to enjoy information is in computational notebooks. if we can capture documentation as data we can expose multitudes of interaction capabilities than statically generated sites anca acheive.
the following presents that concept of nobook which treats documentation as data.
it lets the learners discover the documentation they desire. i imagine it could make better chat assistants if you are into that kind of thing.
beyond individual documentation¤
a key success criterion for this is approach is that we ought to be able to operate on multiple documentations at once and reuse existing configurations when possible.
(shell := get_ipython()).execution_count > 2 and shell.reset(1,1)
import nbconvert_a11y.builder
def md(frame): display({"text/markdown": "\n".join(frame)}, raw=True)
import midgy
build my personal blog¤
a goal is to build multiple sites at once. the first we build is my personal blog. i do a lot of weird shit in documents so my works are a good stress tests.
docs = nbconvert_a11y.builder.Contents.Config.new(
root="../../",
include=["**/[0-9][0-9][0-9][0-9]-*.ipynb"],
exclude=""".ipynb_checkpoints\n../../site/run""")
docs = (await docs.discover()).head(20)
tfast = docs = await (await docs.expand()).compact()
docs.head(2)
build the mast space telescope notebooks¤
https://spacetelescope.github.io/mast_notebooks/
docs = nbconvert_a11y.builder.Contents.Config.new(
root="mast_notebooks/", toc="_toc.yml",
kind=nbconvert_a11y.builder.ConfigKind.jb)
docs = (await docs.discover()).head()
docs = await (await docs.expand()).compact()
docs.head(2)
manipulating the documentation¤
we'll use the dataframes from our previous cells to demonstrate what we can do with documentation frames.
%%
### preview a webpage
tfast.html.tail(2).display.iframe().display()
preview a webpage¤
tfast.html.tail(2).display.iframe().display()
%%
### how many cells?
tfast.cells.cell_type.value_counts().to_frame().T
how many cells?¤
tfast.cells.cell_type.value_counts().to_frame().T
%%
### search cell contents
find all the cells containing a reference to pandas.
tfast.cells[tfast.cells.source.str.contains("pandas")][["source"]].sample(6)
we can invoke more complicated regular expressions to extract information.
search cell contents¤
find all the cells containing a reference to pandas.
tfast.cells[tfast.cells.source.str.contains("pandas")][["source"]].sample(6)
we can invoke more complicated regular expressions to extract information.
%%
### what markdown intermediates can we access?
#### access markdown referneces across documents
markdown provides syntax for link references. these values return from markdown parsing
as structured data, and we'll show some really good information in there.
refs = tfast["env"].series().dropna()["references"].apply(dict.values).explode().series()
refs.apply(
lambda x: F"* [{x.title or x.href}]({x.href})", axis=1
).pipe(md)
what markdown intermediates can we access?¤
access markdown referneces across documents¤
markdown provides syntax for link references. these values return from markdown parsing as structured data, and we'll show some really good information in there.
refs = tfast["env"].series().dropna()["references"].apply(dict.values).explode().series()
refs.apply(
lambda x: F"* [{x.title or x.href}]({x.href})", axis=1
).pipe(md)
- hackmd for the jupyter community call
- updating discourse announcement for the jupyter community calls.
- https://en.wikipedia.org/wiki/Pidgin
- https://en.wikipedia.org/wiki/Metalanguage
- https://deathbeds.github.io/midgy/language/basics/
- #
- fperez's polyglot juypyter demo
- https://web.archive.org/web/20220510083647/http://blog.fperez.org/2013/04/literate-computing-and-computational.html
- http://worrydream.com/Tangle/
- https://www.youtube.com/watch?v=PUv66718DII
- https://docs.github.com/en/rest/gists?apiVersion=2022-11-28
- the mermaid.js documentation
- announcement that github renders mermaid.js
- special code fences for jupyterlab markdown
- mkdocs configuration for mermaid
- https://forum.graphviz.org/t/github-adding-support-for-mermaid-diagrams/998
- https://blog.jupyter.org/improving-the-accessibility-of-jupyter-6c695db518d3
%%
#### access images in the mimebundle
docs.bundles[["image/png", "image/jpeg"]].stack().to_frame("b64").apply(
lambda x: F"", axis=1
).pipe(md)
access images in the mimebundle¤
docs.bundles[["image/png", "image/jpeg"]].stack().to_frame("b64").apply(
lambda x: F"", axis=1
).pipe(md)
