when markdown and python collide¤
a story about literacy written in the literate programming style of pidgy
notebooks commonly communicate with two languages: markdown & python. what happen when we dissolve the boundaries between markdown & python or language & code entirely.
class STATE:
NB = "__file__" not in globals() and __name__ == "__main__"
SCRIPT = "__file__" in globals() and __name__ != "__main__"
MAIN = "__file__" in globals() and __name__ == "__main__"
LITE = __import__("sys").platform == "emscripten"
# execute this code in lite.
if STATE.LITE:
try:
%reload_ext pidgy
except ModuleNotFoundError:
import micropip
await micropip.install("pidgy importnb qrcode midgy ipywidgets".split(), pre=True)
import pidgy, pathlib, IPython
shell = IPython.get_ipython()
if STATE.NB:
%reload_ext pidgy
shell.displays_manager.template_cls = pidgy.weave.IPythonHtml
<style>
img {
height: 500px !important;
}
.jupyter-wrapper .jp-Cell-inputWrapper {
display: none;
}
</style>
<script src="https://cdn.jsdelivr.net/npm/mermaid/dist/mermaid.min.js"></script>
<script>mermaid.initialize({startOnLoad:true, securityLevel: "loose"}, ".mermaid");</script>
## literate computing with literary machines using literate programming for mass computational literacy
literacy rings through the history of computer science.
in Annette Vee's [Understanding Computer Programming as a Literacy] we get
a high level view of literacy through history and in the our little microcosm of computer programming.
> We compare mass ability to read and write software with mass literacy, and predict equally pervasive changes to society.
history =\
```mermaid
gantt
title Literacy in computer programming
dateFormat YYYY
axisFormat %Y
section moderninity
understanding media :mcluhan, 1964, 2022
mother of all demos :englebert, 1968, 2022
section post moderninity
literate programming :knuth, 1984, 2022
literary machines :nelson, 1987, 2022
www :tbl, 1989, 2022
section post web
computer programming for everybody :cp4e, 1999, 2022
ipython :fperez, 2001, 2022
markdown :gruber, 2004, 2022
ipython notebooks :ipynb, 2011, 2022
anaconda :ana, 2012, 2022
literate computing :lc, 2013, 2022
understanding computer programming as a literacy :vee, 2013, 2022
the birth & death of javascript: js, 2014, 2022
big split :split, 2015, 2022
jupyterlab :lab, 2018, 2022
jupyterlite :lab, 2021, 2022
click vee href "https://licsjournal.org/index.php/LiCS/article/view/794/608"
click knuth href "http://www.literateprogramming.com/knuthweb.pdf"
click nelson href "https://monoskop.org/images/b/be/Nelson_Ted_Literary_Machines_c1987_chs_0-1.pdf"
click lc href "https://web.archive.org/web/20220510083647/http://blog.fperez.org/2013/04/literate-computing-and-computational.html"
click cp4e href "https://www.python.org/doc/essays/cp4e/"
```
shell.displays_manager.template_cls = pidgy.weave.IPythonMarkdown
literate computing with literary machines using literate programming for mass computational literacy¤
literacy rings through the history of computer science. in Annette Vee's [Understanding Computer Programming as a Literacy] we get a high level view of literacy through history and in the our little microcosm of computer programming.
> We compare mass ability to read and write software with mass literacy, and predict equally pervasive changes to society.
history =\
mermaid
gantt
title Literacy in computer programming
dateFormat YYYY
axisFormat %Y
section moderninity
understanding media :mcluhan, 1964, 2022
mother of all demos :englebert, 1968, 2022
section post moderninity
literate programming :knuth, 1984, 2022
literary machines :nelson, 1987, 2022
www :tbl, 1989, 2022
section post web
computer programming for everybody :cp4e, 1999, 2022
ipython :fperez, 2001, 2022
markdown :gruber, 2004, 2022
ipython notebooks :ipynb, 2011, 2022
anaconda :ana, 2012, 2022
literate computing :lc, 2013, 2022
understanding computer programming as a literacy :vee, 2013, 2022
the birth & death of javascript: js, 2014, 2022
big split :split, 2015, 2022
jupyterlab :lab, 2018, 2022
jupyterlite :lab, 2021, 2022
click vee href "https://licsjournal.org/index.php/LiCS/article/view/794/608"
click knuth href "http://www.literateprogramming.com/knuthweb.pdf"
click nelson href "https://monoskop.org/images/b/be/Nelson_Ted_Literary_Machines_c1987_chs_0-1.pdf"
click lc href "https://web.archive.org/web/20220510083647/http://blog.fperez.org/2013/04/literate-computing-and-computational.html"
click cp4e href "https://www.python.org/doc/essays/cp4e/"
shell.displays_manager.template_cls = pidgy.weave.IPythonMarkdown
donald knuth's diagram for literate programming¤
# tangle (code) and weave (display)
knit =\
```mermaid
flowchart LR
web-- tangle ---pas
pas-- pascal ---rel
web-- weave ---tex
tex-- TEX ---dvi
```
tangle (code) and weave (display)¤
knit =\
mermaid
flowchart LR
web-- tangle ---pas
pas-- pascal ---rel
web-- weave ---tex
tex-- TEX ---dvi
## `midgy/pidgy` family of input kernels in `IPython`
literate inputs with markdown and python as the document and programming languages
<div style="font-size: 2rem;">
import midgy as tangle, pidgy as weave
</div>
we'll explore a little program in markdown inside a bigger notebook program. hold tight.
midgy/pidgy family of input kernels in IPython¤
literate inputs with markdown and python as the document and programming languages
we'll explore a little program in markdown inside a bigger notebook program. hold tight.
### how does `IPython` python?
* `shell.kernel.do_execute`
* `shell.run_cell`
* tangle
* `shell.transform_cell`
* `shell.compile.ast_parse`
* `shell.transform_ast`
* `shell.run_ast_nodes`
* weave
* if `shell.ast_node_interactivity`
* `shell.display_formatter.format`
`IPython` is very hackable! that is what makes it fun along with all the development happening in python.
how does IPython python?¤
shell.kernel.do_executeshell.run_cell- tangle
shell.transform_cellshell.compile.ast_parseshell.transform_astshell.run_ast_nodes
- weave
- if
shell.ast_node_interactivity shell.display_formatter.format
- if
- tangle
IPython is very hackable! that is what makes it fun along with all the development happening in python.
%%file a-little-markdown-program.md
this markdown file is a program!
δϵατκ=\
death to pseudo code, long live pseudo code
if __name__ == "__main__":
print(δϵατκ.upper())
we can run this like a script. WTF!
!midgy a-little-markdown-program.md
import midgy.loader
with midgy.loader.Markdown() as loader:
import a_little_markdown_program
a_little_markdown_program, a_little_markdown_program.δϵατκ
import midgy.loader
with midgy.loader.Markdown() as loader:
import a_little_markdown_program
a_little_markdown_program, a_little_markdown_program.δϵατκ
`a_little_markdown_program` is a module like any other
`{{a_little_markdown_program}}`
🐇and we even generated a compiled version of the markdown source🎩
if not STATE.LITE:
assert next(
(pathlib.Path(a_little_markdown_program.__file__).parent / "__pycache__").glob("a-little-markdown-program.*.pyc")
),\
verifies there is a compiled file.
a_little_markdown_program is a module like any other
<module '="" 'a-little-markdown-program'="" a-little-markdown-program.md'="" documents="" from="" home="" oct="" tbone="" tonyfast="" xxii="">
🐇and we even generated a compiled version of the markdown source🎩
if not STATE.LITE:
assert next(
(pathlib.Path(a_little_markdown_program.__file__).parent / "__pycache__").glob("a-little-markdown-program.*.pyc")
),\
verifies there is a compiled file.
## [literate computing widgets and `IPython` interactive displays](2022-10-21-pidgy-displays.ipynb)
literate computing widgets and IPython interactive displays¤
## reusing and importing notebooks
we can import this presentation.
import midgy.loader
with midgy.loader.Markdown(extensions=[".ipynb", ".md"]):
import _022_10_21_markdown_future
"__file__" not in locals() and display(
_022_10_21_markdown_future,
_022_10_21_markdown_future.a_little_markdown_program,
_022_10_21_markdown_future.a_little_markdown_program.δϵατκ
)
gantt
title Literacy in computer programming
dateFormat YYYY
axisFormat %Y
section moderninity
understanding media :mcluhan, 1964, 2022
mother of all demos :englebert, 1968, 2022
section post moderninity
literate programming :knuth, 1984, 2022
literary machines :nelson, 1987, 2022
www :tbl, 1989, 2022
section post web
computer programming for everybody :cp4e, 1999, 2022
ipython :fperez, 2001, 2022
markdown :gruber, 2004, 2022
ipython notebooks :ipynb, 2011, 2022
anaconda :ana, 2012, 2022
literate computing :lc, 2013, 2022
understanding computer programming as a literacy :vee, 2013, 2022
the birth & death of javascript: js, 2014, 2022
big split :split, 2015, 2022
jupyterlab :lab, 2018, 2022
jupyterlite :lab, 2021, 2022
click vee href "https://licsjournal.org/index.php/LiCS/article/view/794/608"
click knuth href "http://www.literateprogramming.com/knuthweb.pdf"
click nelson href "https://monoskop.org/images/b/be/Nelson_Ted_Literary_Machines_c1987_chs_0-1.pdf"
click lc href "https://web.archive.org/web/20220510083647/http://blog.fperez.org/2013/04/literate-computing-and-computational.html"
click cp4e href "https://www.python.org/doc/essays/cp4e/"
flowchart LR
web-- tangle ---pas
pas-- pascal ---rel
web-- weave ---tex
tex-- TEX ---dvireusing and importing notebooks¤
we can import this presentation.
import midgy.loader
with midgy.loader.Markdown(extensions=[".ipynb", ".md"]):
import _022_10_21_markdown_future
"__file__" not in locals() and display(
_022_10_21_markdown_future,
_022_10_21_markdown_future.a_little_markdown_program,
_022_10_21_markdown_future.a_little_markdown_program.δϵατκ
)
### what about reusing that diagram
display(
IPython.display.Markdown(_022_10_21_markdown_future.history),
IPython.display.Markdown(_022_10_21_markdown_future.knit)
)
gantt
title Literacy in computer programming
dateFormat YYYY
axisFormat %Y
section moderninity
understanding media :mcluhan, 1964, 2022
mother of all demos :englebert, 1968, 2022
section post moderninity
literate programming :knuth, 1984, 2022
literary machines :nelson, 1987, 2022
www :tbl, 1989, 2022
section post web
computer programming for everybody :cp4e, 1999, 2022
ipython :fperez, 2001, 2022
markdown :gruber, 2004, 2022
ipython notebooks :ipynb, 2011, 2022
anaconda :ana, 2012, 2022
literate computing :lc, 2013, 2022
understanding computer programming as a literacy :vee, 2013, 2022
the birth & death of javascript: js, 2014, 2022
big split :split, 2015, 2022
jupyterlab :lab, 2018, 2022
jupyterlite :lab, 2021, 2022
click vee href "https://licsjournal.org/index.php/LiCS/article/view/794/608"
click knuth href "http://www.literateprogramming.com/knuthweb.pdf"
click nelson href "https://monoskop.org/images/b/be/Nelson_Ted_Literary_Machines_c1987_chs_0-1.pdf"
click lc href "https://web.archive.org/web/20220510083647/http://blog.fperez.org/2013/04/literate-computing-and-computational.html"
click cp4e href "https://www.python.org/doc/essays/cp4e/"
flowchart LR
web-- tangle ---pas
pas-- pascal ---rel
web-- weave ---tex
tex-- TEX ---dviwhat about reusing that diagram¤
display(
IPython.display.Markdown(_022_10_21_markdown_future.history),
IPython.display.Markdown(_022_10_21_markdown_future.knit)
)
i'm not saying importing markdown or notebooks is a good idea.
coffeescript died didn't it?
sometimes it is too soon.
it can be done and some folks might thrive working that way.
i'm not saying importing markdown or notebooks is a good idea. coffeescript died didn't it? sometimes it is too soon. it can be done and some folks might thrive working that way.
### an inclusive future for code
in this demo, we used markdown and notebooks for source.
literate programs explicitly define a document and program language.
notebooks are literate programs.
there are times when when code and narrative are inseparable.
```future
docker up my-paper.pdf # can a paper contain containers?
importpdf my-paper --input-file my-data.csv # could you run a pdf with your data?
with importdocx():
import my_word_document # https://nbviewer.org/github/deathbeds/deathbeds.github.io/blob/master/deathbeds/2019-03-08-say-word.ipynb
```
an inclusive future for code¤
in this demo, we used markdown and notebooks for source.
literate programs explicitly define a document and program language. notebooks are literate programs. there are times when when code and narrative are inseparable.
docker up my-paper.pdf # can a paper contain containers?
importpdf my-paper --input-file my-data.csv # could you run a pdf with your data?
with importdocx():
import my_word_document # https://nbviewer.org/github/deathbeds/deathbeds.github.io/blob/master/deathbeds/2019-03-08-say-word.ipynb
## the power of markdown
often notebooks or markdown are means not an ends.
* how many agendas have we made in `hackmd`?
* how many `README`s guided our way?
* how much good code is lost in notebooks?
the power of markdown¤
often notebooks or markdown are means not an ends.
- how many agendas have we made in
hackmd? - how many
READMEs guided our way? - how much good code is lost in notebooks?
def qr(url, size=8):
q = __import__("qrcode").QRCode(version=1, error_correction=1,
box_size=size, border=1, mask_pattern=None,)
q.add_data(url), q.make()
return q.make_image()
## what if the code was just the start something?
y'all, like all y'all can modify this document in jupyterlite on github pages.
<style>
[data-mime-type="image/png"] img {
max-width: 200px; max-height: 200px;
}
</style>
qr(
https://tonyfast.github.io/tonyfast/run/lab/index.html?path=xxii/oct/2022-10-21-markdown-future.ipynb&room=deathbeds-more-like-deft-breads
)
what if the code was just the start something?¤
y'all, like all y'all can modify this document in jupyterlite on github pages.
qr(
)