`pidgy` literate computing¤

you've heard of interactive computing, we'll call pidgy hyperactive computing.

> this presentation is written for the february 2023 jupyter community call.²

or it didn't happen

    # my pidgy practices always have me indenting cells.
    # this habit means code will be rendered as code in markdown.
    # https://nbviewer.org/github/deathbeds/deathbeds.github.io/blob/master/deathbeds/2018-08-03-A-case-for-indented-code.ipynb

    # this is a presentation about pidgy so we start by loading the extension.
    # there is a pidgy kernel for the hardcore. 
    # in that case we recommend the hybrid `.md.ipynb` extension.

    if LITE := (__import__("sys").platform == "emscripten"): # compatability for jupyterlite
        %pip install pandas pidgy matplotlib ipywidgets toolz

    # activate pidgy
    %reload_ext pidgy
    from toolz.curried import *

# my pidgy practices always have me indenting cells.
# this habit means code will be rendered as code in markdown.
# https://nbviewer.org/github/deathbeds/deathbeds.github.io/blob/master/deathbeds/2018-08-03-A-case-for-indented-code.ipynb

# this is a presentation about pidgy so we start by loading the extension.
# there is a pidgy kernel for the hardcore. 
# in that case we recommend the hybrid `.md.ipynb` extension.

if LITE := (__import__("sys").platform == "emscripten"): # compatability for jupyterlite
    %pip install pandas pidgy matplotlib ipywidgets toolz

# activate pidgy
%reload_ext pidgy
from toolz.curried import *

the `pidgy` polyglot metalanguage¤

pidgy is a pidgin of python programming and jinja templates embedded inside markdown. literate programming is that aim to craft documentation and code at the same time. literate computing is when we do this interactively, and weave live computing into the narrative.

<figure>
<figcaption>

the REPL overlayed with a [literate computing] workflow of tangle and weave.

</figcaption>

{% set repl %}
    input[markdown]--tangle\nread--&gt;IPython
    IPython--eval/template--&gt;jinja
    jinja--weave\nprint--&gt;output[markdown]
    output[markdown]-.loop-.-&gt;input
    html ---&gt; input; css ---&gt; input; javascript ---&gt; input
    julia ---&gt; magics; r---&gt; magics; fortran ---&gt; magics; magics ---&gt; IPython
{% endset %}
```mermaid
flowchart LR
{% for i, line in enumerate(repl.splitlines()) %}
{% if i &gt; extra.value %}%%{% endif %}{{line}}
{% endfor %}
```
<figcaption>mermaid graph syntax uses extended markdown syntax provided by <code>jupyterlab-markup</code></figcaption>
</figure>

        display(extra := IntSlider(6, min=1, max=6, description="show more languages")) # 6 lines in the graph

`pidgy` is interested in exploring the interfaces of languages in computational essays. each language provides extra syntax for telling your story. with markdown we can include html, css, and javascript; code blocks can include more languages like mermaid. [IPython is our polyglot glue language][polyglot notebook]. 

[literate computing]: #
[polyglot notebook]: https://gist.github.com/fperez/5b49246af4e340c37549265a90894ce6 "fperez's polyglot juypyter demo"

[fperez lc]: https://web.archive.org/web/20220510083647/http://blog.fperez.org/2013/04/literate-computing-and-computational.html

the REPL overlayed with a [literate computing] workflow of tangle and weave.

flowchart LR



    input[markdown]--tangle\nread--&gt;IPython

    IPython--eval/template--&gt;jinja

    jinja--weave\nprint--&gt;output[markdown]

    output[markdown]-.loop-.-&gt;input

    html ---&gt; input; css ---&gt; input; javascript ---&gt; input

    julia ---&gt; magics; r---&gt; magics; fortran ---&gt; magics; magics ---&gt; IPython

mermaid graph syntax uses extended markdown syntax provided by jupyterlab-markup

    display(extra := IntSlider(6, min=1, max=6, description="show more languages")) # 6 lines in the graph

pidgy is interested in exploring the interfaces of languages in computational essays. each language provides extra syntax for telling your story. with markdown we can include html, css, and javascript; code blocks can include more languages like mermaid. IPython is our polyglot glue language.

inspecting and doing more with a keypress¤

Ctrl + I ⛔

translating markdown to python¤

the %%tangle magic tangles markdown to python code for previewing and debugging. this magic will help you learn pidgys indenting heuristics and language features provided by midgy.

%%tangle
we use the `%%tangle` magic interactively translate the document into code.

some `pidgy` features are shown below like:

* markdown docstrings

        def my_function():
functions with markdown docstrings

            ...
        ...

* defining blocks of markdown as variables

        my_url =\
https://api.github.com

"""we use the `%%tangle` magic interactively translate the document into code.

some `pidgy` features are shown below like:
    
* markdown docstrings"""

def my_function():
    """functions with markdown docstrings"""

    ...
...
        
"""* defining blocks of markdown as variables"""

my_url =\
"""https://api.github.com""";

an homage knuth's literate programming language¤

<figure>
<figcaption>

a mermaid homage to the dual usage of the WEB file format being translated to a document and programming language.

</figcaption> 

```mermaid
flowchart LR
    WEB--WEAVE--&gt;TEX--TeX--&gt;DVI
    WEB--TANGLE--&gt;PAS--PASCAL--&gt;REL
```
</figure>
<figure markdown="">
<figcaption>knuth's rational for choosing Pascal; the same motivation applies to python.</figcaption>

&gt; I chose PASCAL as the programming language because it has
received such widespread support from educational  in-
stitutions all over the world; it is not my favorite language for system programming, but it has become a
"second language" for so many programmers that it
provides an exceptionally effective medium of communication.
</figure>

a mermaid homage to the dual usage of the WEB file format being translated to a document and programming language.

flowchart LR
    WEB--WEAVE--&gt;TEX--TeX--&gt;DVI
    WEB--TANGLE--&gt;PAS--PASCAL--&gt;REL

knuth's rational for choosing Pascal; the same motivation applies to python.

> I chose PASCAL as the programming language because it has received such widespread support from educational in- stitutions all over the world; it is not my favorite language for system programming, but it has become a "second language" for so many programmers that it provides an exceptionally effective medium of communication.

## reactive `jinja2` templates

<label for="deck">open</label>
<button data-commandlinker-command="deck:toggle" id="deck"><code>jupyterlab-deck</code></button>

`pidgy` relies on a `midgy` to tangle markdown to code, it does not doing any work on displaying the input markdown. including the templating language is the innovation of `pidgy` that inlines live computation.


`pidgy` templates are asynchronous reactive display objects that place live computation directly into the narrative.

reactive `jinja2` templates¤

open

pidgy relies on a midgy to tangle markdown to code, it does not doing any work on displaying the input markdown. including the templating language is the innovation of pidgy that inlines live computation.

pidgy templates are asynchronous reactive display objects that place live computation directly into the narrative.

### worrydream's cookies demo[^inventing]

<!---![](https://media3.giphy.com/media/BsUORZkF3gBqg/giphy.gif "cookie monster happy as fuck eating cookies")--->
<blockquote cite="https://worrydream.com/Tangle/">

[Tangle] is a JavaScript library for creating reactive documents. Your readers can interactively explore possibilities, play with parameters, and see the document update immediately. Tangle is super-simple and easy to learn.

</blockquote>
<figure>
<figcaption>

the source code for the first [TangleJs][tangle] demo

</figcaption>

```html
When you eat <span class="TKAdjustableNumber" data-var="cookies"> cookies</span>,
you consume <span data-var="calories"> calories</span>. 
```

```javascript
var tangle = new Tangle(document, {
    initialize: function () { this.cookies = 3; },
    update:     function () { this.calories = this.cookies * 50; }
});

```
</figure>

[tangle]: http://worrydream.com/Tangle/
[inventing on principle]: https://www.youtube.com/watch?v=PUv66718DII
[^inventing]: many folks working on computational interfaces are inspired by Bret Victor's [inventing on principle]

worrydream's cookies demo³¤

[Tangle] is a JavaScript library for creating reactive documents. Your readers can interactively explore possibilities, play with parameters, and see the document update immediately. Tangle is super-simple and easy to learn.

the source code for the first [TangleJs][tangle] demo

When you eat <span class="TKAdjustableNumber" data-var="cookies"> cookies</span>,
you consume <span data-var="calories"> calories</span>.

var tangle = new Tangle(document, {
    initialize: function () { this.cookies = 3; },
    update:     function () { this.calories = this.cookies * 50; }
});

{% set calories = 50 %}<!--- we can shield variables from the global scope -->
When you eat {{cookies.value}} cookies,
you consume {{cookies.value * calories}} calories. 

<div hidden=""><!--indented code is python-->

        display(cookies := IntSlider(3, description="🍪"))
</div>

{{"🍪" * cookies.value}}


<figure>
<figcaption>

`pidgy` source code using html, markdown, jinja, and ipywidgets to create interactions

</figcaption>

```markdown
{% raw %}
{% set calories = 50 %}<!--- we can shield variables from the global scope -->
When you eat {{cookies.value}} cookies,
you consume {{cookies.value * calories}} calories. 

<div hidden=""><!--indented code is python-->

        display(cookies := IntSlider(3, description="🍪"))
</div>

{{"🍪" * cookies.value}}
{% endraw %}
```

</figure>

When you eat 3 cookies, you consume 150 calories.

🍪🍪🍪

`pidgy` source code using html, markdown, jinja, and ipywidgets to create interactions

{% set calories = 50 %}<!--- we can shield variables from the global scope -->
When you eat {{cookies.value}} cookies,
you consume {{cookies.value * calories}} calories. 

<div hidden=""><!--indented code is python-->

        display(cookies := IntSlider(3, description="🍪"))
</div>

{{"🍪" * cookies.value}}

### live data in your document

<details markdown="">
<summary>imports for the pandas matplotlib demo</summary>

    import pandas
    from pidgy import get_cell_id
    %matplotlib agg

</details>
<details markdown="" open="">
<summary>supporting methods for the interactive demo</summary>`

    @functools.lru_cache
    def get_gist(x, max=100):
[gather gist from the github api][gist].

        return pandas.read_json(F"https://api.github.com/users/{x}/gists?per_page={max}")

[gist]: https://docs.github.com/en/rest/gists?apiVersion=2022-11-28

    def tidy_gist(x):
explode the gist repsonse into a dataframe of gist files

        gists = get_gist(x).set_index("id")
        return gists.files.apply(compose_left(dict.values, list)).explode().apply(pandas.Series).join(gists)

</details>

live data in your document¤

imports for the pandas matplotlib demo

import pandas from pidgy import get_cell_id %matplotlib agg

supporting methods for the interactive demo

` @functools.lru_cache def get_gist(x, max=100): [gather gist from the github api][gist]. return pandas.read_json(F"https://api.github.com/users/{x}/gists?per_page={max}") [gist]: https://docs.github.com/en/rest/gists?apiVersion=2022-11-28 def tidy_gist(x): explode the gist repsonse into a dataframe of gist files gists = get_gist(x).set_index("id") return gists.files.apply(compose_left(dict.values, list)).explode().apply(pandas.Series).join(gists)

#### an application in a cell.

{% set gists = tidy_gist(_github_user.value) %}
{% set lang = gists.language.dropna().value_counts() %}

we've found information on {{gists.index.unique().shape[0]}} github gists, and {{len(gists)}} files, for @{{_github_user.value}}.
in this collection, there are {{len(lang)}} different languages included. their most common language is {{lang.index[0]}}.

{{gists.sample(2)}}

    display(
        _github_user := Text("tonyfast"), button := Button(description="submit"))
    button.on_click(lambda x, id=get_cell_id(): print(id) or shell.weave.displays[id].update())
{{lang.to_frame("ct").T}}

an application in a cell.¤

we've found information on 100 github gists, and 121 files, for @tonyfast. in this collection, there are 6 different languages included. their most common language is Jupyter Notebook.

	filename	type	language	raw_url	size	url	forks_url	commits_url	node_id	git_pull_url	...	files	public	created_at	updated_at	comments	user	comments_url	owner	truncated
id
3181cb04cad6e16a2add26c646893db7	moi.ipynb	text/plain	Jupyter Notebook	https://gist.githubusercontent.com/tonyfast/31...	4814	https://api.github.com/gists/3181cb04cad6e16a2...	https://api.github.com/gists/3181cb04cad6e16a2...	https://api.github.com/gists/3181cb04cad6e16a2...	MDQ6R2lzdDMxODFjYjA0Y2FkNmUxNmEyYWRkMjZjNjQ2OD...	https://gist.github.com/3181cb04cad6e16a2add26...	...	{'moi.ipynb': {'filename': 'moi.ipynb', 'type'...	True	2021-05-05 16:06:56+00:00	2021-05-05 16:07:29+00:00	0	NaN	https://api.github.com/gists/3181cb04cad6e16a2...	{'login': 'tonyfast', 'id': 4236275, 'node_id'...	False
82e8b9744c139a63febb1a3280e91cec	pluck-attachments.ipynb	text/plain	Jupyter Notebook	https://gist.githubusercontent.com/tonyfast/82...	4843	https://api.github.com/gists/82e8b9744c139a63f...	https://api.github.com/gists/82e8b9744c139a63f...	https://api.github.com/gists/82e8b9744c139a63f...	MDQ6R2lzdDgyZThiOTc0NGMxMzlhNjNmZWJiMWEzMjgwZT...	https://gist.github.com/82e8b9744c139a63febb1a...	...	{'collection-ouputs.ipynb': {'filename': 'coll...	True	2021-07-02 17:01:42+00:00	2021-07-05 17:52:46+00:00	0	NaN	https://api.github.com/gists/82e8b9744c139a63f...	{'login': 'tonyfast', 'id': 4236275, 'node_id'...	False

2 rows × 22 columns

display(
    _github_user := Text("tonyfast"), button := Button(description="submit"))
button.on_click(lambda x, id=get_cell_id(): print(id) or shell.weave.displays[id].update())

language	Jupyter Notebook	Python	Text	Markdown	HTML	JSON
ct	84	15	9	8	2	1

### et `voila`

{% if LITE %}
&gt; 😞 unforntunately, this part of the demo is not available on jupyter lite.
{% endif %}

<button %}="" %}disabled{%="" data-commandlinker-command="notebook:render-with-voila" endif="" if="" lite="" {%="">et <code>voila</code></button>

et `voila`¤

## wrap up

* this presentation had a number of incidental demos.
    * jupyterlab-deck
    * jupyterlab-markup
    * voila
    * mermaid
    * midgy


* we have on and off cells now that all of the sources are markdown 
* the template syntax is a powerful way to include python variables inline. the traditional notebook format only allows for block level literate programs.
* since literate programs are pidgin languages we can't expect them to be formalized. there are a lot of inconsistencies with the languages combined.
jinja2 template implementations vary. markdown renderers implementation vary. python keeps adding language features. 

{% set sz = 3 %}
{% set h = 60 %}
<form style="font-size: {{sz}}rem; line-height: {{h}}%;">
<label for="rara">🎗🎗🎗🎗🎗🎗🎗🎗</label><br/><button data-commandlinker-command="runmenu:restart-and-run-all" id="rara" style="font-size: {{sz}}rem; line-height: {{h}}%;">restart and run all</button><br/><label for="rara">or it didn't happen</label>
</form>

wrap up¤

this presentation had a number of incidental demos.
- jupyterlab-deck
- jupyterlab-markup
- voila
- mermaid
- midgy
we have on and off cells now that all of the sources are markdown
the template syntax is a powerful way to include python variables inline. the traditional notebook format only allows for block level literate programs.
since literate programs are pidgin languages we can't expect them to be formalized. there are a lot of inconsistencies with the languages combined. jinja2 template implementations vary. markdown renderers implementation vary. python keeps adding language features.

https://monoskop.org/images/b/be/Nelson_Ted_Literary_Machines_c1987_chs_0-1.pdf https://pure.au.dk/ws/files/173226224/PPIG_2019_camera_ready.pdf

there is massive adoption of markdown as a tool for programmers. folks can use markdown and jekyll templating languages on github pages. github profiles are written in markdown. markdown is a gateway drug to html, javascript, and css. ↩
jupyter community calls are a monthly series of events to showcase what you are doing in and with jupyter. please consider submitting to or hosting a future community call sometime ↩
many folks working on computational interfaces are inspired by Bret Victor's inventing on principle ↩

pidgy literate computing¤

the pidgy polyglot metalanguage¤