notebook toolbar

Activate

index execution_count cell_type toolbar started_at completed_at source loc metadata outputs

index	execution_count	cell_type	source	loc	metadata	outputs
1	In	markdown	# formatting markdown it tokens as python `doctests` # formatting markdown it tokens as python `doctests`	`{'data': {'text/html': 'formatting markdown it tokens as python \n'}}`	1	formatting markdown it tokens as python
2	In	markdown	`midgy` is a tool i've been crafting that translates markdown to valid python. this concept might sound perculiar from a programming perspective, but it was designed as a [literate programming] tool. to avoid feature creep, `midgy` tries to stick fairly close to the commonspec when tokenizing markdown. `midgy` adds a _doctest_ token to the parser. we make this addition because `doctest` is a _[literate programming]_ considered in the core python language. in this document, we convert the `markdown_it` tokens into valid `doctest.DocTest` runners. `midgy` is a tool i've been crafting that translates markdown to valid python. this concept might sound perculiar from a programming perspective, but it was designed as a [literate programming] tool. to avoid feature creep, `midgy` tries to stick fairly close to the commonspec when tokenizing markdown. `midgy` adds a _doctest_ token to the parser. we make this addition because `doctest` is a _[literate programming]_ considered in the core python language. in this document, we convert the `markdown_it` tokens into valid `doctest.DocTest` runners.	{'data': {'text/html': "midgy is a tool i've been crafting that translates markdown to valid python.\nthis concept might sound perculiar from a programming perspective,\nbut it was designed as a [literate programming] tool. \nto avoid feature creep, midgy tries to stick fairly close to the commonspec\nwhen tokenizing markdown. midgy adds a doctest token to the parser.\nwe make this addition because doctest is a [literate programming]\nconsidered in the core python language. \nin this document, we convert the markdown_it tokens into valid doctest.DocTest runners.\n"}}	10	`midgy` is a tool i've been crafting that translates markdown to valid python. this concept might sound perculiar from a programming perspective, but it was designed as a [literate programming] tool. to avoid feature creep, `midgy` tries to stick fairly close to the commonspec when tokenizing markdown. `midgy` adds a doctest token to the parser. we make this addition because `doctest` is a [literate programming] considered in the core python language. in this document, we convert the `markdown_it` tokens into valid `doctest.DocTest` runners.
3	In 1	code	import midgy, doctest, unittest, typing from textwrap import dedent `import midgy, doctest, unittest, typing from textwrap import dedent`	`{}`	2	0 outputs.
4	In	markdown	write some sample doctests to parse `write some sample doctests to parse`	`{'data': {'text/html': 'write some sample doctests to parse\n'}}`	1	write some sample doctests to parse
5	In 2	code	def a_testable_function(): """ >>> range(1) range(0, 1) >>> assert False Traceback (most recent call last): ... AssertionError """ `def a_testable_function(): """ >>> range(1) range(0, 1) >>> assert False Traceback (most recent call last): ... AssertionError """`	`{}`	10	0 outputs.
6	In	markdown	verify that these tests pass. `doctest.testmod` runs `doctest` on the `__main__` module. verify that these tests pass. `doctest.testmod` runs `doctest` on the `__main__` module.	`{'data': {'text/html': 'verify that these tests pass. doctest.testmod runs doctest on the __main__ module.\n'}}`	1	verify that these tests pass. `doctest.testmod` runs `doctest` on the `__main__` module.
7	In 3	code	doctest.testmod(optionflags=doctest.ELLIPSIS) `doctest.testmod(optionflags=doctest.ELLIPSIS)`	`{}`	1	1 outputs. TestResults(failed=0, attempted=2)
8	In	markdown	make some `markdown_it` tokens from the doctests. make some `markdown_it` tokens from the doctests.	`{'data': {'text/html': 'make some markdown_it tokens from the doctests.\n'}}`	1	make some `markdown_it` tokens from the doctests.
9	In 4	code	(tokens := (parser := midgy.Python()).parse(a_testable_function.__doc__)) `(tokens := (parser := midgy.Python()).parse(a_testable_function.__doc__))`	`{}`	1	1 outputs. [Token(type='code_block', tag='code', nesting=0, attrs={}, map=[1, 3], level=0, children=None, content=' >>> range(1)\n range(0, 1)\n', markup='', info='', meta={'first_indent': 4, 'last_indent': 4, 'min_indent': 4, 'is_magic': False, 'is_doctest': True, 'input': [1, 2], 'output': [2, 3]}, block=True, hidden=False), Token(type='code_block', tag='code', nesting=0, attrs={}, map=[4, 8], level=0, children=None, content=' >>> assert False\n Traceback (most recent call last):\n ...\n AssertionError\n', markup='', info='', meta={'first_indent': 4, 'last_indent': 4, 'min_indent': 4, 'is_magic': False, 'is_doctest': True, 'input': [4, 5], 'output': [5, 8]}, block=True, hidden=False)]
10	In	markdown	`get_example_from_token` translates a markdown token to a `doctest` `get_example_from_token` translates a markdown token to a `doctest`	`{'data': {'text/html': 'get_example_from_token translates a markdown token to a doctest\n'}}`	1	`get_example_from_token` translates a markdown token to a `doctest`
11	In 5	code	def get_example_from_token(token) -> typing.Iterable[doctest.Example]: m = doctest.DocTestParser._EXAMPLE_RE.match(token.content) want = dedent(m.group("want")) exc = doctest.DocTestParser._EXCEPTION_RE.match(want) source = "".join(x.lstrip()[4:] for x in m.group("source").splitlines(1)) yield doctest.Example( source=source, want=want, lineno=token.map[0], exc_msg=exc.group("msg") if exc else None, indent=len(m.group("indent")) ) `def get_example_from_token(token) -> typing.Iterable[doctest.Example]: m = doctest.DocTestParser._EXAMPLE_RE.match(token.content) want = dedent(m.group("want")) exc = doctest.DocTestParser._EXCEPTION_RE.match(want) source = "".join(x.lstrip()[4:] for x in m.group("source").splitlines(1)) yield doctest.Example( source=source, want=want, lineno=token.map[0], exc_msg=exc.group("msg") if exc else None, indent=len(m.group("indent")) )`	`{}`	9	0 outputs.
12	In	markdown	`get_examples_from_tokens` aggregates the `doctest.Example`s `get_examples_from_tokens` aggregates the `doctest.Example`s	`{'data': {'text/html': 'get_examples_from_tokens aggregates the doctest.Examples\n'}}`	1	`get_examples_from_tokens` aggregates the `doctest.Example` s
13	In 6	code	def get_examples_from_tokens(tokens) -> typing.Iterable[doctest.Example]: for token in tokens: if token.meta.get("is_doctest"): yield from get_example_from_token(token) `def get_examples_from_tokens(tokens) -> typing.Iterable[doctest.Example]: for token in tokens: if token.meta.get("is_doctest"): yield from get_example_from_token(token)`	`{}`	5	0 outputs.
14	In	markdown	finally we generate a `unittest.TestSuite` finally we generate a `unittest.TestSuite`	`{'data': {'text/html': 'finally we generate a unittest.TestSuite\n'}}`	1	finally we generate a `unittest.TestSuite`
15	In 7	code	def get_suite_from_tokens(tokens) -> unittest.TestSuite: suite = unittest.TestSuite() for example in get_examples_from_tokens(tokens): suite.addTest(doctest.DocTestCase( doctest.DocTest([example], globals(), __name__, None, example.lineno, None), optionflags=doctest.ELLIPSIS )) return suite `def get_suite_from_tokens(tokens) -> unittest.TestSuite: suite = unittest.TestSuite() for example in get_examples_from_tokens(tokens): suite.addTest(doctest.DocTestCase( doctest.DocTest([example], globals(), __name__, None, example.lineno, None), optionflags=doctest.ELLIPSIS )) return suite`	`{}`	8	0 outputs.
16	In 8	code	def run_suite(suite=(suite:=get_suite_from_tokens(tokens))) -> unittest.TestResult: suite.run(result:= unittest.TestResult()) return result `def run_suite(suite=(suite:=get_suite_from_tokens(tokens))) -> unittest.TestResult: suite.run(result:= unittest.TestResult()) return result`	`{}`	3	0 outputs.
17	In	markdown	run our generated `doctest` suite to verify that it doesn't fail. run our generated `doctest` suite to verify that it doesn't fail.	`{'data': {'text/html': "run our generated doctest suite to verify that it doesn't fail.\n"}}`	1	run our generated `doctest` suite to verify that it doesn't fail.
18	In 9	code	run_suite(suite) `run_suite(suite)`	`{}`	1	1 outputs. <unittest.result.TestResult run=2 errors=0 failures=0>

markdown

# formatting markdown it tokens as python `doctests`

# formatting markdown it tokens as python `doctests`

formatting markdown it tokens as python

markdown

`midgy` is a tool i've been crafting that translates markdown to valid python.
this concept might sound perculiar from a programming perspective,
but it was designed as a [literate programming] tool.

to avoid feature creep, `midgy` tries to stick fairly close to the commonspec
when tokenizing markdown. `midgy` adds a _doctest_ token to the parser.
we make this addition because `doctest` is a _[literate programming]_
considered in the core python language.

in this document, we convert the `markdown_it` tokens into valid `doctest.DocTest` runners.

`midgy` is a tool i've been crafting that translates markdown to valid python.
this concept might sound perculiar from a programming perspective,
but it was designed as a [literate programming] tool.

to avoid feature creep, `midgy` tries to stick fairly close to the commonspec
when tokenizing markdown. `midgy` adds a _doctest_ token to the parser.
we make this addition because `doctest` is a _[literate programming]_
considered in the core python language.

in this document, we convert the `markdown_it` tokens into valid `doctest.DocTest` runners.


        {'data': {'text/html': "midgy is a tool i've been crafting that translates markdown to valid python.\nthis concept might sound perculiar from a programming perspective,\nbut it was designed as a [literate programming] tool.
\nto avoid feature creep, midgy tries to stick fairly close to the commonspec\nwhen tokenizing markdown. midgy adds a doctest token to the parser.\nwe make this addition because doctest is a [literate programming]\nconsidered in the core python language.
\nin this document, we convert the markdown_it tokens into valid doctest.DocTest runners.\n"}}

midgy is a tool i've been crafting that translates markdown to valid python. this concept might sound perculiar from a programming perspective, but it was designed as a [literate programming] tool.

to avoid feature creep, midgy tries to stick fairly close to the commonspec when tokenizing markdown. midgy adds a doctest token to the parser. we make this addition because doctest is a [literate programming] considered in the core python language.

in this document, we convert the markdown_it tokens into valid doctest.DocTest runners.

In 1

code

    import midgy, doctest, unittest, typing
    from textwrap import dedent

    import midgy, doctest, unittest, typing
    from textwrap import dedent

markdown

write some sample doctests to parse

write some sample doctests to parse

write some sample doctests to parse

In 2

code

    def a_testable_function():
        """
        >>> range(1)
        range(0, 1)
        
        >>> assert False
        Traceback (most recent call last):
        ...
        AssertionError
        """

    def a_testable_function():
        """
        >>> range(1)
        range(0, 1)
        
        >>> assert False
        Traceback (most recent call last):
        ...
        AssertionError
        """

markdown

verify that these tests pass. `doctest.testmod` runs `doctest` on the `__main__` module.

verify that these tests pass. `doctest.testmod` runs `doctest` on the `__main__` module.


        {'data': {'text/html': 'verify that these tests pass. doctest.testmod runs doctest on the __main__ module.\n'}}

verify that these tests pass. doctest.testmod runs doctest on the __main__ module.

In 3

code

    doctest.testmod(optionflags=doctest.ELLIPSIS)

    doctest.testmod(optionflags=doctest.ELLIPSIS)

TestResults(failed=0, attempted=2)

markdown

make some `markdown_it` tokens from the doctests.

make some `markdown_it` tokens from the doctests.

make some markdown_it tokens from the doctests.

In 4

code

    (tokens := (parser := midgy.Python()).parse(a_testable_function.__doc__))

    (tokens := (parser := midgy.Python()).parse(a_testable_function.__doc__))

[Token(type='code_block', tag='code', nesting=0, attrs={}, map=[1, 3], level=0, children=None, content='    >>> range(1)\n    range(0, 1)\n', markup='', info='', meta={'first_indent': 4, 'last_indent': 4, 'min_indent': 4, 'is_magic': False, 'is_doctest': True, 'input': [1, 2], 'output': [2, 3]}, block=True, hidden=False),
 Token(type='code_block', tag='code', nesting=0, attrs={}, map=[4, 8], level=0, children=None, content='    >>> assert False\n    Traceback (most recent call last):\n    ...\n    AssertionError\n', markup='', info='', meta={'first_indent': 4, 'last_indent': 4, 'min_indent': 4, 'is_magic': False, 'is_doctest': True, 'input': [4, 5], 'output': [5, 8]}, block=True, hidden=False)]

markdown

`get_example_from_token` translates a markdown token to a `doctest`

`get_example_from_token` translates a markdown token to a `doctest`

get_example_from_token translates a markdown token to a doctest

In 5

code

    def get_example_from_token(token) -> typing.Iterable[doctest.Example]:
        m = doctest.DocTestParser._EXAMPLE_RE.match(token.content)
        want = dedent(m.group("want"))
        exc = doctest.DocTestParser._EXCEPTION_RE.match(want)
        source = "".join(x.lstrip()[4:] for x in m.group("source").splitlines(1))
        yield doctest.Example(
            source=source, want=want, lineno=token.map[0],
            exc_msg=exc.group("msg") if exc else None, indent=len(m.group("indent"))
        )

    def get_example_from_token(token) -> typing.Iterable[doctest.Example]:
        m = doctest.DocTestParser._EXAMPLE_RE.match(token.content)
        want = dedent(m.group("want"))
        exc = doctest.DocTestParser._EXCEPTION_RE.match(want)
        source = "".join(x.lstrip()[4:] for x in m.group("source").splitlines(1))
        yield doctest.Example(
            source=source, want=want, lineno=token.map[0],
            exc_msg=exc.group("msg") if exc else None, indent=len(m.group("indent"))
        )

markdown

`get_examples_from_tokens` aggregates the `doctest.Example`s

`get_examples_from_tokens` aggregates the `doctest.Example`s

get_examples_from_tokens aggregates the doctest.Example s

In 6

code

    def get_examples_from_tokens(tokens) -> typing.Iterable[doctest.Example]:
        for token in tokens:
            if token.meta.get("is_doctest"):
                yield from get_example_from_token(token)

    def get_examples_from_tokens(tokens) -> typing.Iterable[doctest.Example]:
        for token in tokens:
            if token.meta.get("is_doctest"):
                yield from get_example_from_token(token)

markdown

finally we generate a `unittest.TestSuite`

finally we generate a `unittest.TestSuite`

finally we generate a unittest.TestSuite

In 7

code

    def get_suite_from_tokens(tokens) -> unittest.TestSuite:
        suite = unittest.TestSuite()            
        for example in get_examples_from_tokens(tokens):
            suite.addTest(doctest.DocTestCase(
                doctest.DocTest([example], globals(), __name__, None, example.lineno, None),
                optionflags=doctest.ELLIPSIS
            ))
        return suite

    def get_suite_from_tokens(tokens) -> unittest.TestSuite:
        suite = unittest.TestSuite()            
        for example in get_examples_from_tokens(tokens):
            suite.addTest(doctest.DocTestCase(
                doctest.DocTest([example], globals(), __name__, None, example.lineno, None),
                optionflags=doctest.ELLIPSIS
            ))
        return suite

In 8

code

    def run_suite(suite=(suite:=get_suite_from_tokens(tokens))) -> unittest.TestResult:
        suite.run(result:= unittest.TestResult())
        return result

    def run_suite(suite=(suite:=get_suite_from_tokens(tokens))) -> unittest.TestResult:
        suite.run(result:= unittest.TestResult())
        return result

markdown

run our generated `doctest` suite to verify that it doesn't fail.

run our generated `doctest` suite to verify that it doesn't fail.

run our generated doctest suite to verify that it doesn't fail.

In 9

code

    run_suite(suite)

    run_suite(suite)

<unittest.result.TestResult run=2 errors=0 failures=0>